Summary of Session 6
Living Labs are increasingly used as a concept and process to co-research with stakeholders in context.
In AgriLink, the aim has been to use six Living Labs involving researchers, farmers and public and private advisors to learn about, monitor, evaluate and promote innovation in agricultural advisory and support systems. But working in context relevant ways means that each Living Lab is unique in its form, focus, stakeholders, composition, methods, practices and outputs. This uniqueness presents problems when monitoring and evaluating Living Labs, especially determining meaningful indicators and measures of progress which are generalisable but also capture and reflect diversity and context.
Rather than focusing on developing lists of individual indicators, we have drawn on systems ideas and concepts to understand Living Labs as learning systems. In particular we have used the established three Es for monitoring and evaluating systems: efficacy, efficiency and effectiveness.
Each criterion is expressed as a question:
| Efficacy – has the Living Lab achieved its purpose (as defined by the stakeholders)? |
| Efficiency – has the Living Lab used resources well (including time, energy and enthusiasm)? |
| Effectiveness – has the Living Lab contributed to its wider purpose within AgriLink? |
Used in our quarterly and annual monitoring and evaluation reports, the three Es allowed each of the Living Lab monitors to document context-specific answers and also monitor and evaluate learning about the aims, processes and activities of their and other Living Labs.
It also provided the wider AgriLink consortium with an understanding of the potential for Living Labs as learning systems for innovative practice in agricultural advisory services.
References for Session 6
Bootcamp Bootleg D.School, (2010) Modes of design thinking. D.School, University of Stanford. Available at: https://dschool.stanford.edu/s/METHODCARDS-v3-slim.pdf (Accessed: 2 April 2018).
Checkland, P. and Scholes, P. (1999) ‘Soft Systems Methodology in Action: Including a 30-Year Retrospective’, Journal of the Operational Research Society, 51(5):648.
Fook, J. (2002) Social Work: Critical Theory and Practice. London, Sage.
Ison, R. (2010) Systems Practice: How to Act in a Climate-Change World. Springer, London.
Van Mierlo, B, Arkesteijn, M, and Leeuwis, C. (2010) ‘Enhancing the reflexivity of system innovation projects with system analyses’, American Journal of Evaluation, 31(2): 143–61.
Go to Session 7: Living with a Living Lab: some dos and don’ts [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)]
Monitoring and evaluation as an iterative process of interventions and learning