Session 6 The monitoring and evaluation of Living Labs
In this sixth session of the course we explain how we went about monitoring and evaluating our Living Labs . First, watch the following video on some key features of our approach to monitoring and evaluation and then do the reflective activity below.
In this session, we look at the approach we use to monitor and evaluate our Living Labs. This approach flowed from our wish to learn in the Living Labs, to learn between the Living Labs, and learn beyond the Living Labs.
There are different contexts and innovatory services involved when we needed to have a flexible and adaptable approach, where we could share reflections and ideas using a common conceptual framework.
In particular, we regularly reflected on three central questions, known as the three E's: Efficacy - has each Living Lab achieved its purpose? Efficiency - has each Living Lab used resources well? Effectiveness - have the Living Labs contributed to their wider purpose within AgriLink?
The answers to these questions at any point in the Living Labs' timelines help monitor and evaluate progress and shape the subsequent focus and activities of the Living Labs.
Reflective Activity 12
What do you understand by monitoring and reflexive monitoring? Write your thoughts in the box below before looking at our answer.
Our answer uses the practice abstract on Reflexive monitoring for learning about Living Labs [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)] written by Kevin Collins of The Open University:
Monitoring is a process to assess and evaluate an activity, usually while it is underway. It is often done by an ‘independent’ monitor, such that the monitoring is ‘removed’ from the experience and understanding of those being monitored. In contrast, reflexive monitoring is undertaken by those directly involved in the situation of interest to gain insight into their own learning and to use this to design improvements during the activity. AgriLink has used reflexive monitoring to learn about the roles and practices of the six Living Labs established to promote innovation in agricultural advisory systems.
The primary mechanism for reflexive monitoring is the appointment of a dedicated monitor in each Living Lab. The monitor works with the Living Lab facilitator to establish and develop reflexive monitoring of its work, focus and activities. The monitor compiles quarterly and annual monitoring reports to document and review changes to the purpose, work and direction of the Living Labs, problems and opportunities encountered, and key learning points. A central AgriLink monitoring team reviews the annual reports and synthesises learning. Subsequent discussion between Living Lab monitors and AgriLink colleagues provides further opportunity to monitor learning and reflect on the next steps for each Living Lab. This includes a process where monitors engage in peer reviews of other Living Labs and, in so doing, reflect on their own Living Lab and ways in which its work could be enhanced.
There are many tools and techniques which can be used to promote reflexive learning, such as the three Es criteria of efficacy, efficiency, effectiveness (see other AgriLink PAs); systems diagrams; structured inquiries during field visits; and co-researching.
Let us look at reflexive monitoring in a bit more detail.
Now go to the next section.