Efficacy, efficiency and effectiveness

Efficacy

Assessing the extent to which the Living Lab achieved their purpose must be qualified by several elements including the changes during the life of the Living Lab as to their purpose and focus, the ongoing nature of some of the Living Lab, and the interruptions to the later activities of the Living Lab in 2020 due to Covid-19. However, taking these elements into account, for the most part the Living Labs did achieve their main goals, even if these changed and developed over the course of the Living Lab.

Overall, the efficacy of the Living Lab was maximised when deployed in the right context in situations of some complexity with a degree of stakeholder interest, a sense of urgency to address a problem situation and a willingness to review existing practices. All of the Living Labs were able to bring farmers, advisors and experts together to identify needs and explore possible improvements to advisory services. From the participants’ point of view, by engaging in the Living Lab they experienced (to greater or lesser extent depending on their own preferences) the opportunity and realities of participating in framing issues and determining possible actions in more collaborative ways with other stakeholders. This has opened up discussions and new ideas for advisory services.

However, as open innovation tools, the efficacy of Living Lab is reduced when used for more defined, small scale initiatives which are largely incontrovertible among the stakeholder and/or user community. In these instances, a Living Lab is too significant and an unnecessary undertaking simply to ‘agree’ on technical aspects.

Efficiency

The efficiency of Living Labs (did they use resources, e.g. time, staff, budgets, well?) resulted in an evaluation dilemma. Many of the Living Labs required up-front investment in resources, especially time to develop relationships and understandings. This investment was often more than expected and sometimes experienced by the researchers and others as ‘delay’. This is perhaps inevitable as the Living Labs were focusing on surfacing understanding of complex situations from diverse perspectives and developing trusting relationships.

An additional factor contributing to the ‘time consuming’ nature of the Living Labs was the common experience of the limited time availability of farmers and advisors to participate in the Living Lab, especially at planting and harvesting. This aspect of efficiency is a significant consideration: the AgriLink Living Labs were competing with many other demands on farmers’ time. To be attractive, the Living Labs had to offer some added value and benefit for farmers to warrant their participation and adjust to the farmers’ schedules which also introduce scope for ‘delay’.

In some cases, the resource use was reduced by cooperation with other projects, though this form of cooperation also caused further delay in at least one Living Lab.

Nonetheless, the monitoring and evaluation revealed that, in retrospect, the degree of early investment, in particular, proved to be essential for the later success (or otherwise) of Living Labs. Even so, the use of Living Labs for largely technical discussions and technical issues was less efficient and could perhaps have been progressed using a ‘lighter’ version of a Living Lab or some other process.

It is important to remember that for the purposes of AgriLink, as a research project, the Living Labs were fully invested in and also included a facilitator and a monitor in order to determine their overall usefulness as innovations for AKIS. In more commercial, advisory contexts, the efficiencies of a Living Lab would require careful scrutiny in relation to the nature of the situation and issue. Sufficient, longer term funding (if needed) of a Living Lab also remains an important consideration and has considerable bearing on efficiency if the work of a Living Lab can only be supported for a limited time and ends before outputs and outcomes are evident.

Effectiveness

The third criterion is focused on whether a Living Lab has contributed to its higher purpose within AgriLink. This is interpreted as whether the Living Labs improved innovation support services that contributed to learning and innovation for more sustainable agriculture. Although this was a difficult criterion to assess because of different perspectives and time lags (and disruptions due to Covid-19), the commentaries and reflections by the AgriLink Living Lab researchers suggested there were some significant successes.

Irrespective of their contributions to innovations in advisory services and sustainable agriculture in their specific localities, all of the Living Labs have provided insight and lessons on applying Living Lab methodologies. In this sense, their effectiveness, even where specific outcomes are limited, can be measured in terms of their contribution to understanding Living Labs as innovations spaces in advisory services. Consistent with the research emphasis of AgriLink, all of the Living Labs have helped determine how, when and why a Living Lab can be used to improve agriculture advisory services.