Skip to content
Skip to main content

About this free course

Become an OU student

Share this free course

Global challenges in practice: designing a development intervention
Global challenges in practice: designing a development intervention

Start this free course now. Just create an account and sign in. Enrol and complete the course for a free statement of participation or digital badge if available.

1 Differences in understanding of impact

In the previous section, you explored what the term ‘impact’ means to you. Now look at the definitions of impact chosen by Simon Hearn and Anne Buffardi in their 2016 Methods Lab opinion piece, What is impact?.

Such definitions range from the very narrow to the very broad. At one extreme is the tightly-bounded and technical definition of the World Bank which states that:

The difference in the indicator of interest (Y) with the intervention (Y1) and without the intervention (Y0). That is, impact = Y1 - Y0 .

(World Bank, as cited by Hearn and Buffardi, 2016, pp. 7–8)

This usually implies measuring something before and after an intervention. With impact being broadly defined as the difference between the two measurements. This requires selection of what is to be measured and then finding a means of measuring it.

At the other extreme is the broad all-encompassing definition used by the Organization for Economic Cooperation and Development – Development Assistance Committee (OECD-DAC), which defines impacts as:

Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.

(OECD, 2002, p. 24)

This latter definition allows for much greater leeway in what is to be used in determining impact but in a sense has no boundaries. In practice, it could be quite a headache to implement!

If you wish, you can read the full article: Hearn and Buffardi (2016) ‘What is impact? [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)] ’.

Such approaches place determination of impact in the control of the development or donor agencies. Certainly, all decisions on the means of measurement that should be used are coming from this direction. Those who are meant to benefit from such development interventions may or may not be given a say in the what, how and why of impact determination.

Hearn and Buffardi argue that different understandings of what impact is leads to confusion and that those in the development sector should be clear about their particular interpretation.

In the next activity, you will imagine you are involved in a development programme deciding on an evaluation of impact.

Activity 2

Timing: Allow around 20 minutes

An educational programme is planned with the aim of reducing illiteracy rates amongst 16- to 18-year-old pre-school leavers in a particular community. This issue has been identified as a barrier for these young people in gaining employment.

It has been decided that evaluation of impact will focus only on the planned programme goals, that is, the reduction in illiteracy rates

Answer the following questions:

  • In what kind of situation would such an approach to evaluation of impact be reasonable?
  • What might be the strengths and weaknesses of limiting evaluation in this way?

Note your answers in the box below.

To use this interactive functionality a free OU account is required. Sign in or register.
Interactive feature not available in single page view (see it in standard view).

Discussion

The foreseen, intended impact is the reduction of illiteracy amongst a group of young people aged between 16 and 18 and the intended outcome is their improved success in finding employment.

If there exists adequate baseline information on existing illiteracy rates, a valid and practical means of measuring such rates, and knowledge of the circumstances of these young people, focusing on the planned programme goals in evaluating impact may be a reasonable approach. Monitoring can be carried out at staged intervals, for example through assessment, and changes attributed to the programme.

However, this is a somewhat simplistic approach as it does not take into account the diverse range of factors that can impact on learning and illiteracy, and that illiteracy may not be the only barrier to employment that this group will experience.

Additionally, changes in the home or social circumstances of particular individuals could arise during the programme that impact adversely on their progress. Without carrying out risk assessment and putting additional monitoring in place, much of this valuable information will not be captured.

Moreover, the evaluation does not consider the local employment opportunities. Or how does this increased literacy rate affect the pre-school leavers understanding of employment opportunities and how to respond; how to complete application forms, present their experience and skills, and prepare for the workplace.

Adaptive management has a number of advantages when monitoring and evaluation is undertaken in that it is more ‘downward-looking’, more participatory, and less bureaucratic as development actors are supported in worrying less about measurement than success on the ground through changing behaviours. Moreover, because it is focused on regular monitoring and reflection measuring outputs and outcomes is more a rolling process rather than a single snapshot. This also makes it easier when evaluating the contribution and attribution of the project because the data evidence is more current and there is a record of evolving change.