3 Distinguishing between monitoring and evaluation

Monitoring and evaluation are two terms that are often found together, be it in project or programme proposals, reports, funding documents, even in the names of specific departments or organisations. Indeed, it is tempting to assume that monitoring and evaluation are one and the same. But is it necessarily right to bundle these two terms together?

In the next activity, you will explore monitoring and evaluation further.

Activity 3

Timing: Allow around 20 minutes

In the following video Thomas Winderl, a consultant working in monitoring and evaluation in development, discusses how and why these two terms can be distinguished and why it is important to do so. Watch the video then answer the question that follows:

Download this video clip.Video player: Video 2 Monitoring evaluation

Show transcript | Hide transcript

Transcript: Video 2 Monitoring evaluation

THOMAS WINDERL:

Monitoring and evaluation– but don’t be fooled. The fact that the term ‘monitoring and evaluation’ often goes together and is called M&E, M&E, is somewhat misleading. It is true that monitoring and evaluation share similar tools and methods. While they are interrelated, monitoring and evaluation are clearly separate activities. In fact, monitoring is very different from evaluations. They are usually carried out by different people and differ in how often they are carried out. Let’s look at that in more detail.

Monitoring and evaluations– let’s first take a look at what monitoring is. In a nutshell, monitoring is like the dashboard of your car when you’re driving. It tells you how fast you go, how much petrol you have left, or maybe if one of the car’s doors has been left open. In governments and development organisations, monitoring is the regular and systematic collection, analysis, reporting, and use of information about programmes, policies, or services. Monitoring is concerned with the performance of a programme, a policy, or a service. And unlike an evaluation, it is typically conducted internally. That means monitoring is carried out by staff that works inside an organisation. And unlike evaluations, it is a continuous process.

That means it is carried out non-stop, during, and sometimes after an activity. Monitoring typically supports the management of programmes, policies, and services and helps to manage risks.

Evaluations, on the other hand, are like the occasional check-up of your car. Evaluations are a systematic and impartial assessment of expected and achieved accomplishments. Evaluations take a step back to look, as the term suggests, at the overall value of a programme, a policy, or a service. Evaluations are usually conducted externally. That means evaluations are carried out by evaluators or specialists with no link to an organisation. Having independent external evaluators should ensure a more unbiased judgement. And unlike monitoring, an evaluation is not carried out all the time but is a one-off activity. Typically, evaluations are carried out during or at the end of an activity. And evaluations are more systematic than monitoring.

Here are some typical questions an evaluation attempts to answer. Is a programme, a policy, a service, or an organisation relevant? Does it suit the priorities and policies of the target group? Is it effective? Does it achieve results? Is it efficient? Does it achieve results at reasonable costs? Does it have impact? What real difference has a programme, a policy, or a service made for beneficiaries? Is it sustainable? Will positive change continue once funding is cut? That is why evaluations tend to be broader in scope than monetary.

End transcript: Video 2 Monitoring evaluation

Download

Video 2 Monitoring evaluation

Interactive feature not available in single page view (see it in standard view).

• What is the difference between monitoring and evaluation?

Note your answer in the box below.

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

Winderl points out that these two processes take place at different stages and often by different actors. Monitoring requires ongoing, systemic collection and analysis of information in order to follow the development and progress of a project or programme and are usually carried out internally.

In contrast, evaluation tends to focus on the ‘what’ or the endpoint of the programme and is usually performed by people external to the organisation so that it is impartial and unbiased.

Delving deeper, the types of monitoring information gathered will be influenced by what it is you wish to assess – the outcomes, the process undertaken, or both. A distinction is sometimes made between outcomes-based and process-based monitoring and evaluation.

Whilst there are many different types of evaluation, results or outcomes-based evaluation is particularly focused on the ‘what’ or the endpoint of the project and so the collection of data will reflect this. To begin, baseline data is needed as you saw in Activity 2. To know that something has changed, you need to know your start point. If you are involved in a programme to improve the sanitation in a town with an overall aim of health improvement, you need to have some idea of the current situation and how it relates to your proposed endpoint. And of course, if the work is going to take place over a period of time, is taking up time and resources, and has a finite life, you will want to follow progress so that you can see if you are on track and going to meet your objectives in the allocated time.

Process-based evaluation focuses on the ‘how’ of the programme. Monitoring data will capture something of the particular path followed by you and your teams in achieving its goals. For instance, you wish to know how the management of the programme was undertaken, how staff were deployed, the challenges and opportunities that revealed themselves during execution.

As ever, nothing sits nicely in its boxes and monitoring with a process or results-based focus can merge and overlap. For instance, budgetary information is useful in deciding if the programme has delivered on its goals and according to what was planned (effectiveness), as well in assessing its efficiency in doing so. The usefulness of differentiating between outcomes and process is that it ensures you consider both the implementation and delivery dimensions of a programme, and so enables you to make informed decisions as to the type of monitoring data that you wish to gather.

This division of outcome and process-based monitoring and evaluation is in itself a rather crude simplification and does not reflect the diversity of approaches taken to evaluation practice and that a combination of approaches is likely to be used.