Evaluation is not always appropriate and ideal evaluations are rarely possible. Evaluations are likely to be particularly limited for smaller projects, where a huge expensive measurement exercise would end dwarfing the costs of the project.

In determining whether and how to evaluate a project, the following constraints should be kept in mind:

• Cost: International development agencies, funders and organisations typically allocate approximately 10-20% of each project’s budget to monitoring and evaluation. While this might seem like a lot, evaluation can enable much larger gains over time by ensuring that the most effective projects are identified and scaled up while the most ineffective (and any harmful!) projects are abandoned.
  

• Burden on implementation: This is a particularly serious concern for OAD projects, which are often delivered by volunteers in their spare time. In the ideal case, evaluation design and data collection would be conducted by a separate (independent) team. This has the advantage of reducing the burden on those implementing the project; and reducing the risk of conflicting interests (since those who designed or implement a project are likely to be invested in its success!). In reality, it is often impossible to budget for or engage independent evaluators.
  

• Restricting outcomes: When outcomes are difficult to measure, there is a risk that attention will be paid only to what is measurable. If measures are poorly designed or inadequate, this can create problems; even when measures are good, they must necessarily focus on a narrow selection of outcomes. There may be others of interest that are then ignored. For example, the selection of science & research indicators may influence policymakers to focus on specific areas of science output that are easily (or currently) measured rather than on those that could be done; or that have more restricted (local) relevance.

• Political/personal/emotional: Constraints on evaluation means that (if an evaluation is appropriate) monitoring and evaluation activities should be embedded in the project and data collection burdens minimized. This can mean streamlining reporting and auditing requirements; limiting the number of outcomes measured; and limiting the kind of data collected (e.g. level of detail about participants). Technological solutions should also be considered wherever possible.

 < BACK     NEXT >