5.4 Monitoring and evaluation

A monitoring and evaluation plan, featuring the appropriate instruments, should form an integral part of any implementation plan to ensure that the planned activities are carried out and the targets achieved. Resources permitting, one department or unit (for example a dedicated monitoring and evaluation unit, the unit responsible for the EMIS or TMIS, or a cross- departmental team) should be entrusted with periodic monitoring to ensure faithful execution of the plan. Where the education authorities’ human or financial resources are limited (for example, in resource-poor countries, small states or decentralized authorities), at least one member of the plan’s design unit should have such responsibilities.

Monitoring activities may be broken down into:

• Periodic monitoring through desk or field reviews, analysis of implementation records and activity logs, ongoing data gathering and structured meetings within and between implementing departments/agencies. This type of monitoring is necessary to assess progress towards the targets and benchmarks (milestones) listed in the implementation plan, identify constraints and generate solutions to problems as they emerge. Annual work plans and periodic reports may be used to help structure these exercises (Kusek and Rist, 2004: 97–98). Trinidad and Tobago’s strategic plan for education bases its monitoring on monthly reports by each officer and all divisions/units involved in the plan (MoE/Government of the Republic of Trinidad and Tobago, 2012: 30–31). Analysis of timely information generated through EMIS/TMIS or statistical units may also be used. Papua New Guinea’s strategic plan for gender equity in education relies on monitoring based on gender-disaggregated data for students, teachers, head teachers and education officers, generated annually (DoE, Papua New Guinea, 2009: 13–25). Periodic monitoring should include decentralized bodies responsible for plan activities.
• Periodic monitoring reports generated by monitoring and evaluation specialists, based on regular reviews by the monitoring team or unit and using agreed upon guidelines and tools (for example, a questionnaire assessing achievement results or a checklist aligned with the implementation activities), are important. The reports should reach key decision-makers at the appropriate levels, to inform them in real time about constraints and recommended corrective action so as to make the necessary adjustments to the implementation plan.
• An annual review with stakeholders should be planned and held by those responsible for monitoring, to review implementation of the plan with key stakeholders – representatives of teachers, students, parents, non-governmental organisations or civil society. In addition to using the results of periodic monitoring and reports, the monitoring team conducting the annual review should have an open consultative process allowing stakeholders to share concerns as well as successes in the plan’s implementation and, critically, to indicate where activities need to be changed. This feedback is crucial to assessing implementation of the policy. The report back from the review should be honest, identifying difficulties and even failures, and not driven by the desire to please higher-level policy- and decision-makers.
• A consolidated annual performance report incorporating the relevant elements from the other monitoring exercises. This report will serve as the basic document for high-level review of achievements, shortcomings and possible improvements. The review should, in turn, serve as a pillar for a revised action plan. To be meaningful, it will need to be aligned with the action plan targets, in terms of the key human and financial resource dimensions.

Monitoring can reveal changes in terms of specific indicators, as measurements of progress or lack thereof, but it cannot necessarily indicate what is responsible for the changes – hence the need for evaluation, which can reveal what happened, why it happened and what difference it makes (Russon, 2010: 108). A mid-term or end-of-term evaluation (or both), preferably carried out by independent personnel to guarantee objectivity, is important in assessing policy implementation and to provide guidance for future implementation. A final review can evaluate impact and outcomes, relevance, cost- effectiveness and sustainability, as well as outline reasons for plan achievement or failure, in particular drawing lessons to guide policy revision and future plans (see the example of Namibia in Box 5.4).

Effective monitoring and evaluation requires clear and measurable indicators linked to the set targets. Key indicators should be:

• Limited in number and focused on the major priorities
• Presented in a results-based or outcome-oriented framework, to meaningfully measure results
• Formulated to enable an analysis of disparities or disequilibria (by gender, urban/rural, disadvantaged populations, etc.) in policy implementation
• Consistent and stable throughout the implementation cycle and, most importantly, easily understandable by all users, from top-level decision-makers to the most directly affected users, i.e. teachers and learners.

In this regard, indicators need to be agreed with and accepted by stakeholders and – if relevant to the policy’s implementation – by development partners (IIEP and GPE, 2012: 15–18; ILO, 2012: 64, 102, 133–134). Through the appropriate social dialogue mechanisms (see Chapter 4), teachers at the school level and teachers’ organizations at other levels can provide valuable input into priority indicators for learning outcomes that are measurable, meaningful and equitable in relation to classroom realities and respect the requirement for consistency over time.

While the designers of an implementation plan will invariably have quantitative indicators to guide implementation, greater reliance on qualitative information from stakeholders or independent researchers can be a valuable supplement for gauging success or failure, thereby influencing policy and strategic planning for greater impact. The approach is crucial in assessing the attitudes and behaviours of perhaps the most important actors in policy implementation – teachers – who, for many reasons related to their personal and professional backgrounds, experiences, and perceptions of their status, often become reluctant implementers of a policy into which they had little or no input during the development process (Smit, 2005; Hargreaves, 1994; Hargreaves, 1998).

In addition to the perceptions and feedback of teachers and school leaders on the plan’s implementation and overall policy objectives, monitoring and evaluation may be an opportunity to use an institutional mechanism – social dialogue – to ensure greater teacher voice in the process at the school level, if that voice is not already present at other policy stages (see Chapters 2–4) (ILO, 2012: 216-217).

Increasingly, a more bottom-up (as opposed to top-down) approach to monitoring and evaluating policy design and implementation is favoured:

• Research from China has shown a divide between teachers and authorities’ appraisal of the success of the urban-to-rural teacher mobility policy. Weaknesses were identified in the incentive system, evaluation mechanisms and the system of local government support, all of which required additional work for the policy to be successful. One of the recommendations for changes resulting from the policy analysis was to provide greater opportunity for teachers to be consulted, provide feedback and participate in policy-making (Liu, Li and Du, 2014: 78–80).
• Similarly, an assessment of Pakistan’s national education plans from 1992 to 2010 in the large province of Punjab identified a number of reasons for failures and weak implementation linked to administration. One of the recommendations for strengthening the implementation of the education plans was to make the implementation mechanism more active, responsible and accountable, using a bottom-up approach structured around greater involvement of PTAs, teachers and other stakeholders (Siddiq, Salfi and Hussain, 2011: 294).
• Finally, an analysis of teachers’ experiences with the implementation of the inclusive education policy in two districts in Ghana found that teachers had limited and often distorted understandings of the policy and the innovations required in their practice. To be successful, the policy initiation process needed to become clearer and more inclusive, to enable stakeholders to understand the purpose of and accept the policy agenda (Alhassan, 2014: 127).