3.5 The ‘hidden’ functions of evaluation
I want to conclude this section by briefly outlining a further dimension of evaluation that is, in actual fact, implicit in the discussion above: evaluation can be used for purposes other than those that are stated, or that many people would regard as legitimate or ethical. The majority of examples of this are concerned with decision-making (or non-decision-making, as the case may be) and as such serve to further highlight the political dimensions of evaluation. Of course, in some cases evaluators will be party to these uses and abuses of evaluation. As Weiss points out, however, evaluators usually start from the premise that the people who commission an evaluation want genuine answers about the development, operation, implementation, outcome and impact of a development, project or programme. The reality may be somewhat different:
When this is not the case, she may in her naivete´ become a pawn in an intraorganisational power struggle, a means of delaying action, or a rallying point for one ideology or another. Some evaluators have found only after their study was done that they had unwittingly played a role in a larger political game. They found that nobody was particularly interested in learning their results, but only in using them (or any quotable piece of them) as ammunition to destroy or justify.
The term pseudo-evaluation has been used to describe this process, which is what emerges ‘when evaluators capitulate’ to the demands of sponsors for an evaluation with a particular outcome that is then suitable for a particular ‘hidden’ purpose (Bamberger et al., 2006).
Weiss (1998) provides four examples of what she refers to as ‘evaluation as subterfuge’, but which can also be described as the hidden agendas for evaluation:
- Postponement: using evaluation to delay a decision.
- Ducking responsibility: using evaluation to ‘make’ a decision that would otherwise attract criticism or hostility to a group/individual (e.g. the management or owners of an organisation, or a government).
- Window dressing: using evaluation to provide legitimacy for a change in a development, project or programme where the decision to make that change had already been made but not made public.
- Public relations: evaluation for the purpose of ‘self-glorification’. Here there may be nothing at fault with the design and conduct of the evaluation, or of its findings, but the results are used to make a successful development or project, and the people associated with it, ‘more visible’.
I would guess that quite a number of you have been either actively or passively involved in, or affected by, one or more of these uses of evaluation. I can certainly vouch for the use of evaluation for all of these purposes in government and public administration, and a good deal in academia too. That said, the commercial sector is not averse to their use either.
Any discussion of this topic should not avoid noting another related issue that can often face evaluators: values can clash with interests, particularly in the short term. In other words, individually or collectively (if working in a team) we may subscribe to the view that evaluation should be designed and used for legitimate and ethical purposes, and conducted accordingly (e.g. in accordance with the guidelines produced by the various national evaluation associations). However, an occasion arises where to do so would run counter to personal or group interests. Where, for example, negative findings from one evaluation could jeopardise future contracts with an organisation that regularly commissions evaluations; or where a well-designed and rigorously conducted evaluation could reflect badly on colleagues who work for another part of the same organisation and who had been instrumental in the design and/or implementation of the technology or project/programme being evaluated. It can – and in some circles frequently is – argued that situations such as these require ‘realpolitik’: an approach based on practical rather than moral or ideological considerations. In response to this I would support Weiss’s observation that:
Evaluation, then, is a rational enterprise often undertaken for nonrational, or at least non-informational reasons ... the important point is that such motives have consequences for the evaluation that can be serious and bleak.
Activity 10 Addressing the hidden functions of evaluation
Using information available from the UK or American Evaluation Association, identify one term or condition that you believe would offer an evaluator protection from becoming a pseudo evaluator and therefore ought to be included in a set of guidelines.,
You might like to add your contribution, including justification of your choice, to the Comments section below.