Theories in Technology Evaluation
Theories in Technology Evaluation

This free course is available to start right now. Review the full course description and key learning outcomes and create an account and enrol if you want a free statement of participation.

Free course

Theories in Technology Evaluation

3.5 The ‘hidden’ functions of evaluation

I want to conclude this section by briefly outlining a further dimension of evaluation that is, in actual fact, implicit in the discussion above: evaluation can be used for purposes other than those that are stated, or that many people would regard as legitimate or ethical. The majority of examples of this are concerned with decision-making (or non-decision-making, as the case may be) and as such serve to further highlight the political dimensions of evaluation. Of course, in some cases evaluators will be party to these uses and abuses of evaluation. As Weiss points out, however, evaluators usually start from the premise that the people who commission an evaluation want genuine answers about the development, operation, implementation, outcome and impact of a development, project or programme. The reality may be somewhat different:

When this is not the case, she may in her naivete´ become a pawn in an intraorganisational power struggle, a means of delaying action, or a rallying point for one ideology or another. Some evaluators have found only after their study was done that they had unwittingly played a role in a larger political game. They found that nobody was particularly interested in learning their results, but only in using them (or any quotable piece of them) as ammunition to destroy or justify.

(Weiss, 1998, p. 23)

The term pseudo-evaluation has been used to describe this process, which is what emerges ‘when evaluators capitulate’ to the demands of sponsors for an evaluation with a particular outcome that is then suitable for a particular ‘hidden’ purpose (Bamberger et al., 2006).

Weiss (1998) provides four examples of what she refers to as ‘evaluation as subterfuge’, but which can also be described as the hidden agendas for evaluation:

  • Postponement: using evaluation to delay a decision.
  • Ducking responsibility: using evaluation to ‘make’ a decision that would otherwise attract criticism or hostility to a group/individual (e.g. the management or owners of an organisation, or a government).
  • Window dressing: using evaluation to provide legitimacy for a change in a development, project or programme where the decision to make that change had already been made but not made public.
  • Public relations: evaluation for the purpose of ‘self-glorification’. Here there may be nothing at fault with the design and conduct of the evaluation, or of its findings, but the results are used to make a successful development or project, and the people associated with it, ‘more visible’.

I would guess that quite a number of you have been either actively or passively involved in, or affected by, one or more of these uses of evaluation. I can certainly vouch for the use of evaluation for all of these purposes in government and public administration, and a good deal in academia too. That said, the commercial sector is not averse to their use either.

Any discussion of this topic should not avoid noting another related issue that can often face evaluators: values can clash with interests, particularly in the short term. In other words, individually or collectively (if working in a team) we may subscribe to the view that evaluation should be designed and used for legitimate and ethical purposes, and conducted accordingly (e.g. in accordance with the guidelines produced by the various national evaluation associations). However, an occasion arises where to do so would run counter to personal or group interests. Where, for example, negative findings from one evaluation could jeopardise future contracts with an organisation that regularly commissions evaluations; or where a well-designed and rigorously conducted evaluation could reflect badly on colleagues who work for another part of the same organisation and who had been instrumental in the design and/or implementation of the technology or project/programme being evaluated. It can – and in some circles frequently is – argued that situations such as these require ‘realpolitik’: an approach based on practical rather than moral or ideological considerations. In response to this I would support Weiss’s observation that:

Evaluation, then, is a rational enterprise often undertaken for nonrational, or at least non-informational reasons ... the important point is that such motives have consequences for the evaluation that can be serious and bleak.

(Weiss, 1998, p. 23)

Activity 10 Addressing the hidden functions of evaluation

Using information available from the European [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)] , UK or American Evaluation Association, identify one term or condition that you believe would offer an evaluator protection from becoming a pseudo evaluator and therefore ought to be included in a set of guidelines.

You might like to add your contribution, including justification of your choice, to the Comments section below.

T887_2

Take your learning further

Making the decision to study can be a big step, which is why you'll want a trusted University. The Open University has over 40 years’ experience delivering flexible learning and 170,000 students are studying with us right now. Take a look at all Open University courses.

If you are new to university level study, find out more about the types of qualifications we offer, including our entry level Access courses and Certificates.

Not ready for University study then browse over 900 free courses on OpenLearn and sign up to our newsletter to hear about new free courses as they are released.

Every year, thousands of students decide to study with The Open University. With over 120 qualifications, we’ve got the right course for you.

Request an Open University prospectus