1.5 Whose theories?
To conclude this section and provide a logical link to the next, I want to return briefly to the discussion of which theories (of and in evaluation) can, or do, take precedence in evaluation. This can become a significant issue for evaluations and evaluators, leading to tension and conflict.
In the initial stages of an evaluation it is highly likely that the theories and paradigms that dominate will be some combination of those of the evaluators, the commissioners/funders/clients of the evaluation and/or the owners of the development to be evaluated. There is no inherent reason why these might differ, particularly where acceptance of certain theories is widespread, or where evaluations include only a limited range of stakeholders. However, in fields of technology assessment and evaluation where user-based or user-centred approaches have become increasingly popular, and participatory approaches to evaluation have become the norm, there is likely to be increasing diversity of theories and beliefs in the role and purpose of evaluation and of the theories in evaluation.
Evaluators who have previous experience of theory-based evaluation may have become skilled at making explicit their own theoretical assumptions and beliefs. However, it is likely that for many stakeholders these assumptions and beliefs may remain implicit. They might perhaps only surface when an evaluation has been completed and the findings are not what certain stakeholders assumed they would be. To address these issues, the theories and models against which stakeholders compare both the evaluation itself and what is being evaluated need to be surfaced (Weiss, 1998; Chen, 2005; Bamberger et al., 2006).
The advocates of theory-based evaluation, and the critics of it, both point out that helping stakeholders surface and/or develop their theories is likely to prove difficult and time-consuming, and it may create obstacles to an evaluation by, for example, increasing the complexity of the endeavour. As we shall see, these are arguments that can be, and often are, levelled at the general practice of involving stakeholders, particularly where this amounts to active participation. That said, there are also costs associated with limiting the scope of an evaluation’s theory base. For example, while restricting theoretical input to experts might increase the ‘scientific’ credibility and value of an evaluation, it would do little or nothing for its credibility and acceptance amongst a wider population of stakeholders. Once again, context is all-important, except that now the context becomes the one in which the evaluation is to be used rather than where it was conducted.
I am aware, of course, that there are practitioners and scholars of evaluation who favour ignoring human factors and concentrating instead solely on the intrinsic merits of a technology, project or programme. Clearly, if you are evaluating an inanimate entity, such as a technology, you do not need to be concerned with its views or feelings as it obviously has none. It is simply possible to base an evaluation on the intrinsic merits of the entity. However, given that recognition of context and contextual factors is a fundamental feature of this unit’s approach to technology evaluation, I would argue that technological objects are always part of wider systems that inevitably include animate objects (i.e. humans and other living things). Concentrating an evaluation only on the intrinsic merits of a technology would therefore only be acceptable where it is clearly beyond doubt that the context is such that the technology has no potential for impacting on anything other than other inanimate objects (although even here they might then impact on animate objects). Consequently, the issue of whose theories, beliefs and values feature in an evaluation, and how they are dealt with, is very real and should not be ignored.