4 Ethics, validity, credibility and trust
My aim in this section is to introduce and discuss in broad terms the ways in which the ulterior motives of some of those who commission, conduct and use evaluation – and the tensions and pressures this creates – can be addressed.
One of the most significant of the various topics of debate and disagreement between the advocates of quantitative and qualitative methodologies during the ‘paradigm wars’ was the issue of the validity and credibility of the methods used for data collection. Advocates of the use of quantitative methods, for example, did, and to an extent still do, argue that because qualitative methods are by definition subjective they are less valid than their, more ‘scientific’ approaches. However, the practitioners of both qualitative and mixed-methods approaches point out that ‘All data sets are biased by decisions about what to observe, how to categorise, what to record, and how to interpret’ (Bamberger et al., 2006, p. 145). Furthermore, if we accept the view that ‘all human ways of knowing are necessarily subjective [and that] subjective minds are all evaluators have’ (op. cit.) it follows that rather than pursue unwinnable arguments about methods, trustworthiness should become the principle that underpins evaluation (Guba and Lincoln, 1989, cited in Bamberger et al., 2006).
Putting the emphasis on trust fundamentally affects evaluation in many ways. It means evaluators:
- accepting that there are arguments for and against the many different theories, paradigms, models, methods, and values and beliefs that can enter into the arena of any evaluation
- recognising the possible shortcomings or challenges these multiple meanings might create
- putting their efforts into devising strategies to address these multiple – and possibly competing – meanings.
Addressing these issues might include ensuring that evaluators/data collectors have had adequate training in, and experience of, the methods they intend to use. But it may also include incorporating mechanisms that allow evaluators to be open and transparent about the beliefs and values they bring to an evaluation, and, where necessary, the assumptions and judgements on which they base their findings. In short, establishing trustworthiness means taking action to ensure as fully as possible that the stakeholders of an evaluation can believe in the process, the participants and the outcome.
As you will have observed from your research on Activity 10, all of the actions I’ve noted above, and many more, form the basis of the good practice guidelines and standards that the various national evaluation bodies/associations promote. They therefore confirm the centrality of trust to evaluation.