Theories in Technology Evaluation
Theories in Technology Evaluation

This free course is available to start right now. Review the full course description and key learning outcomes and create an account and enrol if you want a free statement of participation.

Free course

Theories in Technology Evaluation

4 Ethics, validity, credibility and trust

My aim in this section is to introduce and discuss in broad terms the ways in which the ulterior motives of some of those who commission, conduct and use evaluation – and the tensions and pressures this creates – can be addressed.

One of the most significant of the various topics of debate and disagreement between the advocates of quantitative and qualitative methodologies during the ‘paradigm wars’ was the issue of the validity and credibility of the methods used for data collection. Advocates of the use of quantitative methods, for example, did, and to an extent still do, argue that because qualitative methods are by definition subjective they are less valid than their, more ‘scientific’ approaches. However, the practitioners of both qualitative and mixed-methods approaches point out that ‘All data sets are biased by decisions about what to observe, how to categorise, what to record, and how to interpret’ (Bamberger et al., 2006, p. 145). Furthermore, if we accept the view that ‘all human ways of knowing are necessarily subjective [and that] subjective minds are all evaluators have’ (op. cit.) it follows that rather than pursue unwinnable arguments about methods, trustworthiness should become the principle that underpins evaluation (Guba and Lincoln, 1989, cited in Bamberger et al., 2006).

Putting the emphasis on trust fundamentally affects evaluation in many ways. It means evaluators:

  • accepting that there are arguments for and against the many different theories, paradigms, models, methods, and values and beliefs that can enter into the arena of any evaluation
  • recognising the possible shortcomings or challenges these multiple meanings might create
  • putting their efforts into devising strategies to address these multiple – and possibly competing – meanings.

Addressing these issues might include ensuring that evaluators/data collectors have had adequate training in, and experience of, the methods they intend to use. But it may also include incorporating mechanisms that allow evaluators to be open and transparent about the beliefs and values they bring to an evaluation, and, where necessary, the assumptions and judgements on which they base their findings. In short, establishing trustworthiness means taking action to ensure as fully as possible that the stakeholders of an evaluation can believe in the process, the participants and the outcome.

As you will have observed from your research on Activity 10, all of the actions I’ve noted above, and many more, form the basis of the good practice guidelines and standards that the various national evaluation bodies/associations promote. They therefore confirm the centrality of trust to evaluation.

T887_2

Take your learning further

Making the decision to study can be a big step, which is why you'll want a trusted University. The Open University has over 40 years’ experience delivering flexible learning and 170,000 students are studying with us right now. Take a look at all Open University courses.

If you are new to university level study, find out more about the types of qualifications we offer, including our entry level Access courses and Certificates.

Not ready for University study then browse over 900 free courses on OpenLearn and sign up to our newsletter to hear about new free courses as they are released.

Every year, thousands of students decide to study with The Open University. With over 120 qualifications, we’ve got the right course for you.

Request an Open University prospectus