Theories in Technology Evaluation
Theories in Technology Evaluation

Start this free course now. Just create an account and sign in. Enrol and complete the course for a free statement of participation or digital badge if available.

Free course

Theories in Technology Evaluation

4 Ethics, validity, credibility and trust

My aim in this section is to introduce and discuss in broad terms the ways in which the ulterior motives of some of those who commission, conduct and use evaluation – and the tensions and pressures this creates – can be addressed.

One of the most significant of the various topics of debate and disagreement between the advocates of quantitative and qualitative methodologies during the ‘paradigm wars’ was the issue of the validity and credibility of the methods used for data collection. Advocates of the use of quantitative methods, for example, did, and to an extent still do, argue that because qualitative methods are by definition subjective they are less valid than their, more ‘scientific’ approaches. However, the practitioners of both qualitative and mixed-methods approaches point out that ‘All data sets are biased by decisions about what to observe, how to categorise, what to record, and how to interpret’ (Bamberger et al., 2006, p. 145). Furthermore, if we accept the view that ‘all human ways of knowing are necessarily subjective [and that] subjective minds are all evaluators have’ (op. cit.) it follows that rather than pursue unwinnable arguments about methods, trustworthiness should become the principle that underpins evaluation (Guba and Lincoln, 1989, cited in Bamberger et al., 2006).

Putting the emphasis on trust fundamentally affects evaluation in many ways. It means evaluators:

  • accepting that there are arguments for and against the many different theories, paradigms, models, methods, and values and beliefs that can enter into the arena of any evaluation
  • recognising the possible shortcomings or challenges these multiple meanings might create
  • putting their efforts into devising strategies to address these multiple – and possibly competing – meanings.

Addressing these issues might include ensuring that evaluators/data collectors have had adequate training in, and experience of, the methods they intend to use. But it may also include incorporating mechanisms that allow evaluators to be open and transparent about the beliefs and values they bring to an evaluation, and, where necessary, the assumptions and judgements on which they base their findings. In short, establishing trustworthiness means taking action to ensure as fully as possible that the stakeholders of an evaluation can believe in the process, the participants and the outcome.

As you will have observed from your research on Activity 10, all of the actions I’ve noted above, and many more, form the basis of the good practice guidelines and standards that the various national evaluation bodies/associations promote. They therefore confirm the centrality of trust to evaluation.

T887_2

Take your learning further371

Making the decision to study can be a big step, which is why you'll want a trusted University. The Open University has 50 years’ experience delivering flexible learning and 170,000 students are studying with us right now. Take a look at all Open University courses372.

If you are new to university level study, we offer two introductory routes to our qualifications. Find out Where to take your learning next?373 You could either choose to start with an Access courses374or an open box module, which allows you to count your previous learning towards an Open University qualification.

Not ready for University study then browse over 1000 free courses on OpenLearn375 and sign up to our newsletter376 to hear about new free courses as they are released.

Every year, thousands of students decide to study with The Open University. With over 120 qualifications, we’ve got the right course for you.

Request an Open University prospectus371