1 Theories, paradigms and evaluation
Theories and paradigms are interrelated and interdependent. Take the example of a recently developed military aircraft – the Eurofighter Typhoon. A variety of theories and paradigms will have informed the design of the aircraft. These will also be reflected in the various forms of evaluation applied to its component parts (i.e. its systems, weapons) or attributes (i.e. range and other capabilities) as well as to the aircraft as a whole. These evaluations will range from how various materials perform under stress, through how combinations of components function, to theories and paradigms (and assumptions and beliefs) about the nature of current and future warfare. Furthermore, when evaluating the Eurofighter against other fighters, assumptions will have been made about potential enemies, in order to identify which aircraft the Eurofighter might meet in combat. These assumptions will have been informed by theories and paradigms of the current and future state of global politics and power relations.
The Eurofighter example also demonstrates that theories and paradigms form into networks and hierarchies, which can be complex and also extremely influential in the development of technologies, and these occur at all stages of assessment and evaluation. Activity 1, towards the end of this page, provides you with an opportunity to test this out. Here, however, I want to focus on two ways (or forms) in which theories and paradigms enter into evaluation:
- theories and paradigms of evaluation
- theories and paradigms in evaluation.
The first is about the doing of evaluation. We draw on examples of theories and paradigms of evaluation when we seek to answer a series of basic questions about evaluation:
- Why evaluate?
- Is it possible to evaluate entity A in context B?
- Are there approaches and methods for data collection and analysis that are superior to others?
These questions form the basis of sections 1.1 and 1.2, which also examine some general methodological issues and discuss causality. These topics include significant issues concerning paradigms and theories, as well as being about the ‘doing’ of evaluation. They are therefore significant to the design, application, reporting and use of technology evaluation.
Sections 1.3 and 1.4 move on to examine and discuss theories and paradigms in evaluation. That means the ideas, assumptions and beliefs that we draw on when we seek to construct an explanation for the processes and outcomes of whatever activity or entity is being evaluated, such as the Eurofighter example, or whether CCTV reduces crime and why, or whether nuclear weapons are a deterrent and why.
A point I want to highlight here is that theories and paradigms enter into the arena of any evaluation, albeit usually implicitly. It is likely that those that take precedence – at least initially – will be some combination of those of the evaluators, the commissioners/funders/clients of the evaluation and/or the owners of the development to be evaluated. There is no inherent reason, of course, why these might differ, particularly where acceptance of certain theories is widespread. However, where they do differ (sometimes considerably) this can lead to tensions as an evaluation progresses.
Activity 1 Theories and paradigms that underpin evaluation and assessment
Using the case studies available insuch as the Dreamliner and Superjumbo or any example of your choice, identify at least two (but no more than four) theories/paradigms that underpin your chosen development. You should then reflect on, and identify or propose, at least two examples of how these theories/paradigms may have impacted on any evaluation/assessment of your example technology. If a network or hierarchy of theories and paradigms emerges record its primary levels or features.
In a general sense, theory performs many functions and is a vital element in the evaluation process.
As our knowledge of the world has grown, so some theories and paradigms that were once dominant have been jettisoned in favour of new ones. These are, or at least appear to be, better able to explain the phenomena around us. This process is probably most noticeable, or more widely reported, in the natural sciences. Here long-accepted theories – such as why the dinosaurs died out, or the origins of the universe – remain open to interpretation, further review, or are contested.
As with any research endeavour, the study and practice of evaluation has not escaped these tensions. However, the purpose of unit is not to delve too deeply into this. Instead I want to focus on aspects of the topic where lack of understanding of different theoretical and paradigmatic positions could undermine the design, conduct and outcome of evaluation. This is a particularly important approach to take because of the subject matter and potential scale and scope of technology evaluations. It is likely that students of this course come from a wide range of academic, professional and organisational backgrounds and traditions, subscribing to a diverse selection of paradigms and theories. Consequently, taking some time to explore some of these should promote an appreciation of different traditions and perspectives that we, as evaluators, are likely to come across in the course of evaluations. This should leave us better prepared to promote more inclusive, and therefore effective, forms of evaluation whenever a situation demands it.