1.2.1 Experimental and quantitative approaches
Historically the scientific paradigm has dominated the field of evaluation (Clarke, 1999). This means that quantitative methods and a preference for experimental evaluation designs that test a hypothesis have been preferred. As with a scientific experiment, advocates of this approach argue that evaluations should aim to establish internal validity by, for example, working with randomised control groups (in the case of programme evaluation) who are subjected to whatever development or project is under investigation. The outcome is then used to infer a causal relationship between input and outcome.
Applying this approach under laboratory conditions, or in some other form of ‘closed’ system, such as could be utilised when testing a component or relationship between one technology and another, can work, of course. However, trying to employ this approach in an ‘open’ system, such as an office, hospital, or an even larger social system, such as a housing development or town, means that creating the ‘controlled’ environment necessary for the ‘experiment’ is impossible (e.g. by restricting or trying to factor in the number of variables that might come into play).
In an attempt to combat these problems, proponents and practitioners of the scientific approach to programme evaluation responded by developing ‘quasi-experimental’ designs for evaluations. These methods tried to incorporate the identification and control of factors that might give rise to alternative explanations for observed outcomes. They also incorporated some device which ‘compensated’ (in some way or another) for potential errors that might arise due to the lack of controlled experimental conditions. Despite these developments Clarke (1999, p. 51) notes that ‘There is much controversy surrounding the legitimacy of the quantitative-experimental paradigm in evaluation research.’ Nevertheless, one significant variant of this type of evaluation is computer modelling. It is widely used in a number of fields of technology evaluation. The next activity provides an opportunity to investigate the use of modelling for evaluation.
Activity 2 Computer modelling and technology evaluation and assessment
Using any source of your choice, identify one example of a technology or technological development where its operation or output/outcome, or impact, is largely modelled using computing. Add your findings to the below text box noting:
- a) the main reason(s) why modelling is used;
- b) what is modelled (operation, outcome, impact);
- c) the real or perceived strength of using modelling for your example;
- d) any real or perceived weaknesses in the use of modelling for your example.