Another important factor in favour of adopting a mixed-methods approach is its use as a means of triangulation. This process is regarded as significant to the reliability, credibility and validity of an evaluation. Taken together with attention to ethics and standards, these features are crucial if trust in the integrity of an evaluation is to be established and maintained. I discuss these issues in more detail at the end of the unit. However, I want to say something more about triangulation here.
Put simply, triangulation is a commonly used technique to improve the internal validity of an evaluation (or research in general). However, the detail of the actual means employed to do this tends to vary, depending on which book on evaluation or research methods you read. To some authors, triangulation is synonymous with the use of mixed methods for data collection (e.g. Burns, 2000). This is probably the most common understanding – and therefore design of – triangulation, and could include, for example, the use of interviews, observation, document collection and recording of events and activities. The sets of data would then be combined to create a situation where ‘having a cumulative view of data drawn from different contexts, we may, as in trigonometry, be able to triangulate the ‘‘true’’ state of affairs by examining where the data intersects’ (Silverman, 2000, p. 98).
Others adopt a more complex interpretation of triangulation. Drawing on the work of Denzin (1970), Clarke (1999) outlines four types of triangulation: data, investigator, theory and methods. In similar vein Bamberger et al. (2006, pp. 147–8) state that triangulation ‘involves deliberate attempts to confirm, elaborate, and disconfirm facts and interpretations’ by use of:
- multiple data sources
- multiple methods of data collection
- multiple evaluators or data collectors
- repeated observations over time
- multiple analytical perspectives.
Each, or a combination of any of these, will deliver some degree of triangulation; the more that are used, the higher the degree of triangulation. In practice, however, resource, and time and budget constraints are likely to dictate the degree of triangulation that can be achieved in any evaluation. That said, for the reasons I set out when introducing the subject, it is advisable to design some degree or form of triangulation into any evaluation. The next activity provides an opportunity for you to work through this.
Activity 3 Triangulation in evaluation
Choose a technology and then construct an evaluation for this technology that includes triangulation. You must be specific as to how you have achieved this. For example, if you are going to use multiple data sources or multiple methods of data collection, or any combination of the five categories noted in the unit, you must be specific about the data sources or what the data collection methods are. Feel free to use the text box below.
You should consider:
- a. details of how you propose to achieve triangulation;
- b. why you selected each of your chosen means/methods;
- c. at least one possible alternative to one of your chosen means/methods.
I want to round off the first part of this discussion of the relationship between theory and evaluation, and role of theory in and on evaluation, by considering causality, which is one further dimension of evaluation (and social research generally) where important paradigmatic differences exist.