4.2.4 Ex-post evaluation
Ex-post evaluation is closely related to summative evaluation, the distinction being that ex-post typically means post-implementation. However, as evaluators and scholars of traditional programme evaluation, such as Weiss (1998) and Chen (2005) point out, owing to the scale and scope of these types of programme they often have no clearly definable end point, or the full implementation of all elements of a programme may take years, or never happen at all. Consequently, referring to an evaluation as post-implementation is something of a misnomer and may well be misleading.
By contrast, in the context of technology evaluation there are likely to be plenty of occasions where it is appropriate to use the terms ex-post – or post-implementation – evaluation to define, describe, design and carry out the type of evaluation to be undertaken. This is because the nature of technology means that, in general terms, defining when a technology has been implemented or come into operation in a particular setting or context is relatively straightforward. In short, there is a ‘before the technology’ and ‘after the technology’ point in time. Hence, ‘The purpose of ex-post evaluation is to assess and confirm, or refute, the value of an IT investment’ (Remenyi et al., 2000, p. 25). The same can be said for the opening of a new power station, the implementation of a new traffic-management system, the use of a new MRSI scanner in a hospital, the deployment of a new weapon, or the purchase of a new wifi laptop (plus wireless connection) for home use. In each case post-implementation evaluations could be used to confirm or refute the value of these specific investments in technology.
Return to the notes you made for Activities 1 to 4. Now examine how your behaviour has changed (if at all), since you ‘implemented’ your chosen technology. Whether the technology has impacted your behaviour or not, try to ascribe a financial value to any changes or lack of change. Alternatively you could base your judgement on user satisfaction. Try to be as objective as possible when you undertake this exercise (exclude post-implementation rationalisation – the benefit of hindsight – from your evaluation).