1.5 Training evaluation
The final stage in the standard approach to training (as shown in Figure 1) is evaluation. Although evaluation typically takes place at the end of the training cycle, deciding on the approach to evaluation is something which should normally be part of TNA work: evaluation criteria should be built into a training programme from the outset, and not as an afterthought!
A great deal of work in this area is based on the Kirkpatrick (1979) model, which has become a classic in the field of instructional design. It is easy to understand, well tested and forms something of a common currency among training evaluators and HRD professionals. The model proposes four levels of evaluation:
- reactions
- learning
- behaviour (job impact)
- results (business impact).
Level 1 – reactions
Reactions are usually captured using attitude questionnaires or surveys administered at the end of a course. These ask students what they thought of the programme, whether the setting was conducive to learning, which parts they particularly liked, and whether there were any aspects they did not like. Questionnaires measure subjective perceptions of training, not whether it will have any impact on behaviour or performance. This subjectivity is both a strength and a limitation. On the one hand, such surveys can capture rich, often qualitative, data on the student experience, sometimes revealing aspects of the training that course designers and facilitators may not have been aware of. On the other hand, by focusing on students’ likes and dislikes, such surveys may distort an instructional design towards what will be popular and/or enjoyable, rather than what will be most effective or informative.
Level 2 – learning
Learning relates to the absorption of new knowledge and content. Evaluation at this level is usually undertaken using pre-test/post-test comparison – that is, a measurement of the changes in skills and/or knowledge that can be directly attributed to the training intervention. Formal assessments, qualifications and exams are all examples of measuring achievement at this level of evaluation.
Level 3 – behaviour
Behaviour refers to the successful application of learning – that is, the transition from the classroom to the workplace. Level 3 evaluations can be performed using formal assessment or through more informal approaches, such as observation. This sort of evaluation normally needs to be conducted by someone with in-depth understanding of the job in question and the degree of performance improvement that can realistically be expected from the training. This type of training evaluation should be aligned with performance management reviews for the individual trainee.
Level 4 – results
Results refer to the link between impact on the job and impact on the organisation. If training has been well designed and has met its objectives in terms of individual performance (level 3), there should be a feed-through to enhanced business performance. It is at this level that training can start to be evaluated in terms of its return on investment (ROI). Thus, HRD strategy often involves gauging the rate of return for an organisation’s investment in its people. Training and development often make up substantial proportions of this investment; so level 4 evaluation is considered a crucial competency for HRD professionals in corporate and business strategy.