5.1 Validity and reliability of assessment
The validity of an assessment is the extent to which it measures what it is supposed to measure. The reliability of an assessment is the extent to which it measures the skill, learning or competence consistently (Quinn and Hughes, 2007).
Activity 5 How reliable is your assessment?
Think of a time when you have been involved in the assessment of a student. This can be an initial diagnostic episode, or formative or summative assessment.
- How reliable do you think your assessment was?
- What do you think can affect the reliability of an assessment?
Click on ‘Reveal discussion’ to reveal possible factors that can affect the reliability of assessment.
Reliability of assessment can be affected by:
- only being able to observe one episode of a particular skill or competence
- bias towards the student (both positive and negative)
- being unable to gather evidence from colleagues or service users
- being unsure of documentation requirements
- using only one method of assessment (such as discussion)
- the readiness of the student
- adequate preparation
- your mood – a very busy or rushed day versus a calm, relaxed day.
If you are completing this as part of an NMC mentor preparation programme, write this as a critical reflection for your portfolio of evidence. On completion of your mentorship preparation programme, a subsequent reflection can be included relating to a recent assessment to demonstrate new learning.
A range of assessment methods should be utilised to give an overall picture of how the student is learning and progressing in practice. This will also increase the validity and reliability of assessment. The assessment documentation to be completed will have been scrutinised as part of required quality assurance mechanisms, which may include professional bodies such as the NMC and by the programme provider or university.
However, a study by Fahey et al. (2011) suggests that the language of competency documentation can often be misinterpreted by both students and mentors, so you must get clarity from the programme provider or practice link facilitator if there are any uncertainties. A study by Brown et al. (2012) suggested that even experienced mentors lacked confidence in their assessment of students, in particular with failing students (this is considered in more detail in Week 7), so always seek support from your own supervisor, mentor colleagues, educational lead and practice link tutor.
Remember that your assessment of a student’s learning should be based on required criteria, and not norm-referenced. Norm-referenced assessment is where the student is assessed against another student who is at a similar stage of their learning (Anderson, 2011).