4.1 Assessment validity and reliability

When considering whether assessment is fair to all students, two issues are important: validity and reliability. Validity asks whether grades generated by a testing system represent a student’s achievement in the whole of geography. Can a series of timed, written tests at the end of a key stage assess all those things we think are important for students to learn about in school geography? Butt (n.d.) describes AoL as generating ‘high stakes’ results. By contrast, he describes teacher assessment outputs as ‘low stakes’. Teacher assessment is often seen as being of lower status than the results of tests even though it is more likely to be valid in these terms.

An over-reliance on test results may lead teachers to make generalisations and judgements about a student’s capability in all aspects of geography, based on the formal testing of a subset. For example, a grade ascribed on a short answer test says nothing about a student’s problem-solving or creative capacities in geography, nor about their ability to work in groups or engage in extended tasks. Perhaps all that can be said is that the tests simply tell us about the capabilities of students to answer questions at a particular time and of a particular type (and in the conditions and circumstances of the test) – no more and no less.

Reliability asks whether a student’s performance changes (or not) depending on the particular questions that are set. Ideally, assessments should give every student an optimal opportunity to demonstrate what they know. In practice, however, tests have been found to be biased against students from particular backgrounds, socio-economic classes, ethnic groups or gender (Pullin, 1993; Cooper, 1998). Equity issues are particularly important when assessment results are used to label students or deny them access to courses or careers in the future.

Research shows that working-class children:

… go into lower sets …

… are set less challenging test items …

… and therefore they have to answer more ‘realistic’ test items …

… so they achieve lower test results than they deserve …

… so they are confirmed as being appropriately placed in lower sets …

… so they may experience a less rich curriculum, and will learn less content …

… which restricts their opportunities to get high grades in examinations at the end of their schooling …

… which restricts their career opportunities …

… which means they and their family may be locked into a cycle of underachievement.

As you set assessments look for bias that might disadvantage different groups of students. Avoid setting a ceiling in terms of what students can achieve.