4.1 Comparing national achievement outcomes

[MUSIC PLAYING]

VOICE ON RADIO

These are—

NARRATOR

If you read or listen to news reports about education, you've probably noticed periodic surges of interest in which country's students do best in reading or mathematics or science and where your country fits into the grand scheme of things. You've probably also heard or read the word PISA in connection with some of these reports.

What is PISA? PISA is an acronym that stands for the Programme for International Student Assessment. It's the brainchild of the OECD. And what's the OECD? It's another acronym that stands for the Organisation for Economic Cooperation and Development. The OECD brings together 34 countries with the aim of developing better policies for better lives.

In the late 1990s, countries that are members of the OECD came up with the idea to measure whether 15-year-olds around the world are well-prepared to participate in society. We chose 15-year-olds, rather than 12- or 17-year-olds, because most 15-year-olds are about to complete their compulsory education. Experts in the field of education from around the world worked together to create a two-hour test that focuses on core subjects like reading, mathematics and science. Participating countries decided to administer this test every three years and to rotate the main focus of the test among the three core subjects. All very well, but testing students is nothing new.

So what's so special about PISA? PISA surveys are designed to find out whether students can use what they have learned in school and apply that knowledge to real-life situations and problems. PISA is less interested in knowing whether students can repeat, like parrots, what they have been taught in class. Rather, the survey is designed to find out whether, for example, students can use the reading skills they have learned at school to make sense of the information they find in a book, a newspaper, on a government form or in an instruction manual.

But the point of PISA is not to tell each individual student how well he or she has mastered a set of skills. Instead, PISA results are analysed and extrapolated to the national level. Picture one student sitting at a desk in a classroom somewhere taking the PISA test. Now zoom out as though you're on the space shuttle, and you can see the entire country in which that student is sitting. That's what PISA does with its test results.

PISA shows countries where they stand, in relation to other countries and just by themselves, in how effectively they educate their children. While PISA doesn't say this education policy or practice causes that effect, it shows what's possible and shows similarities and differences between education systems around the world. That helps governments rethink their own policies and design new ones to improve their students' performance in school.

It also helps governments, educators and parents track their country's progress towards a more successful education system. In fact, many countries now set national goals and benchmarks based on PISA's international results.

PISA considers an education system successful not only if its students achieve high scores on the PISA surveys, but also if all students from all backgrounds perform well on the tests, not just those who come from wealthier or more intellectual or more culturally sophisticated families. For example, a relatively large percentage of disadvantaged students in places like Hong Kong, Shanghai, Korea and Finland achieve some of the highest scores in PISA.

Analysts then look at the PISA test results, along with the responses to questionnaires that are given to students and school principals, and try to determine the main characteristics of these successful education systems. Are teachers in these systems paid more? Are classes generally larger or smaller? Do individual schools get to decide what their teachers teach, or is the curriculum determined by a central government authority? Once the profile of a successful system emerges, it can be used as a model for others.

So what's the test like? Here are a couple of problems that students had to solve. The first one was part of the reading assessment. Students were shown a graph and an accompanying text that read, ‘Figure 1 shows changing levels of Lake Chad in Saharan North Africa. Lake Chad disappeared completely in about 20,000 BC, during the last ice age. In about 11,000 BC, it reappeared. Today, its level is about the same as it was in AD 1000.’

Students were asked, ‘why has the author chosen to start the graph at this point?’ This was considered a rather difficult question. In fact, across OECD countries, only 37 per cent of students answered it correctly. Those students showed an ability not only to read, but to think about what they read and draw some conclusions from it. They figured out that the reason the graph starts at around 11,000 BC is because that's when Lake Chad reappeared after having disappeared completely during the Ice Age.

The second problem was part of the mathematics assessment. It sets a scene: ‘in a pizza restaurant, you can get a basic pizza with two toppings, cheese and tomato. You can also make up your own pizza with extra toppings. You can choose from four different extra toppings: olives, ham, mushrooms and salami. Ross wants to order a pizza with two different extra toppings.’ Students are then asked, ‘how many different combinations can Ross choose from?’ Now, this is a problem we can all relate to. To answer the question, students had to show their ability to make connections, in this case, juggling several foodstuffs and realising that the choices are not, in fact, infinite, much as the students might like them to be. This was also a relatively tough problem for students to solve. Just 49 per cent of students could calculate that only six different variations were possible.

So when we take individual students' scores in PISA and their responses to those questionnaires that we circulate with the test and then zoom out to see a whole country, what kinds of things do we learn? Well, one thing we've learned is that girls do better in reading than boys in every country that participates in PISA. And among the countries that are members of the OECD, girls do so much better than boys in reading, it's as if they had gone through an additional year of school. We also found out that boys generally do better than girls in mathematics and that there's no real difference between boys and girls in how they do in science.

We also found out that some school policies may not be very good for students. For example, early tracking of students, which means deciding that some students should go through an academic programme, while others should go through a vocational programme, is not associated with better overall performance. Tracking is also related to greater inequalities between advantaged and disadvantaged students. Students are often tracked in the mistaken belief that not everyone can learn the same things, that only some children are gifted and can reach for the stars. But PISA results show that if given the opportunity and support to excel, all children have the potential to do so.

Having students repeat grades is also not associated with high scores in PISA. School systems that invest in helping students learn their subjects the first time around do much better than those where teachers know that they can, if necessary, drill the same material year after year after year into the heads of the same struggling students.

As we mentioned before, the most successful schools, according to PISA, are the ones whose students do well regardless of where they come from. Still, results from PISA show that home background has a major influence on students' success in school. In many ways, this finding is all too obvious. We know, for example, that by the time they're three, children in advantaged families are exposed to many more words than their less advantaged peers. In fact, a recent study in the United States put the number at around 30 million more words. And in general, if there are no books at home or if children don't see their parents reading, they'll be less inclined to read themselves.

PISA results also show that regardless of their own backgrounds, students who attend schools that have a largely disadvantaged student population tend to do worse than students who attend schools with relatively advantaged peers. Why would this be so? Well, there are many possible reasons. For example, PISA found that in most OECD countries, disadvantaged students have access to the same number of teachers, and to sometimes even more teachers, than their more advantaged peers. The problem, though, is that more isn't necessarily better. In fact, the best teachers are often found in schools attended by advantaged students, who generally do well on their subjects anyway, but not in disadvantaged schools, where high-quality teachers are most desperately needed.

Governments around the world can be inspired by two significant findings from PISA. The first is that a country doesn't have to be wealthy to provide high-quality education to its students. Shanghai and Poland, for example, score above the OECD average in reading, but rank below the OECD average in measures of national wealth.

And a country's PISA ranking is not carved in stone. Trends in PISA have revealed the great capacity for all countries to improve. Countries as diverse as Chile, Germany, Poland and Portugal, among others, showed improvements in student reading performance between 2000 and 2009.

Although it seems obvious, it's still worth reminding ourselves successful education systems make education a priority. They share the belief that skills can be learned and that all students can achieve at high levels. They show that they value the teaching profession by investing in it so that they can attract highly qualified candidates, train them well and retain the best teachers among them. Just as every student has the potential to achieve, every country has the potential to raise the standards of its education.

So PISA is about a lot more than test scores. Sure, countries that participate in the PISA survey are keen to know where their students rank compared to students in other countries. But PISA's ultimate aim is not to create a competition for its own sake. Its aim is to encourage all participating countries to use the survey findings to improve their own teaching and student performance to give every student the best opportunities to achieve the best possible results.