IQ Tests, PISA scores and polar bears

Nigel Williams, 5 December 2013

Boris Johnson earned criticism this week for making remarks about IQ tests at a Margaret Thatcher lecture and then falling foul of a trick question in an interview on LBC. Almost simultaneously, the OECD’s PISA tests ranked UK countries as fair to middling.

The figures in the IQ remarks were nothing exceptional. IQ tests are calibrated in just the way he described. A test designed to fit a normal distribution with a mean of 100 and a standard deviation of 15 will give results above 130 in about 2½ percent of cases and below 85 in 16 per cent. It’s no more controversial than suggesting that there are 100 centimetres in a metre.

Where opinions divide is in how people interpret IQ scores. A question that caught the mayor out shows the dangers of excessive reliance. If a man builds a rectangular house with all four walls facing south and a bear goes by, what colour is it? Of course the location where the only way is south is the North Pole, but the question is set in an imaginary world. Even polar bears are very rare that far north, and if a man can transport the building materials, he can also transport a bear of whatever colour he chooses. So it is a mistake to interpret that question as indicative of a person’s entire worth. It’s still funny.

Work by Peter Saunders has shown that IQ scores are a good predictor of a person’s social success, by the standards by which our society chooses to judge. If we wish to break the connection between IQ and social worth, it will take more than changing the scale; we will need to reappraise the qualities that we choose to reward with status and high salaries. Such qualities could include hard work, caring, attention to detail, tenacity and the humility to perform tasks that others find unappealing. These qualities can be found among people with high or low IQs. Contrary to the Mayor of London, greed does not belong in such a list.

Meanwhile, we expect our schools to prepare every child for a world of attainment. Gustav Holst was reported to say that high art that was just for the elite, but that the only way to tell who the elite were was to expose everybody to high art. It’s an approach in keeping with social mobility but it does risk wasting a lot of people’s time. If a school can prepare a child from any background to fill what we subjectively consider a top job, that is impressive. But a great many jobs matter that are not deemed ‘top’ and if people leave school equipped to do those jobs, one job each, to a high standard, that is even more of an achievement. Society can choose how it wishes to distribute status.

PISA scores from the OECD measure some aspects of how schools prepare children in reading, maths and science. Of course it would be nice to be top but to be around the average for a developed country in reading and maths and above average in science is hardly a crisis. Those marks come with a range of possible ranks, so it is possible that the UK’s 15-year-olds were actually above Denmark’s or below Norway’s. Where children did not answer every question, answers were imputed. It is a recognised procedure, a bit like concluding that, after falling for two trick questions, Boris Johnson would be caught out by a third. But imputation cannot reduce uncertainty and is a major reason why PISA report large spreads of possible ranks. The level of imputation in the process has persuaded David Spiegielhalter, Winton Professor for the Public Understanding of Risk at Cambridge, that it is unwise to drill down below the headline country figures and that some questions are more difficult in some countries than in others.

How should schools respond? There is no need for any blame. Schools are preparing people for life in Britain, not as a competition with the rest of the world. We may have to learn that cleverness alone will not solve every economic ill. If children want to read about Gustav Holst or the North Pole, they can find them in the Core Knowledge UK series of books. It may save one or two from future embarrassment on LBC. It may help a lot more discover something they want to learn more about. Whether they get top jobs or not, education is about more than league tables.