Fairness in the Oxford Admissions System
Nigel Williams, 28 February 2013
It’s easy to make a headline out of a detailed table of figures. If there are enough categories and they can be sliced in enough different ways, an outlier is almost guaranteed. As a rule of thumb, one in twenty random facts can be claimed as statistically significant by the unscrupulous. That is why statisticians are forever having to justify their figures and their claims to a mistrustful public.
To be fair, when the Guardian claimed on Tuesday ‘Oxford University accused of bias against ethnic minority applicants’, they also published the data on which their claims were based. It is a challenge to investigate for anyone unwilling to take the claim at face value.
All things are not equal at the time people make their applications to Oxford. Anyone obliged to attend a school where other students disrupt classes is at a disadvantage from the start. GCSEs will be affected, making a large component of the data on which admissions tutors make their decisions. Where a school aims above all to improve its pass rate at Grade C or above, some candidates will miss out on the support that could get them As. Teaching to the test will inevitably deprive candidates to Oxford of some of the breadth that will later help them at interview. These things are not the specific point of the Guardian’s accusation but they affect many more people than one year’s unsuccessful applicants.
The data include aggregated exam results, so relate to candidates that have emerged from the education system with the credentials to make an application. The question is one of whether people with comparable academic records are receiving equitable treatment. The admissions system is not totally fluid. Applicants choose a subject and a college. Usually, there are single-figure places available for each combination, but there is no perfect clearing system for moving a well-qualified candidate to a course with more room. A college will understandably look more favourably on a candidate making it their first choice. Candidates are not likely to switch subjects mid-application in order to improve their chances, although second applications in other subjects are not unknown. For all their rigour, even A Levels are not a perfect or permanent indication of anyone’s academic prowess. If the same candidates resat, there would be some shuffling of their relative positions, especially in subjects with any degree of subjectivity in their answers. An admissions tutor is not bound to reach the same conclusions as an A Level examiner. It should not therefore be surprising if there are some students at Oxford with a weaker academic record than rejected candidates.
The Guardian deny none of these caveats and even draw attention to several of them themselves. The question they set themselves is whether differences are systematic. If there are discrepancies, it is a further matter to demonstrate that they are the result of bias among the admissions tutors. However imperfect, A Level results are probably the best independent measure of academic prowess.
What does bias mean here, so far as we can identify it from the data? It means offering places to less well-qualified candidates on the basis of some other criterion. The Guardian’s article looked at overall ratios of applicants to successes, followed up with success rates for people getting three A*s at A Level and applying for medicine. The first of these, quite obviously, depends on the number of applicants, without controlling for their academic standard. If something is encouraging higher proportions of ethnic minorities to apply to Oxford, that is no reason for censure. All that may be required is to assist people in getting best value from the system. The data risk getting disclosive, but supposing it were easy to look up numbers of applicants and the median and minimum A Level scores of successful candidates for each combination of course and college, that information could save a lot of disappointment.
For the university as a whole, all subjects, there were enough candidates with two A*s and an A to fill all places. Among white candidates, 2,246 were offered places but got worse A Levels. 3,523 reached that standard but were still declined. In both cases that is just over 60 per cent of candidates in that situation. However, the spread is uneven. Offers went to 3,278 candidates at the standard of two A*s and an A. 5,570 attained that standard but were rejected. 3,667 offers went to people with worse A Levels. Among the many reasons why, there are over-and under-subscribed courses and colleges and different performance at interview and exam before any question of racial prejudice arises. In that total, candidates of black background or dual heritage constituted a larger share of those receiving offers with weaker A levels than of those missing out despite good results. Asian and Chinese candidates fared less well but the most remarkable grouping was candidates of unknown race or refusing to answer. More on them later.
The three over-subscribed subjects in the data, Medicine, Law and Economics and Management, show such different patterns that it is obviously dangerous to generalize. In Economics and Management, 194 offers were made, which could have gone entirely to people gained three A*s. They did not. Only 41 achieved that level, and only 136 got three As. 154 candidates with three A*s were turned down, 153 offers went to people with lesser grades. White candidates represented 54 per cent of those offers. Other races getting three A*s, only 13 out of 44 were offered places and 20 places went to candidates with lower grades. The biggest hazard for candidates with top A levels appears to be not answering the question. 60 per cent of candidates for this course gave no answer for whatever reason. Only four out of 97 with three A*s got offers. Only 51 others got places, against 82 from the smaller number of admittedly white candidates. This does raise questions. Why so many not answering the question, especially in a course where surveys and forms could be expected to play an important part? Do the admissions tutors know who lacks ethnicity data and can it possibly be affecting their decisions? Do candidates that dislike forms perform worse at interview?
Medicine could be more than filled with candidates with three A*s. Again, the lowest success went to candidates of unknown ethnicity. Four out of 56 with three A*s got offers. White candidates enjoyed higher rates of offers for lower grades, coupled with lower rates of candidates with three A*s turned down.
In Law, not answering the question became an advantage. Selecting on A Levels alone would require an A* and two As. Oxford’s admissions system awarded the highest proportion of offers to the 15 per cent of Law candidates of unknown ethnicity, awarding to them more than half of the instances where offers went to candidates with weaker A levels. White candidates produced good exam results, with 45 per cent of the pool receiving two thirds of the results reaching the threshold, but they benefited less from places given to candidates with weaker grades.
At the level of the university as a whole, 29 per cent of offers to candidates getting worse A Levels than two A*s and an A were where their ethnicity was unknown. The same group had only 22 per cent of cases where those A levels received no offer. It has the makings of a story by itself, but, given the different results for law, medicine and economics and management, it shows that the pattern is far from uniform. The inference is that, whatever your ethnicity, for medicine you need to know about your genes, for management you should complete all the forms but if you like avoiding questions you should study law.