Exam Algorithms: Some Lessons

One of the most significant policy decisions made by the UK Government in response to the COVID-19 pandemic has been to close all schools and cancel all GCSE and A level examinations in summer 2020.

The UK Government directed the UK’s Office of Qualifications and Examinations Regulation (“Ofqual”) to produce alternative arrangements for grading students – and in response, Ofqual developed an algorithm that aimed to predict the grades that each student would have obtained had they sat their exams as per normal.

Ofqual’s algorithm has been met with widespread public outcry about the A level results, which have been perceived as unfair, discriminatory, and arbitrary. Although the algorithm has now been withdrawn, it has raised a number of important questions around the proper design and use of algorithms in general.

This issue will be of increasing importance in the future, as the analysis of ever larger and richer datasets continues to inform the design and evaluation of policy decisions and business strategy.

In this article, Dr Meloria Meschi, David Eastwood and Ravi Kanabar – experts in statistical, economic and data analysis – highlight some of the important lessons that are already emerging from the controversy.

These lessons – and others that will emerge in coming weeks, as Ofqual’s algorithm is subject to independent scrutiny by the UK Office for Statistical Regulation – should be taken seriously by all organisations involved in the design and use of algorithms for decision making.