This article is written by Tim Hill, Data Standards and Technical Lead at the Open Data Institute

Four days before the UK went into lockdown, Neil Ferguson, head of the Medical Research Council (MRC) Centre for Global Infectious Disease Analysis, sent out a faintly apologetic tweet.

Though a lot of people had wanted access to the MRC's Covid-modelling codebase, Ferguson admitted it consisted of “thousands of lines of undocumented C” written “thirteen years earlier to model flu pandemics”. They were working with Microsoft and GitHub to clean up and rationalise the code. Until then, it was unavailable.

Want to write for us? Take a look at Apolitical’s guide for contributors

The tweet was easily lost amid the welter of other coronavirus-related stories that week. But for a software developer like myself, the admission was jolting. Thousands of lines produced over thirteen years? In an error-inducing language like C? The very thought produced a feeling of vertigo. How would you even start finding and fixing all the bugs that might have crept in?

Even worse – how could you defend public policy based on it, especially (terrifyingly) when that policy had implications as enormous as the then-pending Covid-19 lockdown?

Assessing Unrealities

Recent years have seen growing concern regarding the use of computer models in public decision-making – in particular, with questions of bias. Pervasive though such algorithmic biases are, however, public-policy understanding of these computational ‘black boxes’ is growing, and techniques for detecting and remedying their shortcomings are increasingly well-developed.

Ferguson’s Covid-19 code, however, isn’t like the majority of computer models in use today. Most algorithms to have come under recent scrutiny are ‘forecasting’ models: given a range of known factors (today’s temperature and humidity say), we attempt to predict an outcome (such as tomorrow’s rainfall). If our prediction is correct, we know the algorithm is working. If not, we refine it with new evidence.

Ferguson’s code, by contrast, is a ‘simulation’. Rather than tracking the effects of individual factors on known outcomes, simulation models help scientists explore the possible effects of large-scale policy decisions. In the context of Covid-19, a forecasting model might be used to answer questions about how factors such as age, ethnicity, and income affect survival rates, while a simulation model might ask questions like ‘If we close all airports, what is the likely effect on transmission rates? What if we keep them open?’

Simulation models are fundamental to exploring policy options in some of the most important issues governments face today

This difference in focus makes simulation models hard to evaluate. Of the many scenarios considered, normally only one will come to pass. If you estimate that 150,000 deaths will occur unless airports are closed and after such measures are taken, the death toll is lower, that doesn’t make the initial projection wrong. It does however make it hard or impossible to assess the accuracy of that original estimate, precisely because the entire scenario was avoided.

Equally, updating the model as new data emerges isn’t simply a question of adding it to a training set. Instead, it’s an open-ended process of revising assumptions and parameters with new information, and at times developing new scenarios.

From an academic perspective, there’s nothing wrong with this ad hoc approach. For public policy, however, it’s deeply problematic. Simulation models are fundamental to exploring policy options in some of the most important issues governments face today, including global warming and vaccine refusal. But how can decisions based on such models be justified? Or, looked at another way: how can such models be trusted?

Open Methodologies for Algorithmic Auditing

The question of trust, after Ferguson’s fateful tweet, was an urgent one. Ideological opponents of the lockdown measures being contemplated were quick to raise doubts about code quality; one tech-savvy MP declared the situation potentially ‘scandalous’.

Given the difficulty of auditing simulation models, the only way to counter the rising tide of criticism was transparency: people outside government needed to be able to review the MRC’s code, data, and results and to assess their conclusions.

Fortunately, thanks to the work of academics, advocacy groups, and software engineers over the past two decades there are now clear methodologies for doing so. These approaches go under different names and have different emphases, but what they share in common is an emphasis on openness: open access; open data; open science; open source software.

All of these ‘open’ approaches had a role to play in vindicating the MRC’s work. Once it had embraced these, progress was rapid. On 22 April, Ferguson and his team released a streamlined and refactored version of their code as open-source software. Three days later, they fixed a bug identified by an external team. And finally, on 1 June 2020, an independent audit was able to replicate results the MRC had earlier published under an open-access license, confirming that neither the refactoring nor the bug had altered the code’s results.

Ten weeks after Ferguson’s sheepish tweet, the validity of the MRC’s work was confirmed. But those ten weeks were a problem: ten weeks of breakdown of trust among the public; ten weeks of distraction for the team, and ten weeks of uncertainty for public servants unclear on whether they could rely on the information they were being given.

In part 2 of this article, we’ll look at how to avoid these kinds of catastrophic delays, and make sure open methodologies are, where appropriate, baked into your projects from the start — Tim Hill

We want to hear what you think about this article. Pitch an article to us or send your comments to hello@apolitical.co

(Photo credit: UnSplash)