This post is written by Paul Waller, Visiting Fellow, University of Bradford & Research Principal, Thorney Isle Research.

  • The problem: Public sector bodies in several countries are using algorithms, AI, and similar methods in their administrative functions that have sometimes led to bad outcomes that could have been avoided. Uncertainty also exists about how such uses should be governed.
  • Why it matters: People are being harmed and discriminated against in cases where public bodies make algorithmic predictions about them, and uses are being judged to be unlawful.
  • The solution: In most parliamentary democracies, a variety of laws and standards for public administration combine to set enough rules to guide their proper use in the public sector. The challenge is to work out what is lawful, safe and effective to use.

This article will use the term ‘algorithm’ specifically to mean an algebraic or logical formula that typically uses mathematical or statistical techniques and operations. This then includes things that get called artificial intelligence (AI), machine learning, predictive analytics, data analytics, algorithmic decision making, automated decision making, robotic process automation, or something similar. Any algorithm could potentially be used for a range of purposes: some sensible and acceptable, others inappropriate or dangerous.

A lack of a framework

Countless organisations have produced papers and reports on ‘ethics’ in relation to algorithms, AI, automated decision making, or data use, leading to some commonly accepted broad principles. Many documents discuss how particular aspects of these might relate to the public sector. However, there still appears to be a lack of a recognised overarching framework for public officials to use when considering the use of algorithms. This is a surprise, as the key elements have existed for ages, and indeed in 2021 a Private Members Bill was introduced in the UK House of Lords aiming to codify one (although this has not progressed further). It seems that the technical aspect has distracted people from recognising that what matters is the public administrative decision or action taken, and that is well-covered by laws and standards.

This confusion is evident in scandals arising from several governments’ efforts to use algorithmic methods in a range of areas of public administration. These include Australia’s ‘Robodebt’ where a Royal Commission enquiry is proposed, a child benefit case in the Netherlands that caused the government to fall (and more problems are being reported there), and those in the USA including in criminal justice and child protection, and several making headlines in the UK such as on predicting student’s A-level grades. A recent UK House of Lords committee report described a “new Wild West” in algorithmic systems in the justice system as vendors sold in products and services of dubious quality that should never be used in public administration, under the cover of confidentiality clauses.

The good, the bad, and the risky

Generally speaking, uses for algorithms include predicting unknown quantities, estimating which group an observation falls into, choosing a likely option from many available, finding clusters in data observations, text and spoken language processing, or goal-seeking (e.g. finding a winning move in Go or chess). Some fields of application in the public sector are well-suited to algorithmic methods, such as ‘physical’ systems or where reliable patterns exist even if hidden, like predicting traffic flows or energy demand. Other domains depend more on human observation and judgement, such as profiling or making predictions about people or groups based on data collected previously about other people. These include predicting benefit fraud or rent arrears, the committing of a crime, the need for social or children’s services intervention, or the presence of a medical condition.

The utmost care must be taken when using an algorithmic approach to make any calculation, estimate, classification, profile or prediction of any characteristics, circumstances, activities, opinions, or behaviour of people, individually or collectively. These are the ones being shown to be highly risky, often inaccurate, and even downright unlawful, with bias against particular groups being commonly found. The problems arise when they are used to make decisions that form part of a public administrative function, whether automatically or by providing a result to a human who makes the decision.

The laws and principles of good public administration are well-established

Any decision or action by a public sector body carrying out a function supported by an algorithm has to meet the same requirements as a decision made or action taken by any other means. At least in a democratic system operating under the rule of law, citizens are entitled to contest and seek remedy for any decision that adversely affects them. Public officials and politicians are accountable, and subject to scrutiny by the judiciary: they will be expected to show evidence of appropriate risk and impact mitigation measures, and have systems for redress, in relation to the use of algorithms as much as anything else done in the course of public functions. They must act within the law.

It is generally recognised that data protection law applies. However, it is likely that human rights, equality, and public records legislation does too. In the UK these laws are the Data Protection Act 2018 (which has specific clauses on profiling and automated decision making), the Human Rights Act 1998, the Equality Act 2010 including its Public Sector Equality Duty (s149), and the Public Records Act 1958. But setting these aside, some of the cases mentioned earlier were accused of contravening the fundamental laws concerning the public functions being exercised — administrative law that determines what a public sector body is and the way it works. It looks like the enthusiasm to adopt a computerised method overcame a basic step of public administrative due diligence: “Is it legal for us to do this?”.

A public body working in a system governed by the rule of law has to be transparent, fully accountable, making consistent, equitable and predictable decisions that it can explain (both the process and an individual decision). The UK Parliamentary and Health Service Ombudsman’s principles of good public administration set out these criteria. One of the best guides on their application to algorithms is published by the New South Wales Ombudsman in Australia. Some algorithmic systems would immediately contravene them, such as ‘black boxes’ that obscure their workings, ones where contractual terms restrict scrutiny or adaptive ones that change their behaviour while in operation.

Handle with care or end up in court

The approach when considering whether to use an algorithmic approach in a public sector setting is to focus on the rigorous steps needed when introducing any new system or process into public administration. The officials and political leaders will be accountable for whatever happens, possibly in court, so need to apply and document due diligence.

Potential adopters of an algorithmic method need to answer the following questions, engaging with everyone involved and affected.

  • Is the method appropriate for the application under consideration?
  • Will it work and be better than anything else for that purpose?
  • Will its use in the proposed context be lawful, safe, acceptable to stakeholders, wise, and not have bad side effects? Will it conform to the principles of good public administration?
  • If and when used operationally by a public body, how do the officials faced with its outputs work with those results? Do they understand the outputs presented? How do they change their decision-making? Do they understand it, can they explain it? How much trust do they put in the system? What is the political context telling them about how to respond?
  • Can it be maintained to the required standards as long as is necessary? What’s the fallback if there’s a problem?

These are not quick and easy answers to find, but dozens of potential risks can trap anyone not doing a thorough job. Several tools and techniques, such as auditing algorithms, are being developed to aid some parts of the investigation, and it is a good idea to have people with professional expertise in data analytics and statistics to advise on the choice of algorithm and assess its performance. The requirements in the Bill mentioned earlier are well worth following too, such as proactively publicising the use of an algorithmic method. Otherwise, newspaper headlines and a court appearance may be lying in wait. Had they applied these basic rules, none of the scandals mentioned (and others besides) and their harmful effects may ever have happened.

👋 You can create a post like this one! Share your thoughts with a community of public servants. Learn more

(Image Credit: Unsplash)