The stakes are high when it comes to using machine learning in government. Accusations of ethical missteps resulting from automated decision-making include false debt accusations and allegedly racially-biased prison sentences.

But there are guidelines out there to help you avoid biases, be transparent and drive positive impacts for society.

Drawn from conversations with experts and guidelines from the Ethical AI Institute, the UK Government and the Alan Turing Institute, building these three key considerations into your work should help you prevent your project from causing unintended harm.

1. Consider the impacts of your system

Before you even begin a project, you should take a step back and ask yourself whether the decision to automate the process is the right choice, according to Andrew Harding, Senior Technology and Policy Adviser at the Centre for Data Ethics and Innovation. “Make sure the moment of automation is one of reflection and pause,” Harding said.

When you automate a process, he explained, it can become harder to inspect, observe and test. This means automation is not always the best choice — especially for systems you don’t understand well or cases where the policy environment isn’t clear.

The UK Government’s Guide to using AI in the Public Sector suggests that you “consider the impacts [your project] may have on the wellbeing of affected stakeholders and communities”. Taking time to do this before you implement your system is a good way to predict and avoid harmful effects.

A co-author of the guide and former adviser at the UK Government’s Office for Artificial Intelligence, Sébastien Krier, said that he recommends using the Principle of Discriminatory Non-Harm. This can help you ensure your system doesn’t have “discriminatory or inequitable impacts” on the lives of the people it affects — or involve protected characteristics, including sexual orientation, religion and disabilities, where it shouldn’t.

He recommends working closely with end users, civil society and experts to understand these effects.

You might also want to take a look at the human rights framework, said Harding. Ask yourself: Does your system affect anyone’s right to private life? Does it make it harder for people to enjoy the democratic system?

Kate Carruthers, Chief Data Officer at the University of South Wales and member of the Ethical AI Network, used the example of a vendor who proposed a machine learning solution for university admissions that would predict the characteristics of top performing students. She explained that, when the university considered the impact of this software, there was the risk that it would encourage universities to admit rich, white men, rather than boosting diversity.

2. Identify and mitigate biases

“Always keep in mind that these technologies, no matter how neutral they may seem, are designed and produced by human beings, who are bound by the limitations of their contexts and biases,” warns the Alan Turing Institute’s Dr David Leslie in their chapter on understanding AI ethics and safety.

Humans can inject their bias into systems at any stage of the process, from data collection to implementation, while datasets may already contain cultural biases and discrimination.

For instance, Krier argued that by using datasets on historical arrests, predictive policing systems are not predicting crime but police decisions to arrest. Accused of being built with “dirty data” that already reflects systemic biases, the systems allegedly amplify corrupt, illegal, and unethical policing practices amid a legacy of discrimination.

Bias is neither necessarily intentional nor immediately evident

Bias can also shape the development processes for technologies. A study by MIT showed that facial recognition software achieves lower accuracy rates on darker female faces than lighter males. It suggested that this is because many companies do not test how systems perform across different subgroups, and benchmark datasets tend to overrepresent lighter individuals, especially men.

“Data isn’t neutral,” said Carruthers, “as soon as you've decided to use one piece of data not another, you’ve already got bias creeping in.”

Bias is neither necessarily intentional nor immediately evident. So how can we detect and mitigate it?

First of all, be careful not to rely too much on the technology. “It’s called AI because technologies can be intelligent and make decisions. But they don't have context, awareness and sense of fairness — and ultimately they don't have judgement,” said Harding. Ensure that humans are checking the predictions and making a final call.

The Ethical AI Network recommends adding “human-in-the-loop review processes” to avoid bad predictions.

But first, make sure that your team has the necessary training. The Principle of Discriminatory Non-Harm advises ensuring that systems are deployed by people who have enough training to implement them responsibly and without bias. There is sometimes a tendency to rely on an algorithm’s output without properly questioning the results, Krier explained. It is therefore important to ensure people are adequately trained to work with AI.

And remember to think about who is involved in your project — a range of perspectives is a must. It’s really important to have diverse teams, said Krier, and in some cases to involve academics and policy experts, as domain knowledge can significantly improve how algorithms are designed.

You might want to consult an NGO that is active in your field, suggested Harding. For instance, if your system was affecting LGBT+ people in the UK you could consult an organisation like Consortium.

There is plenty of expertise out there to help you mitigate bias. Frameworks, workbooks and templates from the likes of the UK government, AI Now and the Open Data Institute can help you detect and mitigate biases in your data, while regulators like the Financial Conduct Authority can be called on to review your processes.

Krier recommended the Toward Trustworthy AI Development report by Cornell University. Meanwhile, Harding’s organisation, the CDEI, is conducting a Review into bias in algorithmic decision making.

3. Be transparent

In order to develop a transparent machine learning system, you need to be able to both explain the rationale behind its decisions and behaviours and justify the design and implementation processes.

Take Krier’s example of predictive policing systems adopted by law enforcement agencies to forecast crimes. He explained that, in some cases, the opacity of how the models are developed meant that the algorithms could not be fully understood and scrutinised by the law enforcement agencies.

The AI Institute recommends that policymakers make public their use of automated systems and inform citizens affected by the decisions they make, while Nesta advises that the public be made aware of all the inputs which govern how the system comes to a decision or recommendation.

If policymakers fail to win trust, they risk severely damaging government-citizen relationships

The best way to ensure transparency is to start thinking about it early — not only after you’ve developed your model. Often machine learning draws on statistical assumptions. If you develop your model in a statistically robust and scientific way, and you’re honest about those assumptions, it will be easier to explain it afterwards, said Harding.

It’s also important that citizens have “meaningful agency” he added: engage with the people your system affects to ensure they can feed into the process you’re designing.

But how can public servants get citizens on board? To prove that their system is worthy of public trust, policymakers should guarantee as much as possible its “safety, accuracy, reliability, security, and robustness” explains the UK’s public sector AI guide. For instance, by ensuring the privacy of citizen data.

If policymakers fail to win trust, they risk severely damaging government-citizen relationships. The Australian government was accused of a “cover-up,” after their Robodebt program — an automated system to notify citizens of debts they owe government — allegedly had a 20% error rate.

“There is a genuine ability to deliver poorer outcomes for citizens at scale through the use of technologies like machine learning,” warned Carruthers.

But with new technologies comes great potential. By instilling good ethical judgments in our machines, Harding explained, we can enable them to render decisions that are less biased and judgemental than people can, whether through lack of time, pressure or falling back on cultural prejudices. — Anna Goulden

(Picture credit: Unsplash)