Watchdog to monitor how public bodies use AI, issues guidance on avoiding breaches of equality law

The Equality and Human Rights Committee is to monitor the use of artificial intelligence by public bodies for the first time “to ensure technologies are not discriminating against people”.

The equality watchdog said monitoring projects would last several months and would report initial findings early next year.

The EHRC said: “There is emerging evidence that bias built into algorithms can lead to less favourable treatment of people with protected characteristics such as race and sex.”

Tackling discrimination in AI will be a major strand of its new three-year strategy.

The EHRC has also today published new guidance, Artificial Intelligence in Public Services, to help organisations avoid breaches of equality law, including the public sector equality duty (PSED). The guidance gives practical examples of how AI systems may be causing discriminatory outcomes.

From October, the Commission will work with a cross-section of 30 local authorities and other public bodies in England and Scotland to understand how they are using AI to deliver essential services, such as benefits payments, amid concerns that automated systems are inappropriately flagging certain families as a fraud risk.

The EHRC is also exploring how best to use its powers to examine how organisations are using facial recognition technology, “following concerns that the software may be disproportionately affecting people from ethnic minorities”.

These interventions will improve how organisations use AI and encourage public bodies to take action to address any negative equality and human rights impacts, the EHRC suggested.

Marcial Boo, chief executive of the EHRC, said: “While technology is often a force for good, there is evidence that some innovation, such as the use of artificial intelligence, can perpetuate bias and discrimination if poorly implemented.

“Many organisations may not know they could be breaking equality law, and people may not know how AI is used to make decisions about them."

Boo added: “It’s vital for organisations to understand these potential biases and to address any equality and human rights impacts.

“As part of this, we are monitoring how public bodies use technology to make sure they are meeting their legal responsibilities, in line with our guidance published today. The EHRC is committed to working with partners across sectors to make sure technology benefits everyone, regardless of their background.”

The new guidance advises organisations to consider how the PSED applies to automated processes, to be transparent about how the technology is used and to keep systems under constant review.