Cabinet Office launches inquiry into potential for bias in use of algorithms in local government

Government advisory body the Centre for Data Ethics and Innovation is to probe the potential for bias in the use of algorithms in local government as well as crime and justice, financial services and recruitment.

Announcing the investigation, the Cabinet Office said algorithms had “huge potential for preventing crime, protecting the public and improving the way services are delivered”, but there was “a risk that any human bias in that data will be reflected in recommendations made by the algorithm”.

It wanted to ensure those using the technology understood the potential for bias.

Algorithms could be used in crime and justice to assess the likelihood of re-offending and inform decisions about policing, probation and parole.

The Centre was set up as part of the Government’s industrial strategy on data-driven technologies and artificial intelligence.

Its chair Roger Taylor said: “We want to work with organisations so they can maximise the benefits of data driven technology and use it to ensure the decisions they make are fair.

“As a first step we will be exploring the potential for bias in key sectors where the decisions made by algorithms can have a big impact on people’s lives.”

The Centre is also to partner the Cabinet Office’s Race Disparity Unit in finding out whether algorithms introduce bias.

Mr Taylor said algorithms used in recruitment to screen CVs and shortlist candidates could potentially limit the impact of unconscious bias but “there have also been reports of such technology inadvertently exacerbating gender bias”.

Mark Smulian

See also: Predictive analysis – an introduction and considerations: Paul Feild examines the legal and other issues raised by the use by local authorities of predictive analysis, 'Big Data' and machine learning.