The ICO says that discrimination due to AI could lead to job applications being rejected or the unlawful denial of bank loans or benefits.
The UK's data watchdog is to investigate whether artificial intelligence systems show racial bias when dealing with job applications.
The Information Commissioner's Office (ICO) said that discrimination due to AI could have "harmful consequences for people" and lead to someone being rejected for a job or being wrongfully denied a bank loan or welfare provision.
It will investigate the algorithms that review job applications, amid concerns that they affect employment opportunities for people from ethnic minorities.
"We will investigate concerns about the use of algorithms to screen job applications, which could negatively impact employment opportunities for those from diverse backgrounds," the ICO said.
The investigation is announced as part of a three-year plan for the ICO under the UK's new information commissioner, John Edwards, who joined the ICO in January after managing its counterpart in New Zealand.
See also: Google's DeepMind: We're close to achieving 'human-level' artificial intelligence
In a speech on Thursday, Edwards is expected to say that the ICO will “consider the impact that the use of AI could have on groups of people who are not involved in trials for this software, such as neurodeviant people or people from ethnic minorities.”.
The CEO of ZipRecruiter, a job site, told the Guardian this year that at least three-quarters of all resumes submitted for jobs in the US are read by algorithms. A survey of hiring executives conducted by research and consulting firm Gartnerlast year found that nearly all reported using AI in the hiring process – for example, screening applications before anyone sees them.
Under the UK General Data Protection Regulation, enforced by the ICO, individuals have the right not to be discriminated against when their data is processed. The ICO has previously warned that systems based on artificial intelligence could produce results that disadvantage certain groups if they are not accurately or fairly represented in the dataset on which the algorithm is trained and tested. The UK Equality Act 2010 also offers individuals protection from discrimination, whether caused by a human or an automated decision-making system.
Dr David Leslie, director of ethics and responsible innovation research at the Alan Turing, said: “The use of data-driven AI models in recruitment processes raises a number of ethical issues, which require forward thinking and careful assessment on the part of both system designers and suppliers.”
“Basically, predictive models that could be used to filter job applications through supervised machine learning techniques run the risk of reproducing or even increasing patterns of discrimination and structural inequalities that could be embedded in the datasets used to train them.”.
Elsewhere in its three-year plan, the UK watchdog will consider whether to prioritise the public interest when dealing with complaints about freedom of information requests. The ICO said changes to the complaints process for the FoI scheme – which gives members of the public the legal right to request official information from public bodies – are needed due to rising complaints and a lack of resources to deal with them.
See also: In the battle between AI and the Metaverse, CEOs choose AI
Edwards said the current system for dealing with FoI complaints – for reasons including late responses or incomplete disclosures – was not working and that he would consult on a new regime. The number of complaints reached 6,361 in the year to 30 April 2022, which was slightly down on the previous year but compared with 5,433 in 2016/17.
"I want to explore whether we should be able to highlight some cases or withdraw some and say: these need to be resolved quickly, because they are in the public interest," Edwards said.
Edwards said the organization expects to avoid accusations of bias toward what it prioritized by publishing a list of criteria for reviewing complaints.
Edwards said the ICO would consider prioritising complaints from journalists and MPs, although it would also use a “blind application” system where the identity of the person or organisation making the submission would be hidden. Information held by Scottish is covered by the country’s Ombudsman and the Freedom of Information Act.
“We need to distinguish between who is making the request… and say that a journalist is acting on behalf of the wider public to shed light on a particular issue. So the request gets extra points. A member of parliament, for example, has a very special role in holding the executive and public authorities to account, so there may be a case for adding extra weight to these requests.”.
Source: theguardian.com
