The news: UK regulators have signaled that they will clamp down on artificial intelligence (AI) use in banking that might be used to discriminate against people, per the FT.
Banks which use AI to approve loan applications must be able to prove the tech will not worsen discrimination against minorities.
The bigger picture: AI is a significant growth area in banking. Its market size is projected to soar globally from $3.88 billion in 2020 to $64.03 billion in 2030, with a CAGR of 32.6%, per a Research and Markets report.
AI in banking is maturing, and as data analysis improves, it brings the potential for more accurate decision-making. But concerns about misuse have led to heightened regulatory scrutiny:
- US: Earlier this month, the Consumer Financial Protection Bureau (CFPB) warned it would get tougher on AI misuse in banking. CFPB Director Rohit Chopra cautioned that AI could be abused to advance “digital redlining” and “robo discrimination.” The chairs of two US congressional committees last year asked regulators to ensure the country’s lenders implemented safeguards ensuring AI improved access to credit for low- and middle-income families and people of color.
- Europe: EU regulators last week urged lawmakers to consider “further analysing the use of data in AI/machine learning models and potential bias leading to discrimination and exclusion.”
- UK: The Office for AI will release its white paper on governing and regulating AI in early 2022. This could lead to a shift from the government’s current sector-led approach to blanket AI-specific regulations.
The problem: Banks using AI need to be aware of the risks that come with the technology:
- The complexity of AI models can create a “black box” problem in which decisions are made with very little transparency regarding how they reached their conclusions, making accountability and error detection a challenge.
- Baked-in biases that are difficult to root out are another risk. For example, Apple and Goldman Sachs found themselves in hot water in 2019 over claims that technology used to measure creditworthiness might be biased against women.
- Unintended biases in AI models can arise from flawed training data. AI algorithms that are trained on incomplete, biased, or extraneous data can yield judgments that are biased, causing a range of issues, including inadvertent discrimination.