Fairness in credit decision-making
The five-day Data Study Group ran from April 16 to 20, 2018, at The Alan Turing Institute, the United Kingdom’s national institute for data science and artificial intelligence, housed at the British Library in London. Data Study Groups are week-long events in which interdisciplinary teams work on real-world problems supplied by partner organizations. Here the challenge, set with Accenture, focused on fairness in machine-learning decisions about retail-bank credit allowance, a setting where model outputs may influence access to financial services. Dr. Costilla Reyes, then completing his PhD at the University of Manchester, joined the team after taking part in the institute’s December 2017 Data Study Group on extremist motivations.
He framed the week’s question plainly at the outset: if people are not fair, can the algorithms they use to make decisions at least be made fair? By the second day the team was working through what fairness means and how it can be operationalized in a machine-learning pipeline. The project mapped different definitions of fairness onto the data-science workflow and organized the fairness literature into a framework for analysis and discussion. The Alan Turing Institute published the team’s final report on February 5, 2019, listing him among its members.
Why the question matters
Credit models learn from historical lending data, so patterns of past disadvantage can be reproduced in new decisions unless fairness is defined, measured, and checked explicitly. Fairness can be defined in several ways, for example by comparing error rates or approval rates across groups, and these definitions can conflict with one another, which is why mapping them onto concrete workflow stages is a prerequisite for meaningful evaluation. The team’s framework organized those definitions and the literature around them for analysis and discussion. The December 2017 Data Study Group on extremist motivations, which preceded it, is described on its own page.