Product Design, Manufacturing & Innovation Resources

Home » Algorithmic Confounding

Algorithmic Confounding

2020

Sharad Goel
Ravi Shroff
Jennifer Skeem
Christopher Slobogin

(generated image for illustration only)

Algorithmic confounding occurs when a proxy variable used by an algorithm is correlated with a protected attribute (like race or gender) and also with the outcome of interest. The algorithm may inadvertently learn to discriminate based on the protected attribute by using the proxy, even if the protected attribute itself is explicitly excluded from the model’s input data.

Algorithmic confounding is a subtle but powerful source of bias. It arises because machine learning models are exceptionally good at finding statistical correlations, even spurious ones. While a developer might remove a sensitive feature like ‘race’ to prevent discrimination, the model can latch onto other features that act as proxies. A classic example is the use of ZIP codes in loan applications. Due to historical residential segregation, ZIP codes can be highly correlated with race. An algorithm might learn that applicants from certain ZIP codes are higher risk, not because of their location, but because the location is a proxy for a racial group that has historically been denied loans, creating a feedback loop of discrimination.

This is distinct from traditional statistical confounding because the algorithm isn’t just being misled; it’s actively learning a discriminatory policy from the data. Identifying and mitigating this requires more than just feature removal. It often involves causal inference techniques to understand the true relationships between variables, or the use of fairness-aware algorithms that can be constrained to ignore the influence of known proxies. The challenge lies in the fact that almost any variable can be a proxy to some extent, making complete elimination difficult.

Algorithms, Artificial Intelligence (AI), Machine Learning, Risk Management

UNESCO Nomenclature: 1203

– Computer science

Type

Abstract System

Disruption

Incremental

Usage

Widespread Use

Precursors

concept of confounding variables in statistics and epidemiology
legal doctrine of disparate impact
research on redlining and housing discrimination
development of machine learning classification algorithms

Applications

auditing of pre-trial risk assessment tools like COMPAS
development of proxy-aware bias detection methods
design of fair credit scoring models that avoid redlining proxies
improving fairness in automated hiring systems by identifying and mitigating confounding variables

Patents:

Potential Innovations Ideas

Due to scrapping bot traffic, currently more than 40k per day, this content is reserved to community members.
> Login < or > Register < (100% free) to access this, so as all other restricted content and tools.

Related to: algorithmic confounding, proxy variable, disparate impact, algorithmic bias, machine learning, fairness, redlining, protected attributes, indirect discrimination, causal inference.

Historical Context

Computer workstation with R programming interface and statistical graphs in software engineering.

The Comprehensive R Archive Network (CRAN)

CRAN is the primary repository for the R software, its documentation, and thousands of user-contributed extension packages. It is a network of FTP and web servers around the world that store identical, up-to-date versions of R code and documentation. This centralized, yet distributed, system is fundamental to R's ecosystem, ensuring easy access and reproducibility for users globally.

Agile Project Management

Agile project management is an iterative approach to delivering a project throughout its life cycle. It breaks down large projects into smaller, manageable tasks completed in short iterations or 'sprints.' This allows for frequent reassessment, adaptation of plans, and flexibility in response to change. It prioritizes customer collaboration, working software, and responding to change over comprehensive documentation and rigid plans.

Data scientists collaborating on bias mitigation techniques in artificial intelligence.

Bias Mitigation Processing Stages

Algorithmic bias mitigation techniques are categorized into three main stages relative to the model training process. Pre-processing methods modify the training data itself (e.g., reweighing, resampling). In-processing methods incorporate fairness constraints directly into the model's learning algorithm. Post-processing methods adjust the model's predictions after they have been made to improve fairness.

Algorithmic Confounding

1997-04-23

2001

2010

2020

1993

1998

2010

2016

Usability testing lab with participants evaluating digital interfaces in human-computer interaction.

Nielsen’s Five Components of Usability

Jakob Nielsen, a prominent usability consultant in UI and webdesign mainly, defined usability through five quality components: Learnability (how easy is it for users to accomplish basic tasks the first time?), Efficiency (how quickly can they perform tasks once learned?), Memorability (can users reestablish proficiency after a period of not using it?), Errors (how many errors do users make?), and Satisfaction (how pleasant is it to use?).

Usability testing lab with users evaluating software applications in human-computer interaction.

ISO 9241-11 Definition of Usability

The international standard ISO 9241-11 defines usability as the "extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use." This definition provides a framework for measuring usability by breaking it down into three distinct, quantifiable components, moving beyond purely subjective assessments.

R programming workspace with Tidyverse data analysis tools and ggplot2 visualizations.

The R Tidyverse Ecosystem

The Tidyverse is a collection of R packages designed for data science that share an underlying design philosophy, grammar, and data structures. Developed by Hadley Wickham and others, it provides a consistent and powerful toolkit for data import, tidying, transformation, visualization, and modeling. Key packages include `ggplot2`, `dplyr`, `tidyr`, and `readr`, which compose together using pipes.

Team of data scientists analyzing fairness metrics in machine learning.

Fairness Impossibility Theorem (machine learning)

In fair machine learning, impossibility theorems demonstrate that it is mathematically impossible for an algorithm to satisfy multiple, seemingly intuitive fairness criteria simultaneously, except in trivial cases. For example, an algorithm cannot generally satisfy both demographic parity (equal positive rates across groups) and equalized odds (equal true positive and false positive rates across groups) if the base rates differ between groups.

(if date is unknown or not relevant, e.g. "fluid mechanics", a rounded estimation of its notable emergence is provided)