Product Design, Manufacturing & Innovation Resources
Home » Fairness Impossibility Theorem (machine learning)

Fairness Impossibility Theorem (machine learning)

2016
  • Jon Kleinberg
  • Sendhil Mullainathan
  • Manish Raghavan
Team of data scientists analyzing fairness metrics in machine learning.

(generated image for illustration only)

In fair machine learning, impossibility theorems demonstrate that it is mathematically impossible for an algorithm to satisfy multiple, seemingly intuitive fairness criteria simultaneously, except in trivial cases. For example, an algorithm cannot generally satisfy both demographic parity (equal positive rates across groups) and equalized odds (equal true positive and false positive rates across groups) if the base rates differ between groups.

The fairness impossibility theorem highlights a fundamental tension in defining and achieving fairness. It stems from the mathematical relationships between different fairness metrics. For instance, ‘demographic parity’ requires that the probability of a positive outcome is the same across different protected groups. ‘Equalized odds’ requires that the true positive rate and false positive rate are equal across groups. ‘Predictive parity’ (or calibration) requires that for a given prediction score, the probability of a true positive outcome is the same across groups.

The theorem, articulated by Kleinberg and others, proves that unless the prevalence of the positive outcome is equal across all groups (a rare occurrence in reality) or the classifier is perfect, these three metrics cannot all be satisfied at once. This forces practitioners and policymakers to make a choice about which definition of fairness is most appropriate for a given context, acknowledging the inherent trade-offs. For example, prioritizing demographic parity might lead to less accurate predictions for all groups, while prioritizing predictive parity might result in different selection rates. This discovery shifted the conversation from finding a single ‘fair’ algorithm to understanding and navigating the landscape of fairness trade-offs.

UNESCO Nomenclature: 1203
– Computer science

Type

Abstract System

Disruption

Incremental

Usage

Widespread Use

Precursors

  • Arrow’s impossibility theorem in social choice theory
  • developments in statistical learning theory
  • early work on disparate impact in legal studies
  • foundational concepts of probability and statistics

Applications

  • development of fairness-aware machine learning frameworks
  • auditing tools for AI systems
  • policy-making and regulation for AI ethics
  • design of context-specific fairness definitions in credit scoring and hiring algorithms

Patents:

NA

Potential Innovations Ideas

Due to scrapping bot traffic, currently more than 40k per day, this content is reserved to community members.
> Login < or > Register < (100% free) to access this, so as all other restricted content and tools.

Related to: fairness, impossibility theorem, machine learning, algorithmic bias, demographic parity, equalized odds, predictive parity, AI ethics, trade-off, COMPAS.

Historical Context

1993
1998
2010
2016
1993
1997-04-23
2001
2010
2020

(if date is unknown or not relevant, e.g. "fluid mechanics", a rounded estimation of its notable emergence is provided)

Related Invention, Innovation & Technical Principles

Full size images and downloads are only available, 100% free, for registered members.

> Login <