Kearns, Michael

professor of computer science at the University of Pennsylvania; co-author of "The Ethical Algorithm"

Michael Kearns: Algorithmic Fairness, Privacy & Ethics

Lex Fridman Podcast · Nov 19, 2019

• Michael Kearns
• algorithmic fairness
• differential privacy
• fairness gerrymandering
• Pareto curve trade-offs
• data anonymization
• no-regret learning
• game theory
• algorithmic trading
• social media engagement optimization

Core ideas

Kearns frames his and co-author Aaron Roth's approach to algorithmic ethics as narrowly technical rather than philosophical: rather than resolving deep questions of what fairness ultimately is, their goal is to show that once a society or stakeholder group agrees on a specific, precise definition of fairness or privacy, that definition can often be directly encoded as a constraint or term in an algorithm's objective function — analogous to how computer science already reasons about trade-offs between running time and memory. He stresses this only handles "low-hanging fruit": algorithms with no fairness consideration at all can be egregiously unfair, but once you push into genuinely contested territory, you run headlong into unresolved philosophical debates (echoing centuries-old disputes, e.g., around Rawlsian justice) about who deserves protection and what constitutes harm — and notes that, unlike privacy (where the field has converged on differential privacy as a strong, widely accepted definition), algorithmic fairness is much messier, with published theorems proving that several independently reasonable fairness definitions cannot all be simultaneously satisfied.

He describes a genuine tension between group and individual fairness: standard fairness definitions typically equalize some error rate (e.g., false rejection rate on a loan) across a handful of demographic groups (race, gender, age), but enforcing fairness marginally across each such group independently doesn't prevent discrimination against specific intersectional combinations — a phenomenon he and Roth call "fairness gerrymandering," by analogy to political gerrymandering, where an aggregate guarantee is achieved by favoring some subgroups at the expense of narrower ones (e.g., disabled Hispanic women over 55 making under $50,000 a year). He notes that pushing group-based fairness definitions toward finer and finer subgroup granularity is, in the limit, equivalent to individual-level fairness (since every individual is ultimately a "group of size one"), and describes early algorithmic work that lets practitioners creep toward that finer-grained end of the spectrum, while acknowledging the field currently has no tractable way to guarantee fairness at the fully individual level. He also flags that current fairness definitions are "received wisdom" — invented by researchers rather than empirically elicited from the public — and argues the field needs much more behavioral/human-subjects research (which he expects computer scientists, as serial "imperialists" into other disciplines' territory, will initially do clumsily, citing reinforcement learning's early reinvention of decades-old control theory as a precedent) to understand what people actually consider fair.

On the politically loaded nature of fairness, Kearns observes that unlike privacy (where essentially everyone agrees more of it is better), fairness debates inevitably reopen long-unresolved societal disputes — citing affirmative action and criminal-sentencing fairness (where defining fairness purely in terms of equalized false-incarceration rates across racial groups implicitly ignores fairness to potential crime victims) as examples where reasonable people disagree on first principles, meaning researchers cannot avoid engaging in extra-algorithmic public debate about what society should even be protecting. He illustrates concretely how bias enters machine learning systems (e.g., a face-recognition system trained predominantly on white male faces will minimize overall error by sacrificing accuracy on underrepresented groups) and stresses that "fixing" this — whether by adding fairness constraints, gathering more data on underrepresented groups, or training separate models — always has real costs (accuracy, money, engineering time), even when it avoids a strict accuracy/fairness trade-off. He introduces the Pareto curve (trading off numerical error against a numerical fairness/unfairness measure) as potentially the right interface between technical researchers and the non-technical policymakers who should decide where on that curve a given system should sit — arguing that ignoring the trade-off doesn't eliminate it, it just leaves the choice made implicitly and unaccountably.

On social media and engagement optimization, Kearns argues platforms weren't deliberately engineered to polarize society — Facebook and Twitter began as curiosities whose eventual societal influence surprised even their own builders — but that since these systems are themselves large-scale machine-learning-driven optimization processes (embedding users and content in a shared metric space to maximize individual engagement), the same underlying model that predicts what will make a user happy also predicts what would make them maximally unhappy or offended, meaning platforms already possess the technical capacity to deliberately show people some content further from their preferences (e.g., via a user-controlled "diversity slider") as a partial remedy — though he's careful to note this would have real revenue costs, since engagement-optimization got platforms "where we are," and that a "different" solution isn't automatically know to be societally better just because it isn't engagement-first.

Turning to privacy, Kearns explains that conventional data anonymization (redacting names, coarsening ages/zip codes) is fundamentally broken because it implicitly assumes a given dataset is the only data that exists or ever will — a false assumption repeatedly exploited (the Netflix Prize dataset being de-anonymized by cross-referencing IMDb; a landmark study showing Facebook "likes" alone can predict sexual orientation, substance use, and parental divorce). He explains differential privacy as a fundamentally different, counterfactual guarantee: an analysis is differentially private if the outcome for any individual is nearly identical whether or not that individual's specific record was included in the dataset — meaning harms that follow from an analysis (e.g., the 1950s study linking smoking to lung cancer, which harmed smokers' insurance premiums) would have occurred anyway since the underlying pattern is discoverable from the population at large, not any one person's data. He describes the field's central technical surprise: differential privacy, despite initially appearing so strict it might block most useful computation, has proven compatible (via carefully calibrated noise injection) with nearly all major machine learning and statistical methods, from neural networks and decision trees to classical hypothesis testing — meaning practitioners largely don't have to sacrifice the benefits of data science to achieve strong, mathematically provable privacy guarantees.

On game theory, Kearns describes it as a mathematical framework for reasoning about collective outcomes among interacting self-interested agents, and highlights the deep connection between game theory and "no-regret learning" in machine learning as one of algorithmic game theory's most important results — used to explain how everyday systems like Google Maps or Waze, by computing each user's individually optimal route in response to everyone else's behavior in real time, effectively nudge an entire population toward a Nash equilibrium that can be collectively worse (in aggregate driving time) than an alternative, non-selfish coordination scheme would achieve. He's candidly skeptical of historical attempts to apply simple two-player game-theoretic models (like Cold War-era RAND Corporation analyses of US-USSR nuclear deterrence) to hugely complex, multi-actor real-world political situations, even while acknowledging that mutually assured destruction has, so far, held. On algorithmic trading, he distinguishes short-horizon "stat arb" strategies (seconds to days, where machines already dominate because they only need to track exchange-level data) from long-horizon, Warren-Buffett-style investing (years to decades, spanning recessions, wars, and shifting economic cycles), arguing the latter remains comparatively safe from full automation because it demands synthesizing many wildly different, slow-moving, qualitative sources of information that current machine learning has no strong track record of handling — a view he says was reinforced by a joint workshop he helped organize with the Federal Reserve Bank of Philadelphia on machine learning for macroeconomic prediction.

Excerpts