COMPAS AI Bias: The Sentencing Algorithm That Sparked a Reckoning

In 2016, ProPublica published an investigation that changed how the world thinks about AI. Their finding: COMPAS — a risk-assessment algorithm used in US courts to predict whether defendants would reoffend — was biased against Black defendants. It falsely flagged Black defendants as future criminals at nearly twice the rate of white defendants. The story ignited the modern debate over what AI bias is and why it matters — because here, the stakes weren’t ad clicks. They were years of people’s lives.

Table of Contents

• The Short Version
• What COMPAS Is
• What ProPublica Found
• The Company’s Defense
• Why Both Sides Were (Partly) Right
• The Impossible Math of Fairness
• What Changed Afterward
• Frequently Asked Questions

The Short Version

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), built by Northpointe (now Equivant), scores criminal defendants on their likelihood of reoffending. Judges in several states used these scores in sentencing and parole decisions. ProPublica’s 2016 analysis of 7,000 cases in Broward County, Florida, found the algorithm falsely labeled Black defendants as high-risk at roughly twice the rate of white defendants — while white defendants were more often wrongly labeled low-risk. The company disputed the methodology but the case became the defining example of algorithmic bias in criminal justice.

What COMPAS Is

COMPAS is a questionnaire-based risk assessment tool. Defendants (or the system, using records) answer 137 questions covering criminal history, employment, education, substance abuse, and social environment. The algorithm processes these into risk scores — typically 1 to 10 — for general recidivism, violent recidivism, and pretrial misconduct.

Courts adopted it with good intentions. Human judges are inconsistent — sentencing varies wildly by judge, time of day, and jurisdiction. An algorithm promised objectivity: the same inputs, the same score, every time. Several states integrated COMPAS scores into sentencing guidelines and parole decisions, affecting thousands of cases.

Importantly, the questionnaire doesn’t ask about race. Northpointe has always emphasized this. The bias ProPublica found emerged anyway — which is precisely what made the case so instructive about how AI models are trained and how bias enters through proxies and history rather than explicit prejudice.

That’s the uncomfortable mechanism: the questions function as proxies. Zip code correlates with race in segregated cities. Family criminal history reflects who was policed, not who committed harm. Employment gaps track discrimination in hiring. None of these inputs mentions race, yet together they encode it — a phenomenon researchers call redundant encoding. This is why removing the race field never fixes algorithmic bias: the information survives in the correlations. COMPAS is the canonical case study, but the mechanism repeats in hiring tools, lending models, and facial recognition. And crucially, the pattern wasn’t discovered by the company — it took external journalists with the data and a two-year follow-up to surface it, a reminder that voluntary self-auditing rarely catches what independent scrutiny does.

What ProPublica Found

The investigation, led by Julia Angwin, analyzed COMPAS scores assigned to more than 7,000 people arrested in Broward County, Florida, in 2013-2014, then tracked who actually reoffended over two years. The key findings:

False positives skewed by race. Among defendants who did not reoffend, the algorithm had labeled Black defendants as high-risk at a rate of about 45%, versus about 23% for white defendants. Nearly double. These are people who would not have reoffended, branded high-risk anyway.

False negatives skewed the other way. Among defendants who did reoffend, white defendants were more likely to have been labeled low-risk (about 48%) than Black defendants (about 28%). The errors ran in opposite directions by race.

Overall accuracy was similar. Here’s the twist: the algorithm was roughly equally “accurate” for both groups — around 60-65%. The types of errors differed dramatically. This is the canonical demonstration that aggregate accuracy hides disparate harm.

ProPublica’s conclusion: the system was biased against Black defendants, producing systematically harsher predictions for them. The findings were published with the underlying data and methodology, inviting scrutiny — which came, forcefully.

The Company’s Defense

Northpointe pushed back hard, and their counterargument deserves a fair hearing because it reveals something deep:

Calibration. The company argued that a score of 7 meant the same thing regardless of race — defendants scored 7 reoffended at similar rates whether Black or white. The scores were “calibrated”: equally predictive across groups. By this metric, the tool was fair.

Different base rates. Black defendants in the dataset reoffended at higher rates overall — a reflection of policing patterns, socioeconomic conditions, and historical over-policing, not of the algorithm. Given different base rates, the company argued, the error-rate disparities ProPublica found were mathematically inevitable.

Methodology disputes. Northpointe questioned ProPublica’s two-year cutoff, their handling of certain offense types, and their statistical choices.

The company’s core claim: our tool predicts equally well for everyone; the disparities reflect unequal realities, not an unfair algorithm.

Why Both Sides Were (Partly) Right

Here’s the uncomfortable truth that makes COMPAS the most important case in AI fairness: both sides’ math checked out. ProPublica was right about the error rates. Northpointe was right about calibration. The dispute wasn’t about arithmetic — it was about which definition of fairness should win.

This isn’t a matter of opinion. It’s a proven mathematical result: when two groups have different base rates of the outcome you’re predicting, you cannot simultaneously have equal false-positive rates and equal calibration across groups. It’s impossible — not difficult, impossible. Researchers formalized this shortly after the COMPAS debate, and it’s now one of the foundational theorems of algorithmic fairness.

So the real question was never “is COMPAS biased?” in some objective sense. It was: which fairness criterion should govern a sentencing tool? ProPublica prioritized equal error rates — the system shouldn’t wrongly brand one group as dangerous more often. Northpointe prioritized calibration — a score should mean the same thing for everyone.

Neither criterion is obviously wrong. But they conflict, and choosing between them is a moral and political decision, not a technical one. The algorithm can’t resolve it. Only humans — legislators, judges, the public — can.

The Impossible Math of Fairness

The COMPAS case taught the field its hardest lesson, worth stating plainly:

When base rates differ between groups — and in criminal justice, they do, for reasons rooted in history and systemic inequality — no risk-assessment tool can be fair by every definition at once. You must choose which unfairness to tolerate:

• Equal error rates (ProPublica’s standard) means accepting that identical scores mean different things for different groups.
• Calibration (Northpointe’s standard) means accepting that one group bears more false “high-risk” labels.

There’s no third option that satisfies both. This impossibility result now underpins AI regulation worldwide: the EU AI Act, for instance, doesn’t demand perfect fairness from high-risk systems — it demands documented trade-offs, human oversight, and the ability to explain which definition was chosen and why.

The deeper issue the math obscures: the inputs themselves reflect a biased system. Arrest records measure policing as much as crime. A model trained on arrest data learns where police look, not where crime happens. Even a mathematically “fair” algorithm built on unfair data reproduces unfairness — it just does so with cleaner statistics.

What Changed Afterward

Policy. The COMPAS debate directly shaped AI regulation. The EU AI Act classifies criminal-justice AI as high-risk, requiring bias testing and human oversight. Several US jurisdictions restricted or abandoned algorithmic sentencing tools.

Practice. Courts became more cautious. Some states added requirements that risk scores supplement — never replace — judicial judgment, with explicit warnings about the tools’ limitations.

Research. The case launched algorithmic fairness as a major field. The impossibility theorems, new fairness metrics, and auditing methods developed since all trace partly to this debate.

The tool itself. Equivant continues to sell COMPAS, maintaining it’s a valid instrument. Critics maintain the fundamental problems persist. The argument never fully resolved — because, as the math shows, it can’t be resolved technically.

Public awareness. Before COMPAS, “algorithmic bias” was an academic phrase. After, it was a front-page concept. The case did more than any other single event to establish that AI systems need scrutiny proportional to their stakes — and that in criminal justice, the stakes are liberty itself. Every subsequent AI bias investigation inherits its vocabulary and its central tension: the tool is only as fair as the history it learned from.

Frequently Asked Questions