Why you must choose your fairness definition in advance

There is a question people ask early and expect a short answer to: is the model fair? The uncomfortable answer is that fairness is not one measurable quantity. It is several, they disagree with each other, and outside of artificial conditions they cannot all be satisfied at once. This is arithmetic, not politics, and it is the single most useful thing a non-technical person can know about this subject.

Three things "fair" can mean

Equal selection rates. The same proportion of each group is selected, approved or shortlisted. This is what most people mean by fair when they first say the word, and it is the shape of what NYC Local Law 144 makes employers publish.

Equal error rates. Among people who would in fact have repaid the loan, the same proportion are wrongly refused in each group. Among people who would in fact have defaulted, the same proportion are wrongly approved. This is fairness to the individual who is misjudged.

Equal meaning of a score. A score of 0.7 carries the same real-world likelihood whatever group the person belongs to. This is what a lender needs in order to price consistently, and it is what the word calibration means.

Each is defensible. Each corresponds to a real complaint someone might bring. And they conflict.

Why they cannot all hold

Work through the arithmetic with numbers small enough to hold in your head. Two groups of 1,000 people each. In group A, 200 will actually default. In group B, 100 will actually default. The rates genuinely differ in the data, for reasons that may themselves be the product of history, but they differ.

Now demand calibration: a score means the same thing in both groups. If the model is well calibrated and the underlying rates differ, then the average score in group A is higher, and a single approval threshold selects a smaller share of group A. Equal selection rates are gone.

So force equal selection rates instead, by using a lower threshold for group A. Now more of group A's genuine defaulters are approved, so the false-approval rate differs between groups, and the false-refusal rate moves with it. Equal error rates are gone.

So force equal error rates. Now the same score produces different thresholds in each group, which means the score no longer means the same thing in both. Calibration is gone.

You can move the failure around. You cannot remove it, so long as the base rates differ. Anyone who tells you their system is simply fair, without naming which definition, has either chosen one without telling you or has not measured.

What follows practically

The choice is a governance decision, not a technical one, and it has to be made before the model is trained and tested, not after the numbers come back. Article 9(6) and (7) of the EU AI Act put this in law for high-risk systems: testing against metrics and probabilistic thresholds defined in advance. The reason is exactly this one. Once you have seen the results, you will pick the definition your model happens to pass, and you will believe you chose it on principle.

So write down, before anyone builds anything: which groups will we measure, which definition of fairness governs this use, what value is acceptable, and what happens if we miss it. The last one is the part people skip, and it is the only part that makes the rest binding.

Who decides

Not the data science team alone. The choice between calibration and equal error rates is a choice about who bears the cost of the model being wrong, and that is a business and legal question. FINMA Guidance 08/2024 names non-discrimination as one of its four themes, and expects roles to be clearly allocated and testing to happen before deployment with continuous monitoring afterwards. It also makes a point worth carrying: responsibility remains with the institution even where the AI is bought in or outsourced. A vendor's fairness claim is not a defence, and it is usually a claim about a definition they picked.

As of 24 September 2026.