Worked example: five incidents, one quarter, five different controls

A mid-sized insurer reviews its AI-related incidents for one quarter. Five items are on the list. They look like a general problem with AI and they are not: they are five different failures, and no single control would have caught more than one of them. Sorting them is the exercise.

Incident 1: the policy memo

A compliance officer drafts an internal memo with an assistant. It cites a Swiss breach notification deadline of 72 hours. The memo circulates for three weeks before a lawyer notices.

Category: fabrication. There is no 72-hour deadline in Swiss law; Article 24 of the revised FADP requires notification to the FDPIC as soon as possible. The figure was imported from the GDPR because it is a plausible continuation after a sentence about breach notification, and plausibility is what the model optimises.

Control that would have caught it: verification of each specific claim against a source. Nothing else would. Monitoring does not look at memos, and no fairness test touches this.

Incident 2: the dental claims

A triage model performs poorly on a product line launched five months earlier. Nobody escalated, because overall accuracy on the quarterly dashboard moved by two points.

Category: distribution shift, specifically use outside the validated envelope. The model had never seen dental claims and produced confident outputs regardless, because that is all it can do.

Control: post-market monitoring under Article 72, measured per segment, with a threshold defined in advance under Article 9 that triggers a named person. A verification habit would not have helped; there was nothing to read.

Incident 3: the complaints from one canton

Customers in one region complain that applications are being declined at a noticeably higher rate. Investigation shows the model performs materially worse for applicants whose documents were submitted in a language that was sparse in the training data.

Category: bias, arising at the edge of the training distribution. The two descriptions are the same fact seen from two directions: the group was thin in the data, so the pattern learned for them rests on few cases.

Control: the Article 10 duties, examination for bias with measures to detect, prevent and mitigate it, plus per-group measurement. Note that the affected group knew months before the organisation did. That is the normal order of discovery, and a complaints channel that routes to the system owner is a detection mechanism, not just a courtesy.

Incident 4: the fast-track model that was working perfectly

A model was built to predict which claims would settle without dispute, and it does so well. It was then used to decide which claims receive scrutiny. Fraudulent claims that are quiet and well-documented settle without dispute, so the model reliably fast-tracks them.

Category: specification failure. Every technical metric is healthy. The objective the model optimises is not the purpose it is being used for, and no amount of accuracy closes that gap.

Control: stating the intended purpose precisely, and reviewing it when the use changes. Under Article 25 of the Act, changing the intended purpose can also change who counts as the provider, so this is not only a quality question.

Incident 5: the approved batch

An agent processed supplier invoices overnight. A reviewer approved the batch from the agent's own summary in two minutes. Four payments were wrong.

Category: oversight failure, of the kind Article 14(4) anticipates when it requires overseers to remain aware of automation bias. The reviewer was competent and the control was documented. The unit of review made real review impossible, and the only evidence available came from the thing being reviewed.

Control: oversight designed around a realistic unit, with irreversible actions gated separately, and independent evidence rather than a self-report.

The point of the exercise

Five incidents, five categories, five controls, and no overlap. The verification habit that caught nothing in incidents 2 to 5 is the only thing that would have caught incident 1. The monitoring plan that would have caught incident 2 is silent on the other four.

So when a system is described as tested, ask which failure category was tested for. And when a risk assessment lists a control, ask which category it addresses and what covers the other five. An organisation with one control and six failure modes is not five sixths safe; it is exposed in five specific, nameable ways, which is a much more useful thing to know.

As of 24 September 2026.