Worked example: explaining a declined application

Explainability and transparency · 7 min read

Lesson 39 of 60
In this module

A health insurer uses a model to support underwriting decisions on individual policies. An applicant is declined. She writes in and asks why. Follow what the organisation can and cannot say.

What exists

Three artefacts. A model score of 0.31 against a threshold of 0.45. A generated paragraph, produced by a separate language model given the application and the score, that reads: "The application was declined principally on the basis of the disclosed condition history and the treatment interval, which together indicate elevated near-term claim likelihood." And a case note from the underwriter saying "score below threshold, declined per policy".

The first answer the team drafts

Somebody proposes sending the paragraph. It is clear, specific and sympathetic in tone. It is also the wrong thing to send, for a reason that has nothing to do with its tone.

The paragraph was generated from the application and the score. It was not derived from the scoring model. Nothing in the process ties its three stated factors to what actually moved the score. It may be right. Nobody in the organisation can say whether it is, and that is the disqualifying fact.

If the insurer sends it as the reason, it has told an applicant something about its own decision process that it cannot support. Should she exercise her right of access under Article 25(2) revFADP, which covers information on the logic of the decision, the organisation will have to either repeat the claim or contradict its own letter.

What the organisation can actually say

Working from the score and the model documentation rather than the paragraph:

  • The application was assessed with the support of an automated scoring model, and the score fell below the threshold at which the insurer offers cover on standard terms.
  • The inputs the model received were these, listed.
  • For this applicant, the inputs that moved the score furthest below the threshold were these, with direction, taken from the model rather than from generated text.
  • An underwriter reviewed the case and applied the declination.
  • She may state her position, and may request review by a natural person.

Notice that this is less fluent and considerably more useful. It names inputs with direction, which she can dispute factually. It separates the model's output from the human application of policy. And the last line is not a courtesy; where a decision is based exclusively on automated processing with a significant effect, Article 21 revFADP requires the person to be told it is automated, to be allowed to state their position, and to be allowed to request review by a natural person.

The complication nobody wants

Was this decision "based exclusively on automated processing"? The underwriter looked at it. If that review was a genuine exercise of judgement with authority to depart from the score, the decision was not exclusively automated. If the underwriter's practice is to apply the threshold without ever departing from it, then the human is a formality, and calling the decision non-automated to avoid Article 21 is a position that will not survive being examined. The case note "score below threshold, declined per policy" is not encouraging on that point.

This is the same question that Module 15 asks about human oversight, arriving from a different direction. A rubber stamp is not review for Article 21 purposes and is not oversight for Article 14 purposes. The two provisions are independent and both fail on the same facts.

The classification point

Risk assessment and pricing in relation to life and health insurance is listed in Annex III at 5(c), which makes this use high-risk under the AI Act. The Digital Omnibus deferred standalone Annex III high-risk obligations to 2 December 2027, so the timing of those duties is not what it was; the Swiss data protection duties above are in force now and do not wait. Annex III 5(c) deployers also fall within the narrow scope of the Article 27 fundamental rights impact assessment.

What to take away

The insurer's exposure did not come from having a model. It came from having a fluent paragraph and no discipline about what it was. Fluency is the risk, because it is what stops people asking.

As of 24 September 2026.

Sign in to save your progress

You can read every lesson without an account. Signing in keeps your place and unlocks the assessment.