Claims triage, part two: monitoring, incident, audit and retirement

The system from part one is live. What follows is module 9 applied to it, plus the artefact the market does not produce and most programmes forget.

Monitoring: indicators with actions attached

The monitoring specification separates KPIs from KRIs, as module 9 requires. Straight-through rate, agreement between routing and outcome, time to settlement by route and the reopened-case rate for fast-tracked claims are performance. The last catches the failure that looks like success: routing more claims to fast-track improves every other number.

The risk indicators are leading. Input distribution shift on the top features. Handler override rate with both a ceiling and a floor, because a floor detects handlers who have stopped looking, which is the automation bias Article 14(4) names. Claims scored with missing fields. Age of the last fairness test, actioned at eleven months. Complaint volume on routed claims. Days on which the production model version differs from the validated version.

Each carries a defined action and an owner, agreed before go-live. Article 26(5) expects a deployer identifying a risk to suspend use and inform the provider, and a pre-agreed suspension route makes that executable on a Tuesday afternoon rather than at the next committee. The Article 72 post-market monitoring plan is written to documentation standard because it forms part of the Annex IV technical documentation, and it is versioned with the pack. The insurer is a deployer, so the plan also specifies what it sends upstream: the vendor's monitoring cannot work without a deployer channel, and the module 8 contract clauses are what make that channel exist.

The incident

Five weeks after launch, a validation analyst notices that health claims from one postal region are routed to investigation at three times the comparable rate. The region correlates with a protected characteristic.

The severity taxonomy, written a year earlier, maps this to the third limb of Article 3(49): infringement of obligations under Union law intended to protect fundamental rights. That mapping takes minutes because the work was done in advance.

The analyst's message is recorded as the moment reasonable likelihood of a causal link was established, and the fifteen-day Article 73 clock runs from it rather than from the conclusion of the investigation. Operations suspends automatic routing for the affected segment and reverts to manual assessment. Compliance files an initial report inside the deadline, stating what is known and what is not, with a complete report to follow, which Article 73 expressly permits. The Swiss data protection question runs in parallel on its own footing, and the runbook does not import a 72-hour deadline that does not exist in Swiss law.

The cause is an address-matching step in the extraction component mislabelling claims as incomplete. It is a data quality failure rather than a model failure, and monitoring the model would never have found it, which is the point module 6 made about testing the system rather than the model.

Audit readiness

The supervisory review probes the four things module 9 set out. Does the documentation match the version in production. Were thresholds set before results. Were findings acted on. Can the people named in the oversight plan describe what they do.

The insurer answers the first with a release gate that blocks promotion without an updated technical pack and a recorded link to the dataset versions behind it. The second with a threshold register dated March. The third with the incident record and a fairness finding investigated and partly accepted with stated reasons, which reads better than a clean sheet. The fourth by naming a person rather than a role and holding recorded competence for them.

Independence is structural rather than declared: FINMA Guidance 08/2024 expects independent review separate from the developing unit, and SR 26-2 carries forward effective challenge and validation covering conceptual soundness, ongoing monitoring and outcomes analysis. The insurer validated the vendor's model on its own claims population against its own pre-declared thresholds, which module 8 established is the expectation under both SR 26-2 and the NAIC bulletin.

Three artefacts had to be produced in-house because our vendor review found no commercial governance platform that generates them: the Statement of Applicability, the AI incident report, and the decommissioning record.

Retirement, which is where the obligations outlive the system

Two years later the system is replaced. The decommissioning record closes the register entry without deleting it, stating the reason and the decision-maker, the dates of last use and shutdown, what replaced it and whether the replacement inherited the same obligations, which decisions remain live or appealable and who answers for them, and which logs, models, prompts and datasets are retained, where, for how long, under which provision, and who deletes them at the end.

The retention periods are the ones module 7 collected: at least six months for logs under Articles 19 and 26(6), three years for compliance records under Colorado SB 26-189 where it applies, and the insurer's own sector record-keeping rules, usually longer.

Both failure directions are common. Deleting everything at shutdown destroys the ability to answer a complaint about last year's decision. Keeping everything forever, particularly personal data in logs, is an open-ended data protection exposure. The record forces the decision per artefact, in writing, before the system goes off.

The full artefact set, in one list

Register entry. Classification record. Impact assessment and the DPIA it cross-refers to. Data provenance record and dataset datasheet. Fairness test record. Accuracy and robustness test record. Human oversight plan. Disclosure record, covering the Article 50 duties that applied from 2 August 2026 and were not deferred. Third-party assessment and the contract clauses relied on. Logging and retention specification. Post-market monitoring plan and incident log. Conformity file and Statement of Applicability. AI policy and literacy record. Decommissioning record.

Fourteen artefacts for one system. That is the honest number, and it is why the work is sequenced rather than assembled. Every one is cheap to maintain forward and impossible to reconstruct backward, which is the single thing this course has been about.

As of 24 September 2026.