Worked example: one tool, four people, four duties

A mid-sized insurer rolls out an assistant that drafts customer correspondence. One tool. Follow four people through a single week and watch what each of them owes.

Marco, who drafts letters

Marco handles customer complaints. He pastes the complaint and the file summary into the assistant and edits the draft reply. He has had no AI training beyond a launch email.

What he owes. Two things, and neither requires a policy to exist. First, he is the last human before a statement reaches a customer, so he cannot let a sentence stand that he would not have written himself. The assistant will produce fluent, confident text that misstates a policy term, and fluency is not a signal of accuracy. Second, he has put a customer's complaint, which is personal data and may be health data, into a tool. Whether that is permitted is not his call to make alone, and the question needs to reach someone who can answer it.

What his organisation owes him. An answer to what may be pasted in, before he needs it rather than after. That is an Article 4 measure, and it is more useful than a course.

Priya, who approves the letters

Priya signs off correspondence on complex cases. Her target is forty letters a day, unchanged since the assistant arrived, and her manager has asked whether it can rise to sixty since drafting is faster.

What she owes. She is in the oversight position. She must understand what the assistant is bad at, read the draft against the actual file rather than for plausibility, and be willing to reject. The automation bias risk here is precise: a well-written draft is harder to disagree with than a badly written one, because the effort of finding the flaw is higher and the social cost of rejecting competent-looking work feels greater.

What she should push back on. The throughput increase. Drafting got faster; checking did not. If the target rises because the tool is faster, the saving comes out of her review time, and the process stops being oversight. Saying this is part of her role, not a complaint about it.

Dan, who bought it

Dan in operations selected the vendor and configured the deployment.

What he owes. He needs to know what the vendor states the system is for, and to have read it. He owns the boundary: this assistant is documented for correspondence drafting, and the moment someone suggests pointing it at complaint triage or at assessing which customers to prioritise, the intended purpose has changed and the Article 25 question is live. He is also the person who will be asked what the system is known to be bad at, and if the vendor documentation does not say, that is a finding to raise at procurement rather than a gap to absorb.

Anna, who owns the register

Anna runs governance for the business unit. She cannot see any of the above from her desk.

What she owes. Not omniscience. She owes a register that is easy to add to, a documented escalation route with a named destination, and training aimed by consequence rather than by seniority, which in this case means Priya gets the deepest version and the executive sponsor does not.

What she cannot do. She cannot notice that Marco is pasting health data, that Priya's target is rising, or that Dan's vendor documentation lacks limitations. Each of those is visible only to the person in the job. Her design choice is whether reporting them is cheap enough that it happens.

What the week shows

Four duties, four people, one tool, and no committee in a position to discharge any of them. Anna's function makes the duties dischargeable. It does not discharge them.

Note also that Article 4 is satisfied or breached across all four at once. A single generic module delivered to everyone would leave Priya without the system-specific training her oversight position requires, and would therefore fail the proportionality test even though everyone was trained. Effort in the wrong place is still the wrong place.

As of 24 September 2026.