Keeping the Workflow Safe: Failure, Oversight and Records

A workflow that looks sound on a diagram can fail in use. This lesson covers the practice that keeps it sound: designing for failure, routing exceptions, controlling the instructions, documenting the steps, making oversight real, logging and limiting access. Most of the common failures come from one cause. Someone assumed that another person, another team or the tool itself would catch the problem.

Design for failure

Output will sometimes be wrong, sometimes be missing a required part and sometimes not arrive because the tool is unavailable. For each case the workflow needs a defined response and a named person who acts on it. A manual fallback should be written down so staff can finish the task and know when to switch. A completeness check before review can block an incomplete draft and return it to an owner, which is better than hoping a busy reader notices a missing section. The tool should not certify its own output.

Route exceptions to a named owner

Cases will arrive that do not fit. They should go to a named role with authority to decide that type of exception, and the decision should be recorded: who decided, on what basis and when. Sending the case to whoever is free confuses availability with authority. A time limit and a named deputy stop exceptions from stalling. Allowing the tool to relabel an awkward case as standard hides exactly the cases that most need a person.

Control the instructions

The instructions that drive the tool shape every output. Treat them as a controlled document with an owner, a version number, a change history and an approval step. Test a new version on routine, unusual and incomplete cases before it goes live. If anyone can edit the wording and nobody records the change, the organisation cannot explain why outputs shifted. If something has gone wrong in this way, restore the last approved version first, then set ownership and approval.

Document so another team can follow

The documentation should state which steps the tool drafts and which a person decides, what data may enter, what each human check looks for, who may override and what to do when output fails. A good test is to ask a different team to run the workflow from the document alone and to note every question they have to ask.

Make oversight something a person can exercise

The EU AI Act sets out what people assigned to oversee high-risk systems must be able to do. They must be able to understand the system's capacities and limitations, remain aware of the tendency to over-rely on its output, interpret that output correctly, decide not to use it or to disregard, override or reverse it, and intervene or stop the system. Deployers must assign oversight to people with competence, training and authority. The practical meaning for any workflow is the same: a reviewer needs time, understanding and the standing to say no without penalty. A queue target that leaves no time to read, or a rule that every override needs a senior signature, makes oversight nominal.

Over-reliance is a design problem as well as a training one. Show the source records beside the draft, ask for confirmation of specific facts, and let reviewers form a view before the tool's recommendation appears. If nearly every draft is approved, do not read that as proof of accuracy. Place known errors in the queue from time to time and see whether they are caught.

Log enough to reconstruct a decision

The AI Act requires high-risk systems to record events automatically over their lifetime, and deployers must keep the logs under their control for at least six months. Those duties apply only where a system is high-risk and the obligations have come into effect. As an operating principle for any workflow, keep what you need to rebuild a decision: the input reference, the tool and instruction versions, the output and what the reviewer did. A copy of the final letter is not enough.

Limit access and design the handovers

Give the tool its own account with the narrowest permissions the workflow needs, read-only where possible, owned by a named person and removed when the workflow ends. Where work passes between steps or teams, state what is handed over, what the sender has checked and what the receiver must still check. Two teams each assuming the other has checked the facts is a classic cause of an error that reaches a customer.

A worked example

Wexcombe Credit Union uses a tool to draft arrears letters for member-services officers. After a complaint, the operations lead finds that nobody can say which wording version produced the letter, the reviewer's changes were not kept, and the tool's shared login can open the whole member database. She restores the last approved instructions and appoints an owner. She adds a log of instruction version, record reference, draft and officer action. She gives the tool a scoped read-only account. She gives officers time to read each letter and written authority to reject drafts without penalty. Hardship cases go to a named team leader, whose decision is recorded. Each of these changes assigns a person to a gap that had been left to assumption.

As of 24 September 2026.