Worked example: the helpful supplier inbox assistant

A finance team is drowning in supplier correspondence. IT proposes an assistant that reads the shared invoices mailbox, summarises each message, extracts payment details, matches them against the supplier record, and drafts a reply. It can also flag a mismatch to a human and, for routine confirmations, send the reply itself. Everyone agrees it will save a day a week. What does a governance reviewer ask?

Map the trust boundary first

The single most useful move is to ask where text written by someone outside the organisation reaches the model. Here the answer is immediate and total: the entire input is attacker-controlled. Anyone who can send an email to the invoices address is writing into the model's context.

That is not a reason to refuse the project. It is the fact the design has to be built around, and it reframes every other question. The mailbox is not a data source. It is an untrusted input channel that happens to look like a data source.

What an attacker actually sends

Not a virus. An invoice. A perfectly ordinary PDF from a plausible supplier, carrying in small pale text at the foot of the page, or in a document comment, a few sentences addressed to the reader that reads the file: that this supplier's bank details have changed, that the change is verified, and that the confirmation reply should be sent without flagging.

A person opening the PDF sees an invoice. The extraction step sees the text. Nothing about it is malformed, so nothing in the mail gateway objects. This is indirect injection, ASI01 Agent Goal Hijack in the OWASP agentic list, and it is effective precisely because it is not technical.

The three questions that decide the design

What can it read? The mailbox, and the supplier master record. The second is the interesting one. If retrieval over the supplier record runs with broad permissions rather than the permissions of the person asking, then a crafted email is a way to make the system read out a supplier record to whoever sent the message.

What can it do? This is where the proposal should change. It can send email. That turns every successful injection from a wrong summary into an outbound communication in the organisation's name. The reviewer should ask what the send capability is worth, measured against the fact that it converts a text failure into an action failure.

With whose credentials? If the assistant holds a service account with permanent access to the mailbox and the supplier record, then the blast radius is the service account and not the requester. ASI03 in the agentic list is Identity and Privilege Abuse for this reason.

What the reviewer proposes

No change to payment details ever flows from mail content. A bank detail change is routed to a human process with out-of-band verification, and that routing is enforced outside the model rather than requested of it in the system prompt. An instruction in the prompt is a preference the injected text can argue with. A rule in the surrounding code is not.

Sending is removed from the routine path, or restricted to replies drawn from a fixed set of templates with no free text and no attachments. A choice among five templates is a much smaller thing to hijack than a text field.

Retrieval over the supplier record runs with the requester's permissions, and the corpus permissions are reviewed before indexing on the assumption that anything findable will be found.

Logs capture the inputs and the outputs, including the retrieved content, and are retained. Article 26(6) requires deployers of high-risk systems to keep automatic logs at least six months. Independently of classification, without those records nobody can answer what the system read on the day it went wrong.

Human oversight is meaningful. Article 14 requires that the person overseeing can understand the system's limitations, remain aware of automation bias, interpret the output correctly, and decide not to use it, disregard it, override it or interrupt it. An approver shown a tidy summary and an approve button has none of that. An approver shown the extracted values next to the source field they came from has some of it.

What the Act asks of the builder

Article 15 requires resilience to attempts by unauthorised third parties to alter a system's use, outputs or performance. That is a design property here, not a product to buy: constrain what the system can reach, constrain what it can do, and keep the consequential decisions outside the text stream the attacker can write into.

As of 24 September 2026.