Leakage through the output, not through the door

The second failure mode is quieter. Nothing is stolen, nothing is breached, and confidential information still reaches someone who should not have it. It leaves through the feature working exactly as designed.

Retrieval inherits whatever permissions were set up badly

Most useful internal assistants work by searching a body of documents and answering from what they find. The security of that arrangement rests entirely on the search step respecting the permissions of the person asking.

That is harder than it sounds, and it fails in a specific way. Organisations accumulate documents whose access controls are wrong but harmless, because nobody could find them. A salary spreadsheet in a forgotten folder, minutes left in a shared drive, an old export. Obscurity was doing the work of access control, quietly, for years.

A retrieval system removes obscurity in a single afternoon. It finds everything it is pointed at and summarises it in response to a plain question. Nobody attacked anything. Somebody asked what the leadership team discussed about restructuring, and the system answered helpfully.

The question to ask before such a tool is switched on is not whether it is secure. It is whether the permissions on the indexed corpus are correct today, for every document, judged on the assumption that anyone can find anything.

Leakage between users and across sessions

Where a system keeps memory, notes about a user carried between conversations, or a shared cache, content entered by one person can surface for another. This is the concern behind ASI06 Memory and Context Poisoning in the OWASP agentic list, which covers both an attacker deliberately planting content that persists and the more mundane case of information crossing a boundary it should not have.

If a feature offers memory, the governance question is whose memory it is, how it is scoped, and how it is cleared. If nobody can answer, treat anything entered as potentially visible to other users of the same deployment.

Leakage back to the model

Separate from retrieval, there is the question of what the vendor does with inputs. This was covered under the privacy perimeter, and it belongs here too for a different reason: from a security standpoint, your input has been copied outside your control, and every property of that copy, its retention, its jurisdiction, who at the vendor can read it, is now part of your attack surface rather than theirs alone.

Leakage of the system itself

Models can be induced to reveal their own configuration: the system prompt, the names of tools they can call, the structure of the data they search. Individually this is embarrassing rather than fatal. Collectively it is reconnaissance, and it makes the next attempt sharper. Treat a system prompt as internal material, not as a secret, and design so that nothing important depends on it staying hidden.

What a non-specialist should ask

You are not expected to test any of this. You are expected to ask four questions before a feature is adopted, and to escalate if the answers are vague.

What can it read? Name the sources. If the answer is everything in the shared drive, the permissions on the shared drive are now the security of this feature.

What can it do? Text out only, or actions. Which actions, with whose credentials, and what is the limit.

Where does untrusted content enter? Any path by which text written outside the organisation reaches the model is an injection path. Email, web, uploaded files, tickets, forms.

What is logged? Article 26(6) requires deployers of high-risk systems to keep the automatic logs for at least six months, mirroring the provider duty in Article 19. Beyond compliance, logs are the only way to answer the question an incident poses, which is what did it read and what did it do. A system that acts without a record of its inputs and outputs cannot be investigated, only apologised for.

As of 24 September 2026.