Using AI to govern AI

Governance work is document-heavy, repetitive and behind schedule, which makes it an obvious candidate for AI assistance. Drafting risk assessments, classifying systems against Annex III, summarising vendor documentation, reviewing logs for anomalies, mapping controls between frameworks: all of it is being done with models now.

Some of this is sensible. Some of it quietly removes the thing governance was for.

The structural problem

The purpose of a governance record is to let someone who was not there reconstruct what was decided and why. That is what technical documentation is for under Article 11 and Annex IV, what instructions for use are for under Article 13(3), and what makes a decision defensible a year later when the people have moved on.

Now consider a risk assessment written by a language model. It is fluent, complete and structurally correct. It reads exactly like one a careful analyst wrote. The reader cannot distinguish them, and neither can the analyst six months later.

The problem is not that the model is wrong. It is that the document no longer carries the information it was supposed to carry. A human-written risk assessment is evidence that a person thought about the risks, because the writing was the thinking. A generated one is evidence that a person asked for a risk assessment. Those are very different facts, and only one of them is what a regulator, an auditor or a successor is looking for.

So the question to ask of any AI-assisted governance artefact is: does this still evidence that someone thought about this? Where the answer is no, you have produced a document that looks like assurance and is not.

Opacity governing opacity

The sharper version arises when a model assesses another model. A classifier that reads system descriptions and assigns risk categories. A monitor that reviews model outputs for anomalies. A tool that reads a vendor's documentation and reports whether it meets Article 13(3).

The difficulty is that the governing system has the same properties that made the governed system need governing. It infers rather than applying a stated rule. Nobody can say exactly why it produced a given answer. It fails in ways that are hard to predict and does so most in unusual cases, which are precisely the cases governance exists for.

Two specific failure modes follow. Correlated blind spots: two systems built on similar models trained on similar data tend to miss similar things, so the monitor is least likely to catch the error when the monitored system is most likely to make it. And a false sense of coverage: a monitoring model produces a steady stream of green results, which reads as evidence that nothing is wrong, when it may equally be evidence that the monitor is not detecting what is wrong.

There is also a plain regulatory point. If a model performs risk classification for high-risk systems, then that model is being used in the service of a safety function, and it needs its own entry in the register, its own oversight, and its own account of what it is bad at. Governance tools are not outside governance.

Where AI helps without hollowing anything out

• Finding things. Searching contracts for AI clauses, log volumes for patterns, documentation for the sections that matter. Retrieval is checkable: a person verifies what came back.
• First drafts of structure. A skeleton with the headings a document needs, filled by a person. The judgement stays human; the formatting does not.
• Consistency checks. Comparing a completed assessment against a template to find missing sections. Mechanical and verifiable.
• Translation and summarising for an audience, where the source document is available and a person can check the summary against it.

The pattern across these is that the model's output is verifiable against something that exists, and the verification is cheap enough that it actually happens.

The rule worth carrying

Use AI to find and to format. Do not use it to conclude. The moment a model supplies the conclusion in a governance artefact, the artefact stops evidencing the thing it was created to evidence, and you will not be able to tell by looking at it.

As of 24 September 2026.