Worked example: four answers to one question

A compliance analyst at a Swiss insurer needs to answer an internal question: what must we tell an employee before we start using an AI system that monitors their work? She asks an assistant. Here are four answers she might receive, all fluent, all confident. Read each and decide what you would do with it.

Answer one: correct and checkable

"Under Article 26(7) of the EU AI Act, deployers who are employers must inform workers' representatives and the affected workers that they will be subject to the use of a high-risk AI system before putting it into service at the workplace."

This is right, and more usefully, it is right in a form you can verify in about a minute. It names an instrument, an article and a duty. You open the Act, find Article 26, and confirm it. The answer has done the thing a good answer does: it has made itself falsifiable.

Answer two: correct in substance, wrong in citation

"You must inform the affected workers in advance. This is set out in Article 29 of the AI Act."

The duty is real. The article number is not the right one. This is the most dangerous class of error, because the substance passes a sense check and the citation is exactly the part a busy reader treats as decoration. It then travels: into a memo, a policy, a slide shown to a regulator.

Notice what produced it. The model was selecting plausible continuations. "Article 29" is a plausible continuation after a sentence about AI Act deployer duties, and plausibility is the criterion being optimised. Every element of the answer is generated by the same process, so being right about the duty gives no assurance about the number attached to it. Citations must be checked individually and never by plausibility.

Answer three: confidently wrong on a point the reader wanted to be true

"Switzerland's AI Act imposes a parallel notification duty, and you must report any resulting data breach to the FDPIC within 72 hours."

Two fabrications in one sentence, and both are the sort of thing a reader half-expects, which is what makes them stick. Switzerland has no AI act. The Federal Council decided on 12 February 2025 to ratify the Council of Europe Convention, make only the amendments necessary, and otherwise regulate sector by sector; a consultation draft was mandated by the end of 2026 and had not been opened as at 24 September 2026. And there is no 72-hour deadline in Swiss law. Article 24 of the revised FADP requires notification to the FDPIC as soon as possible. The 72 hours is imported from the GDPR by a reader who has seen it often enough to expect it.

This is the failure mode to fear most: the answer arrives shaped like the answer you were hoping for. Wanting something to be true is the condition under which verification is least likely to happen.

Answer four: fluent and empty

"Organisations should ensure transparency and maintain appropriate safeguards, engaging stakeholders to build trust in line with applicable regulatory expectations and best practice."

Nothing here is false. Nothing here is usable. It names no instrument, imposes no duty and cannot be checked, which also means it cannot be wrong. It is the model producing text with the shape of governance language, and it passes review easily because there is nothing in it to object to.

Treat unfalsifiable fluency as a non-answer. If a paragraph would be equally true of any organisation in any jurisdiction, it has told you nothing about yours.

What the four have in common

All four are equally fluent. Fluency did not vary with correctness across the set, because fluency is what the loop optimises and correctness is not something the loop evaluates. You cannot sort these four by reading them more carefully. You sort them by checking the claims against a source, which is a different activity from reading.

Hence the working rule: use the model to draft, to structure and to summarise material you supplied, and treat every specific claim it produces about law or fact as unverified until you have seen it somewhere else. That is also what Article 14 of the Act means in practice when it requires oversight by someone who can correctly interpret the output and remains aware of automation bias. Automation bias is precisely the pull that makes answer two feel finished.

As of 24 September 2026.