What counts as an incident, and who decides

Most AI problems are not incidents. A model returns a poor recommendation, someone notices, the case is handled. That is operations. The question this lesson answers is where the line sits between operations and something that starts a legal clock.

The statutory definition

Article 3(49) of the EU AI Act defines a serious incident as an incident or malfunctioning of an AI system that directly or indirectly leads to any of the following:

• the death of a person, or serious harm to a person's health;
• a serious and irreversible disruption of the management or operation of critical infrastructure;
• infringement of obligations under Union law intended to protect fundamental rights;
• serious harm to property or the environment.

Read the third one carefully, because it is the one people miss. Death and infrastructure failures announce themselves. An infringement of fundamental rights protections does not. A recruitment model that has been systematically disfavouring a protected group for eight months does not produce an alarm, a customer complaint or a system outage. It produces a normal-looking pipeline. Yet it falls inside the definition, and the organisations that get caught out are almost always caught out on this limb rather than the dramatic ones.

Note also "directly or indirectly". The chain from an AI output to the harm can run through people and other systems. A model that mis-prioritises cases, causing a person's treatment authorisation to be delayed, has indirectly led to the harm. The indirection does not remove it from scope.

Who decides, and when

The decision that something is a serious incident is not the reporter's to make, and it should not be. Asking a first-line employee to classify against Article 3(49) before raising anything guarantees under-reporting, because the safest individual choice is always to conclude it probably does not qualify.

The workable split is this. The person who sees the problem reports the facts. Someone nominated in advance, with the legal and technical context, makes the classification. The duty of the first person is speed and accuracy about what happened. The duty of the second is the judgement call and the consequences that follow from it.

This split has to be written down before anything happens, because it determines who is sitting on the clock.

What to report when you are the first person

Five things, and none of them requires you to have reached a conclusion.

• Which system. The name people use, plus the register entry if you know it.
• What happened, in observable terms. "The model declined 60 claims on Tuesday where the postcode field was empty" rather than "the model appears to be biased".
• When you became aware, and how. This timestamp is not administrative detail. It is the start of the clock.
• Who might be affected, and how many. A range is fine. "Somewhere between 40 and 400 customers" is more useful than silence while you count.
• Whether it is still happening. An ongoing condition is a different problem from a historical one, and it determines whether anyone needs to reach for the stop function.

The bias towards silence

Every incentive in an ordinary workplace pushes against raising this. Reporting looks like accusing a colleague, it creates work, and if you turn out to be wrong you have cost people a week. So the normal behaviour is to wait for certainty, and certainty arrives well after the deadline would have.

Two things counter it. The first is making clear, in writing, that reporting facts is not making an allegation and that an unfounded report in good faith carries no consequence. The second is designing the route so that raising something is cheap: one named destination, no form to complete first, no requirement to have diagnosed the cause.

The practical test of whether your organisation has this right is simple. Ask three people at random where they would send a suspected AI incident at four in the afternoon. If you get three different answers, or a pause, the route does not exist yet, whatever the policy says.

As of 24 September 2026.