The AI you did not choose

AI at work, and the AI you do not see · 7 min read

Lesson 34 of 60
In this module

Most of the AI in your organisation was never adopted. Nobody wrote a business case for it, nobody assessed it, and nobody announced it. It arrived in a release note.

This is the ordinary way software works now. You license a suite for email, documents and meetings. The vendor ships a summarisation feature, then a drafting feature, then a meeting transcription feature that writes action items. Each one is switched on by default for tenants who have not explicitly opted out. Your organisation deployed three AI systems in a quarter without a single decision being taken.

Why this is a governance problem and not just an IT one

The EU AI Act attaches duties to the deployer: the party that uses an AI system under its own authority in the course of a professional activity. It does not ask whether you wanted the system. It does not offer an exemption for features you did not notice. If your staff are using it on your instructions, in your work, you are deploying it.

Two deployer duties in Article 26 bite immediately and are routinely missed for vendor-shipped features. First, the deployer must use the system in accordance with the instructions for use, which means somebody has to have read them. Second, where a high-risk AI system is used at the workplace, Article 26(7) requires the deployer to inform workers' representatives and the affected workers before putting it into use. A transcription feature that arrives silently on Tuesday has already failed that ordering if the use is in scope.

There is a further trap in Article 25. A deployer becomes a provider, with the much heavier provider obligation set, if it changes the intended purpose of a system so that the system becomes high-risk. The general drafting assistant that someone points at candidate CVs is the standard example. The tool did not change. The purpose did, and employment and worker management are listed as high-risk under Annex III.

How to recognise AI in a tool you already use

Marketing language is useless here. Vendors both overclaim and underclaim, and neither has legal effect. Go back to the definition in Article 3(1): a machine-based system that infers from the input it receives how to generate outputs such as predictions, content, recommendations or decisions. Adaptiveness after deployment is optional. Inference is the constant.

In practice, four signals are reliable:

  • The output is different each time for the same input. Deterministic software does not do this. Generative features do.
  • Nobody can show you the rule. If the vendor's support team can point at a configurable threshold, it is automation. If the answer is "the model decides", it infers.
  • The feature ranks, scores, predicts, suggests or summarises. Those verbs describe inference.
  • There is a feedback control. Thumbs up and thumbs down buttons exist because the vendor knows the output is sometimes wrong and cannot say when.

Note what is not a signal. Whether the feature is free. Whether it is labelled "beta". Whether it runs on your tenant or the vendor's. Those affect your data-protection analysis, not the classification.

The three places it hides

Inside productivity suites. Summaries, smart replies, meeting notes, search ranking, spam and priority sorting. These touch almost every employee, which makes them the widest exposure in most organisations and the one least likely to be on a register.

Inside line-of-business systems. The HR platform that scores applicants for fit. The service desk that routes tickets by predicted urgency. The CRM that suggests which accounts to call. These are narrower in headcount and far more consequential per use, because they sit directly on decisions about people.

Inside things you built. An old statistical model still running nightly is an AI system. It did not stop inferring because the team that built it left.

What to do on Monday

You are not expected to maintain the inventory personally. You are expected to feed it. When a feature appears that ranks, scores, predicts or writes, record three things and send them to whoever holds the register: what the tool is, what decision the output feeds, and whether any personal data goes into it. That is enough for someone to classify it. Waiting until you are sure it counts guarantees the entry is late.

As of 24 September 2026.

Sign in to save your progress

You can read every lesson without an account. Signing in keeps your place and unlocks the assessment.