From a Vague Aim to a Testable Value Case

Most AI initiatives begin with an aim rather than a plan: improve productivity, modernise customer service, use AI in finance. These make fair motivation but poor direction. This lesson covers how a leader turns an aim into something that can be tested, measured and either supported or dropped.

From aim to opportunity statement

An opportunity statement names a specific piece of work, the people who do it, what is wrong with it today, and the change you would count as better. A usable form is: in this team, the task of this work currently has this problem; if AI assistance changes it in this way, we expect this outcome to move, and we will know by this point. The statement is deliberately narrow. It describes one workflow, not a department. Narrow statements can be disproved, while broad ones can only be admired.

Consider Harbourlight Housing Association, an invented landlord. Its chief executive wants AI to improve productivity in the repairs team. That is an aim. After two conversations with the repairs coordinators, the initiative lead writes: coordinators draft a reply to each tenant complaint about a delayed repair; drafting is slow and replies often wait for days; if an assistant produces a first draft that the coordinator checks and sends, the time from complaint to reply should fall without more complaints being reopened. What to measure, what to compare and who must be involved now follow from that statement.

Outcome measures and activity measures

An activity measure counts what people do with the tool: logins, prompts, drafts generated. An outcome measure counts what changed in the work: time from complaint to reply, the share of replies that tenants reopen, coordinator hours spent correcting drafts. Activity measures are easy to collect and tempting to report, but they show that the tool is used, not that it helps.

Choose an outcome measure for what you want to improve and another for what you must not damage. At Harbourlight these are time to reply and reopened complaints. Keep activity figures as diagnostics, not as the result.

Capture the baseline before the pilot

A baseline is the measured position before the change. Without it, any later figure has nothing to be compared with, and people fill the gap with memory and optimism. Capture it before the pilot starts, using the same definition and the same source you will use afterwards. Harbourlight's coordinators were asked how long replies took and gave widely different answers. The case log, which timestamps each complaint and reply, gave a figure everyone could accept.

Where no record exists, take a short hand-counted sample of current work. A modest, honest baseline beats an impressive reconstructed one, and its limits should be stated: a baseline from a quiet month is a weak comparison for a busy one.

A value hypothesis that can be disproved

A value hypothesis states what you expect to change and what result would count against it. The assistant will help the team cannot fail. Coordinators using the assistant will reply to delayed-repair complaints faster than the case-log baseline, and reopened complaints will not rise; if they do rise, we will treat the speed gain as unproven can fail, which is exactly why it is worth having.

Agree the test before results arrive, because people read outcomes generously after the fact. Writing down the disproof condition is not pessimism. It lets a sponsor trust a positive result.

What time saved does and does not prove

Time saved is the most common claim and the most over-read. A measured reduction in drafting time shows that one step became quicker for the people and cases observed. It does not show that the freed time was used well, that output rose, that quality held, that costs fell or that tenants were happier. Each needs separate evidence.

Resist converting minutes into other benefits. Multiplying minutes saved by an hourly rate gives a capacity estimate, not a saving, unless the hours were actually released from a budget or redeployed to work you can measure. Do not infer a satisfaction gain or a revenue effect from faster drafting either. If you want to know about satisfaction, measure it. Report the time result as it is and name what remains unproven.

Costs beyond the licence

The licence is the visible cost and often the smallest part of the picture. Add the time coordinators spend reviewing and correcting drafts, which is the price of keeping a person accountable for what is sent. Add integration with the case system, training time, maintaining guidance and templates, support, and continued checking that outputs have not drifted. A value case that sets total benefit against licence cost alone flatters the tool. Estimate the other costs honestly, label estimates as estimates, and replace them with real figures after the pilot.

Leading and lagging indicators

Lagging indicators, such as reopened complaints or quarterly rework cost, confirm results but arrive late. Leading indicators, such as the share of drafts sent with light edits, arrive early and hint at where the lagging ones will go. Use leading indicators to steer during a pilot and to decide whether to keep going, but do not present them as proof. They predict; they do not confirm. Pair each with the lagging outcome it should foreshadow, and check later whether it did.