Part 1 set out what needs to be in place before launch: a named outcome and owner, a short list of measures across the chain, a recorded baseline and an agreed decision rule. This part covers what happens once the work is running.
Watch what could get worse
Headline KPIs can improve while quality, experience or fairness slips. Pair each initiative with guardrail metrics, and set a threshold for each that triggers a review, even when the headline numbers look strong.
Quality
Error rate in a sample checked by a person, rework, and corrections after release.
Experience
Complaints, escalations to a person, and customer and employee satisfaction.
Fairness and compliance
Outcome differences across groups where relevant, policy exceptions and audit findings.
Cost to run
Usage-based fees, support effort and vendor costs as volume grows.
Four traps that distort the result
Counting time saved as money saved
Hours saved are a benefit only when they are redeployed to valued work or remove a cost. Record where the time went.
Comparing against a different period
Seasonality, staffing and volume changes can look like AI impact. Compare like for like, or use a comparison group.
Leaving out the full cost
Include licences, usage fees, integration, support, training and the time of the people involved, not only the vendor invoice.
Changing the definition midway
If a KPI definition changes, restate the baseline. Otherwise the trend reflects the change in definition, not the work.
Review at 30, 60 and 90 days
Each review answers a different question, in the same order as the measurement chain. Judging value at 30 days is premature; waiting a year to look is too late to correct course.
- Is the target group using it as intended?
- What is getting in the way of use?
- Are guardrail metrics within threshold?
DecideRemove adoption barriers before judging results.
- Are performance KPIs moving against the baseline?
- Is quality holding in checked samples?
- Where does the workflow still break?
DecideAdjust the workflow, training or configuration.
- Has the business outcome moved?
- What is the net value against the full cost?
- What would it take to scale?
DecideScale, adjust or stop, using the rule agreed before launch.
After 90 days, move the initiative into the quarterly review alongside your other operating metrics.
Calculate net value honestly
The calculation is simple. The discipline is in what you count.
(baseline cost per unit − new cost per unit) × annual volume + other measured benefitsannual benefit − annual run cost − build cost spread over its expected lifeExample: invoice processing
| Baseline cost per invoice | $12.00 |
| Cost per invoice after launch | $7.50 |
| Annual volume | 40,000 |
| Annual benefit | $180,000 |
| Annual run cost | −$45,000 |
| Build cost of $60,000 over 3 years | −$20,000 |
| Annual net value | $115,000 |
Count only benefits you can measure, or that the budget owner agrees are real. Revenue effects need the same baseline discipline as cost effects, and the full cost belongs on the other side of the line.
Define every KPI
A measure without a written definition drifts. Before launch, write a short definition card for each KPI that records:
- the KPI name, its layer in the chain and its owner
- the definition and formula
- the data source and who reports it
- the baseline value and the period it covers
- the target, the date and the guardrail threshold
- the decision rule: what result means continue, adjust or stop
The Measuring AI Value PDF includes a printable definition card.
Make value a routine
Measuring AI value is not a one-time business case. It is an operating routine: a baseline before launch, a short set of measures across the chain, guardrails, and reviews that end in a decision. With that routine in place, leadership teams can scale what works, fix what is close and stop what is not paying back, on evidence rather than opinion.