To measure AI ROI, compare the change in a business measure against a baseline taken before the AI was introduced, value that change, and subtract the full cost of implementing and running the AI, including staff time. Then check that quality and risk measures have not got worse. Most AI "ROI" claims skip the baseline, count only licence costs, or ignore quality, which is why they are hard to trust.
Establish a baseline
Before any pilot, measure the process as it is today:
- Volume: how many tasks, cases or documents per week
- Time: how long each takes, from start to finish and in staff effort
- Quality: error rates, rework, complaints or satisfaction
- Cost: staff cost per task, and any external costs
Measure for long enough to see normal variation, typically four to eight weeks. Without a baseline, you cannot show improvement; you can only describe activity.
Where possible, compare a team using AI with a similar team that is not, over the same period. That separates the AI's effect from seasonal changes or other initiatives.
Include implementation and operating costs
Count all costs, not just the software:
| Cost type | Examples |
|---|---|
| Implementation | Integration, data preparation, security review, legal review, project management |
| Licences and usage | Subscriptions, per-use charges, infrastructure |
| People | Training time, time spent reviewing AI outputs, a quality or governance lead |
| Change | Process redesign, communication, temporary drop in productivity while people learn |
| Ongoing | Monitoring, updates, vendor management |
Evaluate quality, risk and business impact
A faster process that makes more mistakes is not a success. Track three groups of measures:
- Efficiency: time per task, throughput, cost per task
- Quality and risk: error rate, rework, complaints, incidents involving data or inaccurate outputs
- Business impact: the objective the AI was meant to serve, such as customer satisfaction, revenue or retention
Also track adoption: if people are not using the tool, the benefits will not appear however good it is.
A transparent worked example
A fictional insurer pilots AI-drafted summaries of claims documents for its 40 claims handlers. All figures are illustrative.
Baseline (8 weeks): each handler spends about 5 hours a week reading and summarising documents. Loaded staff cost is £30 an hour.
Pilot result (12 weeks, 10 handlers): summarising time falls to about 3 hours a week per handler, including time to check each AI summary. Error rate in summaries is unchanged on a sample of 200 checked cases.
Estimated annual benefit if scaled to all 40 handlers: 2 hours saved × 40 handlers × 46 working weeks × £30 = £110,400 of staff time.
Estimated annual costs:
| Item | Year 1 |
|---|---|
| Licences and usage | £28,000 |
| Integration and security review | £25,000 |
| Training (1 day per handler) | £9,600 |
| Quality lead (0.2 FTE) | £12,000 |
| Total | £74,600 |
Year 1 net benefit: about £35,800. Year 2 is higher because integration is a one-off cost.
Uncertainty, stated openly:
- The pilot team may have been more enthusiastic than average; time saved could be lower at scale. Below about 1.35 hours saved per handler per week, year 1 costs would exceed the benefit.
- Time saved only becomes a financial benefit if it is used for other valuable work or reduces future hiring. Otherwise it is capacity, not cash.
- Quality was checked on a sample; a larger error rate in rare cases would change the picture.
This is the kind of honest range, rather than a single headline number, that a board can make a decision on.
AI strategy success metrics beyond single projects
At strategy level, also track:
- number of use cases moved from pilot to scale, and number stopped
- share of staff trained and actively using approved tools
- incidents and near misses
- progress against the business objectives in your strategy
Related reading
- AI strategy subject guide: building, documenting, measuring and presenting an AI strategy
- AI use-case and business-case pack: a worked first-year business case with a downside case
- How long until an AI strategy shows ROI?
- Is my AI strategy working?
- The biggest AI mistakes businesses make
- How to build an AI strategy for your company
Building business cases and measuring outcomes is part of our AI strategy training for business leaders, which covers investment and ROI, governance and performance tracking without requiring technical skills.
The bottom line
Baseline first, count every cost, and check quality and risk alongside savings. Report a range with your assumptions, and treat any promise of guaranteed AI returns with suspicion.