AI in practice: Five questions to ask before adding AI to a claims process

The question facing most claims teams is not whether to use AI, but where to focus it, and how to tell whether it is working once it is there.

These five questions come up in almost every conversation we have on the subject. They are worth answering before a pilot starts rather than after, because each one is considerably cheaper to resolve at the design stage.

1. What does good look like, and how will you know?

The instinct is often to benchmark AI against your best handler and to hold off until it matches them however waiting for that can mean passing over gains that are available now.

Speaking at an MGAA session earlier this year, our CTO Tom Burroughs described work at a UK claims operation where an AI-assisted scope auditing process reached around 70% accuracy. Not a replacement for expert judgement, but consistent enough to reshape how the whole process was organised, with the remaining anomalies routed to people.

It is useful to decide in advance what level of performance would change your process, and what you would do with the cases that fall below it. Both answers should exist before anything goes live rather than retrospectively.

2. Can your data be read the way an AI reads it?

Claims systems are largely deterministic. Rules run, fields populate, outcomes follow. Large language models work differently, on context and meaning rather than fixed logic, which means the shape your data arrives in has a direct bearing on the quality of the answer.

Policy booklets are the clearest example. They are long, densely cross-referenced and written for a different purpose entirely, and handing one over whole tends to produce weaker results than pre-processing it into sections and serving up the relevant part. Email chains behave similarly, and benefit from being summarised and categorised before anything else is asked of them.

This is worth noting because it is a data engineering task rather than an AI one, and it is often the piece that determines whether a promising pilot becomes a production process. It is also consistent with what regulators are seeing more broadly. In the Bank of England and FCA’s 2024 survey of AI in UK financial services, four of the five most commonly reported current risks related to data, covering privacy and protection, quality, security, and bias and representativeness.

3. Are you giving it the right information, or all of it?

There is a reasonable assumption that more context produces a better answer but in practice, the opposite can often apply. Volume introduces noise, and the relevant detail then competes with everything around it.

The skill sits in retrieval rather than supply: identifying which parts of a policy, a file or a correspondence history actually bear on the question, and presenting those as succinctly as possible. Teams that treat this as the core design problem tend to get more reliable output than teams that treat it as a matter of giving the model access to everything.

4. Who owns the output?

This is the one most often left until later, and the one that causes the most difficulty when it is.

An AI process needs the same things any other worker in your operation needs, that is, someone accountable for its performance. A clear measure of what success looks like. Feedback loops that are reviewed and acted on rather than simply collected. Without those, quality drifts quietly, and it is usually a complaint or an audit that surfaces it.

Regulators are already looking at this directly. That same 2024 survey found 84% of firms reporting an accountable person for their AI framework, with 72% saying executive leadership held accountability for AI use cases. It is worth checking whether that accountability exists at the level of the individual process as well as at framework level, since those are not the same thing.

There is an external dimension to this now as well. The EU AI Act’s transparency obligations became applicable on 2 August 2026, requiring people to be told when they are interacting with an AI system and AI-generated content to be marked, and those apply regardless of how a system is classified. The heavier record-keeping requirements attached to high-risk systems sit further out, at December 2027, and the Act’s high-risk list points at life and health underwriting rather than general insurance claims handling.

The UK is taking a different route, with the FCA working through existing frameworks such as the Consumer Duty and the Senior Managers and Certification Regime rather than AI-specific rules, which puts the emphasis on evidencing outcomes and ownership. We will look at what that means for claims operations in more detail in another article coming soon.

5. What happens when work changes?

Because these systems are probabilistic, their performance is tied to the data they are processing. When claim mix shifts, when a new peril or product comes on stream, when correspondence patterns change after a surge event, output can move without anyone changing a setting.

Set up and tested is the start of the process rather than the end of it. That means periodic review built into the operating rhythm, sample checking as a standing activity, and a named owner who notices when something has drifted. The 2024 survey offers a useful prompt here: 34% of firms reported a complete understanding of the AI they use, with 46% reporting partial understanding.

6. Where to start?

None of these questions require a large programme to answer. They require a decision about one process, and honesty about how you would tell whether it was working.

In our experience the fourth question is the one that gets deferred most often, and the one that most reliably determines whether a pilot becomes something the business can depend on.

Which of the five would you find hardest to answer about a process running in your operation today?