I spend most of my time with heads of customer service and CX at banks, insurers and telcos, and what they ask about AI customer service has changed in the last year. Nobody asks anymore whether AI can answer questions. They ask why the business case they signed off on doesn't show up in their cost line.
AI customer service works when it resolves real customer requests from start to finish, hands over to a person smoothly when it can't, and is measured on what customers actually get done. It disappoints when it is measured on how many conversations it keeps away from your team. That difference sounds small, but it decides whether a project gets expanded or reversed.
The promise is big. Gartner forecasts that by 2029, agentic AI will autonomously resolve 80 percent of common customer service issues without human intervention, leading to a 30 percent reduction in operational costs. Early results seemed to confirm it. In February 2024, Klarna announced that its AI assistant was doing the equivalent work of 700 full time agents.
Then came the correction. In May 2025, Klarna's chief executive told Bloomberg that cost had been too predominant a factor and that the result was lower quality. The company started hiring human agents again, and he said customers should know "there will always be a human if you want." In August 2025, Commonwealth Bank of Australia reversed 45 customer service redundancies after replacing the roles with a voice bot, admitting it had not adequately considered all relevant business considerations.
These are not isolated cases. Gartner expects that by 2027, half of the companies currently planning to significantly reduce their service workforce because of AI will abandon those plans, as reported by CX Network. And Forrester predicts that service quality will dip in 2026 as companies wrestle with the complexity of AI deployment.
Customers had warned us. In a Gartner survey, 64 percent of customers said they would prefer companies didn't use AI in customer service, and their biggest worry was that it would become harder to reach a person.
Our AI Use Case Library lists 559 public customer service deployments, each with the name of whoever made the claim. Two of its benchmarks side by side show what the headline numbers leave out.
The median containment rate across 36 organizations is 66.5 percent: two out of three conversations never reach a person. But the median reduction in customer contacts, across the 11 organizations that report it, is 20 percent. The two numbers come from different organizations, so you cannot subtract one from the other. The gap is still a good reason to look closer at what your own containment figure means.
Containment counts conversations that didn't get handed over. It doesn't tell you whether the customer got what they needed, or whether they gave up, called the next day or switched to another channel. A customer stuck in a loop counts as contained. As Len wrote about our first analysis of the library, nobody writes a press release about the pilot that died, and nobody puts the abandoned conversations in the containment rate.
It also matters who reports the number. For handling time reduction, vendors report a median of 65.5 percent, while organizations reporting their own results land at 50 percent. Interestingly, for containment it's the other way around: organizations report higher numbers than their vendors. The lesson is the same either way. Ask where a benchmark comes from before you put it in your business case.
The deployments that hold up, in our own projects and in the public record, share a few traits.
They start with requests that can be finished end to end. Blocking a card, changing a delivery, updating an address, checking a claim. These need integrations with your core systems as well as a knowledge base. An assistant that can only explain the procedure sends the customer somewhere else to finish it, and that shows up as a repeat contact.
They hand over well. When the AI can't help, the customer reaches a person quickly, and the person sees the full conversation and everything already collected, so the customer never starts over. Yuri explains how to design this in Conversational AI Design: Why Bad Bots Are a Design Problem.
They ground every answer. The assistant answers from approved sources only. When Air Canada's chatbot gave a passenger wrong information about a refund, a tribunal held the airline responsible. Your chatbot is your company talking.
They are transparent. Customers are told they are talking to an AI. Since 2 August 2026 this is also a legal obligation in Europe under Article 50 of the EU AI Act.
They measure outcomes. Task completion, first contact resolution, repeat contacts within a few days and customer satisfaction, next to containment. And someone reads the transcripts every week.
If you are comparing AI customer service software or AI customer service agents, these questions separate a demo from a deployment:
- Which of our top requests can it complete end to end, and with which of our systems does it integrate?
- What happens when it can't help? How fast does a customer reach a person, and what does that person see?
- Which numbers will you report, and how are they measured? Ask for task completion and repeat contacts, not only containment.
- How are answers grounded, and how do you prevent made up policies?
- Does it work in our customers' languages and channels, including voice and, in many of our markets, Arabic dialects?
- What can our team change themselves without a new project: flows, answers, tone, handover rules?
The organizations that get good answers to these questions are the ones that expand their AI customer service after year one instead of rolling it back. If you'd like to compare notes on your own setup, or see how we approach AI for customer service at banks and insurers, I'm always happy to talk. Preferably over coffee, and you can get in touch any time.
