Your Customers Don't Care That AI Can Make Mistakes

There is a strange pressure on companies today.

"We have to do something with AI."

And very often, that "something" becomes a chatbot on the website or a voice agent answering the phone.

I understand why. AI can reduce costs, provide 24/7 support and potentially handle thousands of customer interactions without adding people.

But there is a problem.

Bad AI is not better than a human. Bad AI is worse than no AI at all.

Over the past few weeks, I have had several experiences that made me think more deeply about this. And they reinforced something we have learned while building ReplyFabric.

Customers are much less forgiving of AI mistakes than they are of human mistakes.

"AI can make mistakes" is not good enough

We see the disclaimer everywhere.

AI can make mistakes.

Of course it can. Humans make mistakes too.

But I don't think customers treat those mistakes in the same way.

When a human customer service employee misunderstands something, we often give them some credit. We explain it again. We understand that they may have misunderstood the question or don't immediately have all the information.

With AI, that patience disappears remarkably quickly.

Especially when you have already explained the same thing twice.

And definitely when the AI confidently gives you an answer that is simply wrong.

From a technology perspective, we can explain this. Large language models are probabilistic. They can hallucinate. They work with confidence levels. There will always be edge cases.

Your customer doesn't care.

If you decide to put AI between your company and your customer, the AI's mistake becomes your company's mistake.

You can't hide behind a disclaimer.

My telecom operator offered me a link I couldn't open

I experienced a good example recently while I was in Riyadh.

I had a problem with my Belgian mobile subscription. A block on my account meant that I couldn't use mobile data abroad.

So I contacted the helpdesk.

The AI understood enough of the problem to propose a solution. It sent me an SMS with a link where I could find instructions to fix the problem.

There was just one small issue.

I didn't have mobile data.

The AI had answered my question, but it hadn't solved my problem.

That distinction is incredibly important.

Eventually, I called again because I needed a human.

And this brings me to one of the more absurd consequences of poor customer service AI. Customers start learning tricks to escape it.

Clear requests such as "I want to speak to a human" can work with some systems. With voice agents, frustration or aggressive language can sometimes trigger an escalation too.

I'm normally very polite, particularly when I know a conversation is being recorded. But at some point you find yourself saying something along the lines of:

"I know this is being recorded, and I'm sorry, but I'm going to say this because I really need a human. F**k off."

And suddenly you get a human.

Think about that customer journey.

We are training customers to swear at our AI so they can receive customer service.

That's not digital transformation.

Sometimes customers don't complain. They just leave.

I had another experience with my bank.

I opened the chat because I had some specific questions. The AI tried to help, but the answers weren't relevant to what I was asking.

I tried again.

Still not useful.

So I stopped.

No angry message. No request for a human. No thumbs down.

I simply left.

And that is one of the biggest problems with measuring AI customer service.

From the outside, that interaction can look surprisingly successful.

The AI handled the conversation. No employee became involved. The customer stopped asking questions.

Great containment rate.

Except my problem wasn't solved.

I had simply given up.

Customer silence is not necessarily customer success.

Good AI can still create a bad customer experience

I recently experienced the opposite as well.

I contacted a technology company's support team by email. Within a minute, I received an answer.

It was good.

I replied. Within another minute, another good answer arrived.

Clearly AI, but useful AI. I didn't care that it was AI because it was helping me.

Then the AI asked for additional information.

I provided everything it needed.

And then nothing.

One day.

Two days.

My assumption is that the case reached a point where the AI could no longer handle it and it was transferred to a human.

There is absolutely nothing wrong with that.

In fact, that is exactly what should happen.

The problem is that nobody told me.

One simple message would have completely changed the experience:

"Thanks. I have everything I need. I'm handing this over to a member of our support team. They will get back to you as soon as possible."

That's good AI.

The problem isn't handing over to a human.

The problem is pretending the handover didn't happen.

AI should know when to shut up

I think this is one of the most important capabilities we need to build into customer-facing AI.

AI shouldn't always answer.

That sounds obvious, but much of today's generative AI has been designed around exactly the opposite behavior. Ask a question and the model will try very hard to produce an answer.

In customer service, that's dangerous.

Sometimes the smartest thing AI can say is:

"I'm not sure."

And sometimes the smartest thing AI can do is simply shut up and let a human take over.

That's not failed automation.

That's good automation.

Zero tolerance for customer-facing AI errors

This has strongly influenced how we build ReplyFabric.

Our philosophy is effectively zero tolerance for AI errors when they touch the customer.

That doesn't mean pretending we have somehow created infallible AI. We haven't, and nobody has.

It means we don't accept "AI can make mistakes" as an excuse for knowingly sending questionable answers to customers.

ReplyFabric first needs to understand the email, its context and the information required to answer it. The proposed answer then goes through our quality-control process.

If confidence is high enough, the draft can be presented to a human for review.

If our quality control isn't sufficiently confident, we don't just hope for the best.

We reprocess it.

And if we're still not sufficiently confident?

We don't use the AI answer.

The system tells the human, in effect:

"Sorry. I'm not sure about this one. You need to handle it yourself."

I'd much rather have AI do nothing once in a while than confidently do the wrong thing.

That's the standard companies should be thinking about.

Not:

"How many people can we replace?"

But:

"For which interactions is our AI genuinely good enough to represent our company?"

The strange economics of paying per "resolution"

There is another development in AI customer service that deserves more scrutiny: outcome-based pricing.

The logic sounds fantastic.

Don't pay for tokens. Don't pay for messages. Pay when AI solves something.

Intercom's Fin AI Agent has helped popularize pricing based on resolutions, while Salesforce has introduced outcome and conversation-based approaches around Agentforce.

Conceptually, I like paying for results.

But there is a potentially perverse problem hidden inside the model.

Everything depends on what you call a result.

AI support platforms need rules for determining whether an interaction has been resolved. A customer explicitly confirming that their problem has been solved is straightforward.

But what happens when the customer simply stops responding?

Some AI customer-service resolution models can infer a resolution from the fact that the conversation ended without further customer interaction.

Now go back to my conversation with my bank.

I stopped responding.

Was my problem solved?

No.

I gave up.

That's a fundamental difference.

A customer who disappears might be happy. But they might also be frustrated. They might have called instead. They might have opened another support channel. They might have decided to deal with the problem tomorrow.

Or they might simply have concluded that your AI is useless.

Yet from an automation perspective, that interaction can look wonderfully efficient.

No human involved.

Conversation finished.

Problem "resolved."

That creates the risk of optimizing for the wrong KPI.

Containment is not customer success

This is why I would be very careful with containment as the ultimate metric for AI customer service.

If your AI handles 80% of conversations without human intervention, that number sounds fantastic in a board presentation.

But I want to know something else.

How many problems did it actually solve?

And even more importantly:

How many customers left because the AI didn't solve their problem?

A good AI system should have absolutely no incentive to keep a conversation away from a human when a human is the better option.

• It should escalate early when necessary.
• It should explain the handover.
• It should provide the human with everything it already learned so the customer doesn't have to start again.
• And it should consider that a successful outcome.

Maybe one of the most dangerous words in AI customer service today is "resolved."

Because sometimes "resolved" really means:

The customer stopped arguing with the bot.

Don't replace humans with poor AI

I absolutely believe AI will automate a significant amount of work currently done by humans.

I'm building a company around that belief.

But that doesn't mean every human interaction should be replaced by AI as quickly as possible.

If your AI isn't good enough yet, use a human.

Use AI behind the scenes.

Let it categorize the request. Extract information. Search your knowledge base. Look up business data. Prepare a draft. Give the employee context. Reduce five minutes of work to thirty seconds.

There is enormous value there without putting mediocre AI directly in front of your customer.

Then measure.

Learn.

Improve.

And automate more when the quality proves that you're ready.

AI should earn the right to automate

That's perhaps the principle I keep coming back to.

AI should earn the right to automate.

Don't give it that right because your board wants an AI strategy.

Don't give it that right because your competitor launched a chatbot.

And don't give it that right because a vendor tells you it can resolve 70% of your customer conversations.

Give it that right because you've measured the quality and you trust the result.

Until then, keep a human in the loop.

And when the AI isn't sure?

Let it shut up.

Your customers will probably thank you for it.