A closed ticket isn’t a solved problem: What AI support metrics miss

“Resolution is not always case closed.”

That’s how one support leader put it when we asked how AI changed their work. In their case, a ticket had been marked resolved, but the customer came back anyway with a new case, no link to the first ticket, and the original request still sitting in the system as a win. The problem hadn’t moved. Only the status had.

We heard a version of that story more than once, so we ran open-ended interviews with 67 support and CX teams about what AI has actually changed on the front lines. We used unitQ Research to gather responses by text, voice, or video and analyze them at scale, the same way our customers turn thousands of interactions into themes.

One theme ran underneath most of the answers. A closed ticket tells you an interaction ended. Whether the customer’s problem got solved is a mystery, and as AI handles more of the support queue, the distance between “resolved” and “answered” is widening.

The teams in our study weren't anti-AI. Far from it. Over half gave AI a clearly positive read, and 72% volunteered a concrete benefit such as speed, efficiency, time saved, and round-the-clock availability. The tension isn’t whether AI helps, because the majority agree it does. It’s that the dashboard keeps saying resolved, and the people reading it aren’t always sure that’s true.

What a closed ticket actually tells you

A closed ticket tells you whether an interaction met your team’s criteria for closure. Containment, on the other hand, tells you whether a conversation stayed inside an automated channel without reaching a human. Both are workflow facts. Neither proves the customer walked away with what they came for.

Your customer might still need to test a fix before they know it worked. They might discover the explanation doesn't apply to their account. Or they might just stop asking after a bot repeats the same answer a third time. All three close the ticket. None of them solve the problem.

Klarna learned this in public. Klarna reported that its AI assistant handled 2.3 million conversations in its first month and cut average handle time from 11 minutes to under two, a triumph by every number on the wall. Then the CEO told Bloomberg the company had “focused too much on cost,” that “the result was lower quality,” and started rehiring. The metrics that declared victory measured speed and average satisfaction. They said nothing about whether the hard problems got solved.

In our study, 36% of respondents raised some version of this directly: that conventional support metrics don’t fully capture whether a customer got useful help. As one put it, the goal is to catch “the interaction that looks successful in the metrics but wasn’t actually successful for the customer.”

The failures nobody reports

The hardest problem our respondents described isn’t the angry customer. It’s the quiet one.

Thirteen of the 67 teams (19%) named the same blind spot: poor experiences that customers never report. People abandon a conversation, try another channel, or leave without filling out the survey. Nothing in the ticket log flags any of it. The interaction closes clean, and the dissatisfaction leaves no trace.

Satisfaction surveys only capture the people who answer. The customers who give up are, almost by definition, the ones least likely to stop and tell you why. One respondent put it as:

“The biggest blind spot is the interaction that looks successful in the metrics but wasn't actually successful for the customer.”

There are signals, though, if you watch behavior instead of waiting for complaints: a repeated question, return visit, sudden channel switch, or a later call to a human about something a bot already resolved. None is proof on its own, but together they mark the conversations worth a second look.

Most teams only check a fraction, and know it

We also asked how teams catch these misses. The honest answers were uncomfortable.

Six respondents described reviewing a sampled or hand-picked slice of conversations, with open gaps in what that sample could catch. One pointed out that reviewing a small sample could leave refunds and payment errors undetected entirely, and if a customer never complains, the interaction may never reach the review queue at all.

Another was blunter about the gap:

“Currently we do not have a measure. We just assume.”

That’s the default state of a function built to sample human tickets, now facing automated volume it was never sized for. (We dug into that coverage problem in why almost nobody is grading their AI support.)

When the handoff is the failure

The moment AI hands a customer to a human is where a lot of the damage concentrates. Respondents described bots missing intent, repeating information the customer had already given, or continuing to answer after someone explicitly asked for a person.

When those handoffs go wrong, the cost lands downstream. One respondent described routing errors feeding a human-support backlog with waits of “upwards of weeks.” The bot’s ticket closed on time. The customer’s problem waited two weeks for a person.

This is worth saying plainly because the obvious fix is wrong: penalizing every escalation just teaches a bot to cling to conversations it should hand off. A good handoff, one that reaches the right person with the full history intact, is exactly what a scorecard should reward.

What to measure after a ticket closes

If a closed ticket isn’t the finish line, what is? Our respondents, taken together, point at three checks that start where most measurement stops.

1. Follow the issue across contacts

Look for customers coming back with the same problem, including through a different channel. One respondent described comparing the questions customers asked a chatbot against the cases they opened right afterward. Connect those later contacts to the original interaction, then ask why the first answer counted as a resolution.

2. Review what the answer actually did

Check whether the response fit the customer’s situation, applied the right policy, and gave a next step that worked. Pull conversations that carry no complaint and no low rating into the review too. Otherwise the customers who volunteer feedback quietly decide your entire sample for you.

3. Evaluate the handoff

Look at when the AI escalated, whether it reached someone who could help, and how much the customer had to repeat. Could the next agent finish the job from the conversation history, or did the customer start over?

That’s what Support QA was built to do. It scores human and AI interactions across connected channels on custom scorecards covering tone, compliance, deflection accuracy, and handoff quality, graded against what the customer actually experienced rather than whether the ticket closed.

It also lines up with where the broader research points. McKinsey’s work on customer-experience measurement argues for evaluating the full journey and pairing feedback with operational data rather than trusting any single in-the-moment score. For a support team, that means counting the contacts a customer actually needed to get done, not the ones the system recorded.

How to decide whether support worked

Judge the interaction by what the customer could do afterward. Did they fix the problem, reach someone who could help, or start another conversation about the same thing a day later? That’s the measure that survives contact with reality, and it's the one a closed-ticket count can't give you.

Gartner expects agentic AI to autonomously resolve 80% of common customer service issues by 2029. Whether that becomes 80% solved or 80% closed depends entirely on which number teams decide to trust.

Start with a closed ticket, and follow the customer’s next steps.

See what your closed tickets are hiding

Get a demo of unitQ Support QA scoring your real human and AI conversations, graded on what customers actually experienced, not whether the ticket closed.

About this study

unitQ Research conducted open-ended interviews with support and CX professionals about AI and customer support. This analysis covers 67 respondents across support and CX roles. Responses were coded for recurring themes using unitQ Research’s AI analysis, and percentages represent rounded shares of respondents who mentioned a theme. Because the interviews were open-ended, each person could raise multiple themes, and follow-up questions varied. Two respondents didn’t give a codable opening assessment of AI, so sentiment splits are reported out of the 65 who did.