Your Containment Rate Is A Vanity Metric

Containment rate is the vanity metric of voice AI. It is the share of calls an agent handles with no human in the loop, and it is the number teams put on a slide when they want to look like they are winning. It is also, in my experience, the number that predicts the least about whether a deployment survives contact with real customers.

I learned this the slow way, by listening to the calls that went wrong. Almost none of them failed because the agent could not answer. They failed at the seam, the moment the agent should have stopped, handed the call to a person, and instead kept going one step too far, because a high containment number was implicitly the thing being optimized.

The research caught up to the intuition

The pattern is well documented now. The 2026 International AI Safety Report, written by more than a hundred researchers, found that agent success rates drop sharply as a task stretches from minutes to hours; reliability that looks fine in a short demo decays as the interaction gets longer and more branched. Engineers writing about production agents this year call it the "95 percent illusion": a workflow that looks 95 percent reliable in a demo compounds its rare failures across multi-step execution until the real-world number is nothing like the demo number. The pattern is consistent enough that analysts expect a large share of agentic projects to be scrapped over the next two years, and the post-mortems keep landing on the same layer.

That layer is the handoff

It is the least-engineered part of most voice stacks. Teams pour effort into evaluation harnesses, prompt engineering, latency budgets, and interruption handling, all of it real and necessary. Then they treat the escalation to a human as an afterthought: a transfer, a beep, "please hold."

One of the sharper pieces of agent-design work this year puts it plainly: a handoff is a transfer of working state, not a transfer of a conversation. The systems that treat it as just a conversation hand-off consistently fail, because the human picks up with none of the context the agent had, and the caller has to start over. The same work argues that escalation rate is one of the most diagnostic numbers you have: too high and the automation is not earning its keep, and suspiciously low and the agent is probably plowing through calls it should have escalated, quietly burning trust one over-confident answer at a time.

The number worth watching

The reframe is simple: a voice agent is not an autonomy contest. It is a system for getting a caller to the right resolution, human or machine, with the least friction. Autonomy is a means. The handoff is where most of the value, and most of the failure, actually lives, and it is the part nobody claps for.

There is an operator moment that made this concrete. A team was proud of a very high containment rate until they listened to the escalations. The agent, trained implicitly to avoid handing off, would take a genuinely stuck call and keep improvising, three or four turns past the point where any competent human would have said "let me get someone." The fix was not a better model. It was lowering the bar for handoff and making the handoff carry state, so that when the agent stopped, the human started warm. Containment went down. Saved calls went up. That is the trade nobody puts on a dashboard.

Three questions before I would ask your containment rate

• What is your escalation rate, and is it a number you actually calibrated, or one that fell out of the model's confidence?
• When the agent hands off, what state travels with the call? Identity, reason, what was tried, and where the agent was, or a cold transfer and a frustrated person repeating themselves?
• How do you measure a saved call versus a contained-but-worse one? If you cannot measure it, you are optimizing the vanity metric by default.