Your Speech Accuracy Number Is Measured Per Character. Identifiers Fail Per String.

An operator asked a useful question: how reliably does a voice agent capture strings of letters and digits, order numbers and licence plates, rather than ordinary spoken words? And what happens when it mishears one?

Two questions. The first has an uncomfortable answer and the second has a worse one.

The number you were quoted is measured on the wrong unit

Vendors quote accuracy in the high nineties. That figure is almost always word-level accuracy, the share of individual tokens transcribed correctly. For ordinary speech it is the right unit, because ordinary speech is redundant. Drop one word in thirty of a booking request and the booking still happens. Language carries enough spare information to absorb the error.

An identifier carries none. Every character is load-bearing, one wrong character makes the whole string worthless, and the tokens are the ones recognition finds hardest because letters and digits sound like each other. P and B. M and N. T and D. C and Z. The letter O and the digit zero. The letter I and the digit one. There is no context to recover from, which is exactly what makes it an identifier.

What that does to the number

The SNuC corpus, published at LREC 2022 by a University of Sheffield team, is the first published corpus of spoken alphanumeric identifiers of the kind used as serial and part numbers. It contains over 13,000 sequences from more than 50 speakers, nearly 20 hours of speech, with identifiers of roughly 10 to 12 characters.

The authors ran a general-purpose deep-learning ASR system against real-world recordings from an industrial partner and reported both units side by side.

• Word-level accuracy on the real-world data: 96.6 percent
• Whole-identifier accuracy on the same data: 77 percent

Just under 97 percent per character sounds excellent. It means roughly one in four complete identifiers came back wrong. On the clean corpus the same system scored 98.4 percent per word and still only 81.7 percent per identifier.

This is not a defect in one system and it is not an artefact of the year. It is arithmetic. Chain six to fourteen tokens together and require all of them, and a small per-token error rate compounds into a large per-string one. A current model posts better numbers than that baseline. The shape does not change, and the shape is the point: the metric you are shown is structurally incapable of telling you how often the thing that can hurt you goes wrong.

What happens when it gets one wrong

The intuitive answer is that the agent says it did not catch that and asks for a repeat. That is the benign failure. It costs four seconds and it is self-correcting.

That is often not what happens. WildASR, a diagnostic benchmark published in March 2026 and built entirely from real human speech across four languages, evaluated seven widely used ASR systems along environmental degradation, demographic shift and linguistic diversity. Two findings matter here.

Degradation is severe and uneven, and robustness does not transfer across languages or conditions, so a system that holds up in English in a quiet room tells you little about the same system in another language on a speakerphone.

And under partial or degraded input, models often hallucinate plausible but unspoken content. The authors name this as a concrete safety risk for downstream agent behaviour.

A plausible invention is far worse than a refusal. A refusal is visible and self-correcting. An invention is a well-formed string that passes every downstream check built to catch malformed ones. It is written into the record, the wrong part ships, and nobody finds out until the box is opened.

What actually fixes it

None of this requires a better model.

• Constrain the output to strings that can exist. Do not transcribe an open string and hope. Match against reality: the live parts catalogue, open repair orders, booked appointments, active accounts. These are finite sets. In the same study, adapting the system to the identifier domain moved whole-identifier accuracy from 77 percent to 91.7 percent on real-world data while word-level accuracy barely moved, from 96.6 percent to 98.5 percent. Almost the entire win came from the constraint rather than from better hearing.
• Use the check digit you already have. A VIN validates itself. The ninth character is a check digit computed from the other sixteen under the U.S. federal VIN standard. Writing a captured VIN anywhere without running that checksum discards a free deterministic error detector that catches the great majority of single-character mishearings. This is also why plates are harder than VINs and need a different design: a licence plate has no checksum, no fixed length across jurisdictions, and no redundancy.
• Confirm attributes, not characters. Reading a string back invites a reflexive yes, because verifying a random string by ear is a task people are bad at and bored by. Confirm the object instead, the vehicle and the last visit. People validate objects reliably and strings unreliably. Where characters must be read, use a phonetic alphabet on the confusable classes rather than bare letter names.
• Change what you measure. Exact-match accuracy per identifier, not word error rate, and broken out by string length, by caller condition such as handset, speakerphone or hands-free in traffic, and by language. Word error rate is a vendor metric. Exact match is yours.

The honest limit

Some of this is genuinely unsolved. Mixed letter-and-digit strings remain the hardest class in the field, harder than pure digits. In a real service drive, with a compressor running and a hands-free caller in traffic, no design reaches a hundred percent.

The realistic goal is not perfection. It is that every wrong capture is caught by a constraint, a checksum or a confirmation before it becomes a record. That is achievable with the models that already exist.