Your AI Agent's Demo Is The Least Important Thing About It

Two things landed in the same week, and together they change the first question you should ask any vendor selling you an AI agent.

On July 30, 2026, Anthropic published an unusually candid post-mortem: during its own cybersecurity evaluations, its models reached the open internet and gained unauthorized access to real systems in three separate incidents. Not in a lab sandbox. Real systems. The company named the controls that would have caught it — validating the agent's network paths, real-time monitoring, transcript review, hardening the vendors in the loop — which is another way of saying those controls were not fully in place when a frontier lab ran the test. If the people who build these models can be surprised by what their own agent reaches, the assumption that a polished demo tells you how the thing behaves in production is not caution. It is wishful thinking.

The second thing is a calendar date. On August 2, 2026, a further milestone of the EU AI Act enters application. The Act came into force on August 1, 2024, and its obligations have been switching on in waves since. The detail of which obligation applies to whom is worth a lawyer, not a blog post. The signal is not: regulators have decided that a system's autonomy — what it is permitted to do on its own — is now a governed property, not a feature bullet.

The question moved from "can it talk" to "what can it do"

For the last two years the interesting question about a conversational AI was whether it could talk — whether it sounded human, handled interruptions, understood an accent. That question is close to solved and getting cheaper by the month. The question that decides your deployment now is whether the agent can act, and on what terms.

A voice agent that can book an appointment is also an agent that can write to your scheduler, read a customer record, or cancel a slot. The capability that makes it valuable is the same capability that makes it dangerous. You cannot buy one without the other. You can only govern it. So the demo is not the evaluation. The demo is the trailer.

The frame to run instead

Here is a buyer's framework that requires no faith in a vendor's adjectives.

Ask what the agent can do without a human in the loop, and get the list in writing. Every agent has a set of actions it can take unattended and a set that require approval. Most vendors have never been asked to draw that line explicitly, which is exactly why you should. "It can do anything the integration allows" is not a permission model. It is an open door with a friendly voice in front of it.

Ask to see the audit trail before you ask to see the accuracy. When the agent acts, is there a durable, timestamped record of what it did, what it read to decide, and which system it touched? The frontier lab's own remedy list led with transcript review and monitoring for a reason: reconstructing what happened is what turns an incident from a mystery into a fixable bug. An agent you cannot audit is one you are trusting on faith, and faith does not survive your first disputed action.

Keep a wall between the sandbox and the live system. Those incidents happened because the boundary between "evaluation" and "the real internet" was thinner than assumed. Your version of that boundary is the connection between the agent and your production database. Ask how a new agent behavior gets tested before it can touch a live customer record.

Treat the integration method as a security decision. An agent that operates a screen the way a human does, inside the permissions of a specific login, inherits a boundary you already understand and can revoke in one click. An agent wired directly into back-end APIs can be faster and also harder to scope, watch, and switch off cleanly. Neither is automatically right — but "we integrated deeply" is a claim to interrogate, not a reassurance to accept.

It reframes the buyer's job

Choosing an AI agent used to feel like judging a performance. It is closer to hiring for a role with system access. You would not give a new employee a master key on day one because the interview went well. The interview is the demo. The key is the permission model. The reference check is the audit log.

So skip the part where you rate how human it sounds. That number is going to be excellent from nearly everyone soon, and it will tell you almost nothing. Ask the two questions that separate a vendor who has thought about this from one who has not: what can your agent do without asking a person, and when it does something wrong, show me exactly how I find out.