AI agents are easy to demo. They are hard to operate.

If you have sat through a software demo in the last eighteen months, you have probably seen one: an AI agent researches a task, works through a few tools, drafts the email, closes the ticket — to applause. Then the pilot starts, and six weeks later nobody can quite say what the agent changed. That gap, between what agents do on stage and what they do in production, is now the central problem of enterprise AI. The model was never the hard part.

The pilot-to-production gap

The numbers are blunt. MIT's Project NANDA "State of AI in Business" report found that 95% of enterprise generative AI initiatives are getting zero return. For custom enterprise AI tools specifically: 60% of the organizations that evaluated them never got past evaluation, only 20% reached a pilot, and 5% reached production. Gartner, looking at agentic AI in particular, predicts that more than 40% of agentic AI projects will be canceled by the end of 2027 — killed by escalating costs, unclear business value, or inadequate risk controls.

None of this is because the models stopped improving. Demos fail in production for structural reasons: a demo is a single run with a forgiving judge, while a production agent runs unattended, holds state across hours, touches real systems with real credentials, and is ultimately judged by Finance. The reported failure modes — brittle workflows, no memory of the actual process, drift from how the work is really done — are engineering problems, not model problems.

What a production agent actually needs

A useful mental model: the LLM is the smallest layer of the stack. Around it, five layers decide whether the agent is dependable or just impressive:

Teams that skip the middle layers are not building agents — they are building demos with an on-call rotation attached.

Guardrails: least privilege for non-deterministic workers

You would not give a new contractor production credentials and no review. An agent deserves less trust, not more, because its behaviour is probabilistic. The pattern we apply on client engagements:

• Scoped, per-agent credentials — a service identity per agent, with permissions limited to the tools its job requires.
• Allowlisted actions — the agent can call what we enumerated, not whatever its next completion suggests.
• Budgets and rate limits — caps on cost, calls, and blast radius per task, enforced outside the model.
• Human approval gates — irreversible actions such as payments, deletions, or external emails wait for a person.
• A full audit trail — every prompt, tool call, and output logged and reviewable, because "why did it do that?" is a production question, not a philosophical one.

Evals are the new test suite

Nobody ships code without tests, yet most teams ship agents without evals. An eval suite looks a lot like a regression suite: a set of task scenarios, expected outcomes, and a score computed on every change — prompt edits, model upgrades, tool swaps. Combine it with a canary rollout and a rollback path, and "we upgraded the model" stops being a leap of faith and starts being a deploy.

This is where traditional DevOps discipline transfers directly: versioned prompts and configs, CI that runs the eval suite, observability that captures agent trajectories in production, and an incident process that treats a misbehaving agent like any other regression.

The operating model matters more than the model

The last gap is organizational. An agent in production needs an owner, an on-call, a changelog, and a definition of success measured in the business's units — not "it worked in the demo." Gartner's forecast that a third of enterprise software will embed agentic AI by 2028 makes this worse, not better: agent-washed features will add autonomous behaviour to systems nobody is operating as agents. Decide what runs autonomously, what waits for a human, and who is accountable — before the agent decides for you.

The organizations on the right side of the GenAI divide are not the ones with the best demos. They are the ones that treated agents as software: evaluated, guarded, observed, and owned. That is unglamorous work — and it is exactly the work we do at Enriched Code, from DevOps and cloud foundations to data & AI delivery. If you have a pilot that deserves to reach production, talk to us.