AI implementation
Why AI Agents Work in Demos and Fail in Production
Understand the gap between an agent demo and a dependable service: tool failures, permissions, state, evaluation, operating costs, and recovery.
An agent demo proves that one path can work. A production service must also handle incomplete inputs, slow dependencies, changing permissions, retries, and a user who closes the browser halfway through a task.
The gap is often in the surrounding software and operating process. Improving the prompt can help, but it cannot make an unreliable action endpoint transactional or repair an expired credential.
A demo hides assumptions
Consider an agent that updates a customer record after reviewing a support conversation. During the demo, it has a known customer, a clean transcript, working credentials, and an engineer watching every step.
In production, two customers may share a name. The transcript may contain a mistaken identifier. The CRM may accept the update but fail to return a response. Another employee may change the record while the agent is reasoning.
Before rollout, list the assumptions behind the successful demo. Turn each into a condition the system validates or a failure it can recover from.
Five failure patterns to test
| Failure | What the user experiences | Implementation response |
|---|---|---|
| Ambiguous identity | The wrong record is updated | Require a verified identifier; ask when unresolved |
| Uncertain tool result | A completed action is repeated | Use operation IDs and query action status |
| Lost execution state | A long task restarts from the beginning | Persist progress and define restart behavior |
| Excessive privilege | The agent accesses an unrelated account | Enforce scope in the tool or service |
| Endless investigation | A task consumes time and budget without finishing | Set limits and return an explicit incomplete status |
These responses need application behavior, not merely instructions telling the agent to be careful.
Work through an uncertain update
Suppose update_customer times out. There are at least three possibilities: the request never arrived, the update failed, or the update succeeded and the response was lost.
Blindly retrying treats all three as the same condition. Instead, associate the task with a durable operation identifier. If the downstream system supports idempotency, reuse that identifier. If it does not, design a verification or reconciliation step and avoid claiming the action succeeded until its status is established.
Also consider concurrent changes. An update based on an old record can overwrite a newer value. Version checks or conditional writes may be necessary, depending on the system. The model should receive a meaningful conflict result and an allowed next action.
Evaluate completed work, not confident language
A fluent response saying “updated successfully” is weak evidence. Check the underlying application state and whether the correct record changed. For read-only tasks, inspect whether the answer is supported by the retrieved evidence.
Build a test set from representative tasks, with permission to use the data. Include normal requests and failure scenarios: missing records, conflicting instructions, revoked access, a dependency outage, a repeated request, and a task that should be declined.
Repeat selected cases because model behavior can vary. Track task success, incorrect side effects, escalation, duration, and cost separately. Do not let a high average success score hide a permission violation.
Anthropic's agent evaluation guide discusses evaluating both execution and outcomes. Use that distinction when designing checks; a trace explains the route, while the final state establishes what happened.
Make the pilot an operational rehearsal
A pilot needs a named owner for failed tasks. Someone must review escalations, decide whether a result is acceptable, and approve changes to the workflow. If those responsibilities are undefined, the pilot may stall even when the model is capable.
Begin with a restricted set of tools and users. Where useful, run in recommendation mode before granting actions. Record the model and tool versions so a regression can be investigated. Retain only the trace data needed for operations, with appropriate access and retention controls.
Managed agent infrastructure can supply parts of the runtime. For example, Amazon Bedrock AgentCore offers services for operating agents. Such services still require your team to define the business task, permissions, and acceptance criteria.
Set release criteria for your workflow
Avoid borrowing an arbitrary “95% accurate” target. Define what an error means, what it costs, and which failures are unacceptable for this use case. A draft summary and a customer-account mutation need different release decisions.
Before expanding access, demonstrate that the system can recover from the tested dependency failures, show why it took an action, stop within its limits, and route unresolved work to an owner. Include the cost of that review in the business case.
If a fixed sequence can handle the task, a workflow may be sufficient. If an agent is justified, use the implementation cost model to budget for the full service. Dream Hatch Labs builds AI-enabled applications and workflows around these operating requirements.