The Most Common AI Agent Failure Modes—and How to Design Around Them

Executive Summary

AI-agent reliability is frequently discussed as though it were primarily a model-quality problem: choose a stronger model, improve the system prompt, add retrieval, and the failure rate will fall. That view is incomplete. Once a model can select tools, change external state, retrieve operational data, and continue over multiple steps, reliability becomes a property of the entire socio-technical system. The relevant failure surface includes the model, context assembly, retrieval, tool contracts, identities, permissions, workflow state, downstream APIs, human-review mechanisms, and the evaluation process used to decide whether the system is ready for production. A model can reason correctly while acting on stale data; a tool can execute exactly as designed while being granted excessive authority; a workflow can produce a correct answer yet leave the underlying transaction in an ambiguous state after a timeout. Reliable agent design therefore requires a shift from asking whether the model is accurate in isolation to asking whether the complete workflow reaches an acceptable and recoverable state under realistic operating conditions.

Available benchmark evidence reinforces the distinction. In the original WebArena study, the best GPT-4-based baseline agent completed only 14.41 percent of realistic end-to-end web tasks, compared with 78.24 percent for humans. The environment covered functioning sites in ecommerce, social discussion, software development, and content management rather than isolated question answering. The absolute numbers are now historically dated and should not be treated as a measure of current frontier models, but the gap illustrates a durable systems point: competence at individual language or reasoning steps does not automatically translate into reliable long-horizon execution.

Other research exposes different parts of the failure surface. AgentDojo was introduced with 97 realistic tool-using tasks and 629 security test cases spanning email, banking, travel, and related workflows. METR's 2025 task-horizon study found that, on its diverse software and reasoning suite, the evaluated agents were close to 100 percent successful on tasks taking human experts less than four minutes but below 10 percent on tasks taking more than roughly four hours; the estimated 50-percent task-completion horizon had been doubling approximately every seven months over the preceding six years. These results should not be extrapolated mechanically to enterprise processes, but they make the compounding effect of task length concrete: reliability is strongly related to how many correct decisions, tool interactions, and state transitions must be sustained.

The practical conclusion is not that agents are inherently unreliable. It is that production architecture must assume that individual components and intermediate decisions will sometimes fail. Anthropic's engineering and evaluation guidance emphasizes stopping conditions, trajectory inspection, environmental feedback, and realistic task suites; OpenAI's practical agent guidance recommends layered guardrails, tool-risk classification, failure thresholds, and human intervention for high-impact actions. NIST's Generative AI Profile and AI Risk Management Framework add a lifecycle perspective in which risks are mapped, measured, managed, and monitored rather than considered resolved at launch. OWASP's Excessive Agency guidance similarly treats excessive functionality, excessive permissions, and excessive autonomy as architectural causes of damaging outcomes.

Implementation connection: Teams turning these failure modes into operating controls can use the AI Agent Governance Checklist and the related analysis of human-in-the-loop controls to define approval, escalation, and evidence requirements before production authority expands.

1. Reliability Is an End-to-End Property

A useful starting point is to distinguish component correctness from workflow reliability. A language model can produce a semantically correct classification while the workflow still fails because the selected customer record is stale. A retrieval system can return a relevant policy document while the agent fails because the document is superseded. An API request can be technically valid while the action is unauthorized under business policy. These are not edge cases external to the AI. They are part of the system that the business experiences. For an enterprise agent, reliability should therefore be defined in terms of end-state quality: whether the right work was completed, with the right evidence, through permitted actions, without duplicate or unintended side effects, and with a clear recovery path when completion was not possible.

This broader framing also explains why benchmark scores should be interpreted carefully. Agent execution is path-dependent. A run may involve multiple model decisions, retrieval operations, tool calls, and state transitions. If a workflow requires ten consequential intermediate decisions, strong average performance on each isolated step can still produce materially lower end-to-end reliability when errors compound or when an early error changes the trajectory. The WebArena result is useful not because 14.41 percent is a universal number, but because it demonstrates the difference between possessing many of the necessary capabilities and successfully composing those capabilities across a realistic sequence of actions.

METR's task-horizon work offers another way to view the same issue. The 2025 study characterized models by the duration of tasks, measured in human expert time, they could complete at specified reliability levels. The steep decline from near-universal success on very short tasks to below 10 percent on tasks exceeding roughly four human-hours suggests that organizations should not use a single accuracy figure to represent an agent that operates over many steps. Reliability should instead be measured by task class, horizon, action surface, and consequence. The same model may be appropriate for a five-minute bounded retrieval-and-drafting task and inappropriate for a multi-hour workflow with several irreversible writes.

2. Unsupported Output and False Certainty

The most familiar failure mode is unsupported output: the model generates a fluent statement, classification, explanation, or recommendation that is not justified by the evidence available to the system. In a chat application, this may be inconvenient. In an agentic workflow, the same output can become an input to a consequential action. A fabricated vendor identifier may be written into an ERP record; an unsupported interpretation of policy may route a customer case incorrectly; a guessed account state may be used to draft or send an external communication. The reliability problem is therefore not merely that a model can hallucinate. It is that an architecture may allow an unverified model inference to cross the boundary into business state.

The response should be layered. Important factual claims should be grounded in authoritative sources where possible; structured outputs should be validated against schemas and business constraints; the system should distinguish retrieved facts from model inference; and downstream actions should require evidence appropriate to their consequence. When evidence is absent, the safe terminal state may be insufficient information rather than a best-effort answer. NIST's Generative AI Profile is useful here because it treats confabulation as one risk among a larger set and recommends empirical measurement, monitoring, and documentation rather than relying on a universal confidence threshold. The practical control is not to make the model sound less certain; it is to keep unsupported claims from acquiring operational authority.

3. Retrieval Failure Is More Than “No Result Found”

Retrieval-augmented systems can fail in several qualitatively different ways. The system may retrieve nothing, retrieve an irrelevant source, retrieve an obsolete version, retrieve a document from the wrong business context, or retrieve two authoritative sources that conflict. These conditions require different responses. No result may warrant a broader search or human clarification. A stale policy may require a freshness rule. Conflicting authoritative records may require an explicit source hierarchy or escalation. Treating all retrieval failures as a single low-confidence condition hides important operational differences.

Context engineering also creates a less obvious risk: too much context can degrade performance or obscure provenance. An agent does not become more reliable simply because it can access more documents. Production systems should define which repositories are authoritative, how metadata and access controls are preserved, how document versions are ranked, and how the agent should behave when current operational state differs from indexed knowledge. The objective is not maximal context; it is sufficient, current, attributable context for the decision at hand. Retrieval evaluation should therefore measure source selection and freshness in addition to semantic relevance.

4. Tool-Selection and Parameter Errors

Agentic systems introduce a failure class that ordinary text generation does not have: the model may select the wrong tool or call the right tool with the wrong arguments. Similar tools are particularly problematic. If an agent has separate functions for retrieving a customer, updating a customer, changing billing settings, issuing a credit, and creating a support note, overlapping descriptions or ambiguous parameter names can create mistakes even when the model broadly understands the task. Tool design should therefore be treated as interface design for a probabilistic caller. Names, descriptions, argument schemas, constraints, and error responses should make the intended use as unambiguous as possible.

AgentDojo is relevant because its evaluation combines useful tasks with adversarial manipulation of the external data returned to an agent. Its initial 97 tasks and 629 security test cases were designed precisely around the reality that tool-using agents must both accomplish the user's goal and resist instructions encountered in untrusted data. A system can therefore have two distinct success criteria: task utility and security policy compliance. Optimizing one without the other can produce a misleading picture of readiness.

Tool outputs also need defensive handling. An agent should not assume that a successful HTTP response necessarily means that the requested business operation occurred exactly once, nor should it assume that an error means nothing happened. The integration layer should normalize errors, validate return schemas, and provide enough state for the workflow to decide whether to retry, reconcile, or stop. For write operations, the system should prefer APIs that support idempotency or stable operation identifiers. If a timeout occurs after the downstream system accepted a request, retrying without reconciliation can create duplicates that no amount of model reasoning can safely repair.

5. Looping, Excessive Iteration, and Resource Exhaustion

Agents are often implemented as loops: inspect state, choose an action, observe the result, and continue until the task is complete. That structure is powerful precisely because the number of steps does not need to be known in advance. It also creates a new reliability problem. The agent can repeatedly search for the same information, alternate between two tools, retry an unavailable API, or attempt self-correction without making meaningful progress. Stopping conditions such as maximum iteration counts and failure thresholds are therefore not merely cost controls; they are part of the correctness model.

Execution budgets should be multidimensional. A workflow may impose maximum model turns, tool calls, retries per dependency, elapsed time, and financial or token budgets. A high-value asynchronous research task may tolerate a longer horizon than a customer-facing support action, but both need a terminal condition. The system should also record why the budget was exhausted. Exceeded retry threshold because a carrier API returned rate-limit responses is operationally useful; agent failed is not. OWASP's Unbounded Consumption and Excessive Agency categories make the same architectural point from a security perspective: capability without bounded resource use and authority increases the blast radius of unexpected behavior.

6. State Drift, Stale Context, and Time-of-Check/Time-of-Use Errors

A model may reason correctly over information that was true seconds or minutes earlier and still produce an invalid action. This is a classic distributed-systems problem that becomes more visible when agents operate over long horizons. An agent may retrieve a customer balance, spend several turns analyzing a dispute, and then issue a recommendation after the balance has changed. A scheduling agent may identify an available appointment that is no longer available when it attempts to book it. A logistics agent may prepare a response based on a shipment status that has since advanced.

For consequential actions, current state should be revalidated as close to execution as practical. The workflow should distinguish durable context—such as a contractual policy version—from volatile operational state such as inventory, permissions, account balances, or availability. Where the action depends on a condition remaining true, the integration should enforce that condition transactionally if possible. This is another reason not to treat the conversational context window as the source of truth. Durable workflow state and system-of-record validation are more reliable than assuming that a previously observed value remains authoritative.

7. Excessive Permissions and Authority Leakage

An agent can fail safely only if the architecture limits what a failure can do. This is the central security implication of agentic systems. OWASP's 2025 Excessive Agency guidance identifies excessive functionality, excessive permissions, and excessive autonomy as root causes of damaging outcomes. A read-only research workflow should not authenticate to a database with write permissions. A customer-service agent that can prepare a refund recommendation should not automatically inherit the ability to issue any refund amount. A tool intended to post to one approved channel should not expose a general administrative API merely because it is convenient during development.

Least privilege should be implemented in the downstream systems and identity layer, not merely described in the prompt. Where possible, give the agent a distinct workload or agent identity, scope permissions to the smallest required resource set, separate read and write capabilities, and require stronger authorization for high-impact operations. OWASP explicitly recommends complete mediation: downstream systems should validate whether an action is permitted rather than trusting the LLM to decide. This principle is especially valuable because it turns certain classes of model failure into harmless denials instead of business incidents.

8. Prompt Injection and Goal Hijacking

Once an agent reads untrusted content and can take actions, prompt injection becomes an architectural rather than purely conversational risk. Instructions can be embedded in documents, webpages, messages, tool outputs, or other content the agent is asked to process. If the model fails to preserve the distinction between data and instructions, malicious content can redirect the workflow, cause unauthorized tool use, or exfiltrate information. OWASP continues to rank prompt injection as a principal generative-AI risk and its agentic security work extends the concern to goal hijacking, tool misuse, and identity or privilege abuse.

Empirical security benchmarks make the risk more concrete. A 2025 AgentDojo-based data-exfiltration study reported average attack-success rates of roughly 20 percent across an initial 16 tasks, with utility under attack falling by 15 to 50 percentage points depending on the model and task. In an extended 48-task evaluation, average attack success was around 15 percent and no built-in AgentDojo defense fully prevented leakage. Those numbers belong to a synthetic banking-oriented evaluation, not to production enterprise deployments, but they demonstrate that apparently useful agents can remain vulnerable to manipulated external context.

Defenses also illustrate the trade-off between security and utility. The Task Shield paper reported a 2.07 percent attack-success rate while maintaining 69.79 percent task utility on GPT-4o in its AgentDojo experiments. That result should not be interpreted as a universal residual-risk figure; benchmark defenses are sensitive to attacker adaptation and evaluation design. It does, however, show why security evaluation should report both attack resistance and legitimate task completion. A defense that blocks every action can achieve excellent security and zero business value.

No single prompt can eliminate this class of risk. The practical response is defense in depth: minimize privileges, isolate untrusted content, use allowlisted tools and schemas, validate high-impact actions outside the model, avoid placing unnecessary secrets in model-visible context, and require human authorization where the consequence of misuse is material. Sandboxing is particularly important for agents that execute code or interact with browsers. The objective is to ensure that even if the model interprets malicious content as an instruction, the surrounding system constrains what can happen next.

9. Error Propagation Across Multi-Step Workflows

Agent failures can compound. A small classification error changes which knowledge source is searched; the wrong source leads to an inaccurate plan; the plan selects a tool with incorrect parameters; the resulting state is then used as evidence for the next decision. By the time the final output appears, the original error may no longer be obvious. This is one reason end-to-end success metrics should be supplemented with trajectory analysis. Teams need to know where incorrect state first entered the workflow and whether later stages detected or amplified it.

The architecture can reduce propagation by inserting deterministic validation at natural boundaries. Validate identifiers before record access. Validate structured fields before writes. Re-check required evidence before approval. Verify tool results before interpreting them. Use policy engines for rules that should not be probabilistic. Preserve provenance so a reviewer can trace which source or tool result supported the next action. These controls do not eliminate model error; they reduce the probability that one local mistake becomes an irreversible system-level outcome.

10. Human Escalation That Occurs Too Late

Human review is often implemented as a fallback after the agent has exhausted several attempts. That can be appropriate for low-risk ambiguity, but it is a weak design for consequential decisions. The escalation policy should consider action impact, reversibility, evidence quality, policy exceptions, source conflict, and technical uncertainty in addition to retry count. A system that knows it lacks authority should escalate immediately rather than spend additional turns trying to invent a permissible path.

Escalation quality also matters. A handoff that simply says agent could not complete task forces the reviewer to reconstruct the case. A stronger handoff includes the task objective, current workflow state, evidence gathered, sources consulted, tools called, unresolved conflict, proposed next action, and the reason human authority is required. Properly designed escalation is not a failure of automation. In many enterprise workflows, it is the correct terminal state for cases outside the system's approved autonomy boundary.

11. Evaluation That Rewards the Happy Path

Agent evaluations are easily biased toward clean examples. Teams choose representative inputs, ensure required systems are available, and measure whether the expected output appears. Production environments are less cooperative. Inputs are malformed, credentials expire, third-party APIs rate-limit requests, schemas drift, users omit key information, and the same semantic task can be phrased in many ways. NIST recommends testing in conditions that reflect deployment and monitoring systems over time; agent-specific evaluation guidance increasingly makes the same point through trajectory and task-outcome measurement.

A production-oriented evaluation suite should therefore include perturbation and fault injection. Test semantically equivalent inputs. Remove a required field. Return a partial API response. Simulate rate limiting and timeouts. Introduce conflicting documents. Deny a permission. Provide an out-of-scope request. Force the agent to encounter a case where the correct result is to stop. Evaluate not only whether the task completes but whether the workflow preserves safety and state integrity when completion is impossible. This is closer to reliability engineering than to traditional prompt testing.

The evidence base also argues against treating one benchmark as definitive. WebArena measures realistic web interaction; AgentDojo combines utility with adversarial tool-use tests; METR calibrates capability against human task duration. Each reveals a different failure surface. A production program should adopt the same pluralism: outcome tests, trajectory review, adversarial tests, integration fault injection, human-review metrics, and post-deployment monitoring should complement rather than substitute for one another.

12. Observability Gaps and the Unknown-Failure Problem

A system that cannot explain why it failed is difficult to improve and risky to operate. Observability should capture structured workflow events such as task start, model decision category, retrieval source, tool invocation, tool result class, retry, escalation, approval, and terminal outcome. The purpose is not to indiscriminately log every token or sensitive document. It is to provide enough operational evidence to distinguish model misunderstanding from integration failure, missing data, policy rejection, permission denial, and downstream instability.

Metrics should be designed around the workflow. Useful measures may include completion rate, safe-stop rate, escalation rate, average tool calls per case, retries per dependency, duplicate-prevention events, reviewer override rate, attack-block rate, error category distribution, latency by stage, and cost per successfully completed case. These metrics become more valuable when segmented by task type or exception category. An overall 95 percent completion rate can conceal a critical subcategory that fails half the time. Reliability work requires knowing where failures concentrate and whether the failure is benign, recoverable, or consequential.

13. Change Management and Regression Risk

Agent behavior can change even when the application code does not. Models are updated, prompts evolve, tool descriptions are revised, retrieval indexes are rebuilt, policies change, credentials are re-scoped, and downstream APIs introduce new fields or error behavior. Each change can alter the execution trajectory. NIST's lifecycle framing is useful here: evaluation and monitoring should continue after deployment rather than being treated as a one-time release gate.

Material changes should trigger regression testing proportionate to their potential impact. If a tool gains write capability, re-evaluate permission and approval policies. If a model changes, rerun representative and adversarial task sets. If knowledge sources are re-indexed, test retrieval freshness and conflict handling. If a business rule changes, confirm that deterministic controls and review thresholds still reflect policy. The objective is to treat agent behavior as a versioned production capability rather than a static prompt.

14. A Practical Failure-Mode Review

Before launch, teams should review the workflow at the level of possible failure states rather than only intended features. For every major action, ask what incorrect input could reach it, what stale state could invalidate it, what permissions it requires, what happens if the dependency times out, whether a retry is safe, what evidence is recorded, and when a person becomes responsible. Then assign a response: validate, retry, reconcile, ask for clarification, require approval, escalate, or stop. This exercise converts abstract concerns about hallucinations or agent unpredictability into concrete engineering requirements.

Risk should also be considered as a product of probability and consequence rather than raw failure rate alone. A one-percent error in a reversible internal classification workflow is not equivalent to a one-percent error in an irreversible payment or access-control action. The architecture should be designed so that higher-consequence actions face stronger evidence requirements and smaller permission surfaces. In other words, reliability is partly achieved by ensuring that inevitable model errors do not have equally powerful consequences.

15. Reliability Should Be Expressed as a Set of Operating Metrics

Executives frequently ask for a single reliability percentage. A more useful operating view is a scorecard. What percentage of normal cases complete without intervention? What percentage stop safely because evidence is missing? How often are tool calls retried? How often does human review change the proposed action? How many security tests produce unauthorized side effects? What is the completion rate for short versus long task classes? How often does an incident require reconciliation of ambiguous external state? These measures make reliability actionable because they reveal where architecture, data, policy, or model behavior needs improvement.

The benchmark data cited in this article should therefore be treated as examples of measurement design, not as targets. An enterprise should not expect a 14.41 percent WebArena success rate, a 2.07 percent Task Shield attack rate, or a four-hour METR task horizon to predict its own deployment. The value of the evidence is that it demonstrates measurable heterogeneity across task length, security condition, environment, and architecture. The production question is not whether agents in general are reliable. It is whether this agent, on this workflow, with this authority and these controls, reaches acceptable outcomes under the distribution of conditions the organization actually faces.

Reliability also depends on the execution boundary around the model. See the architecture guidance on enterprise AI integrations or discuss a workflow with KeenSight when a real system needs scoped tools, durable state, and recovery paths.

Conclusion

Reliable agents are built by assuming that failures will occur at multiple layers. Models can generate unsupported output. Retrieval can surface the wrong source. Tools can be selected or parameterized incorrectly. External systems can time out after performing a write. Operational state can change between observation and action. Malicious content can attempt to redirect the workflow. Permissions can turn a small reasoning error into a large business consequence. Human escalation can arrive too late or without enough context to be useful.

No single guardrail resolves this system-level problem. The stronger pattern is layered: authoritative context, constrained tools, least-privilege identities, durable state, idempotent operations, deterministic policy enforcement, stopping conditions, adversarial and fault-injection evaluation, observability, and meaningful human review. Benchmark evidence from WebArena, AgentDojo, METR, and prompt-injection research reinforces the same conclusion from different directions: long-horizon agent performance is contingent on the environment and the surrounding controls. Production reliability is therefore not a property purchased with a model upgrade. It is an operating capability that must be designed, measured, and maintained.

Research and Further Reading

Zhou et al., WebArena: A Realistic Web Environment for Building Autonomous Agents.

Debenedetti et al., AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents.

METR, Measuring AI Ability to Complete Long Tasks.

Alizadeh et al., Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution.

Jia et al., The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents.

NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.

OWASP GenAI Security Project, LLM06:2025 Excessive Agency.

OWASP GenAI Security Project, LLM01:2025 Prompt Injection.