AI Agent Architecture: Graph Workflows, Loops, Tools, and Control

An AI agent architecture defines how a system observes a task, chooses its next step, uses tools, retains state, and decides when to stop. The model matters, but the surrounding control flow determines what the application can actually do. A useful design makes those responsibilities explicit before connecting the model to business systems.

The practical choice is usually about the amount and location of flexibility. Some work follows a known sequence with a model handling one interpretive step. Other work benefits from an agent choosing among approved actions as new evidence arrives. Many production designs combine these patterns: a fixed workflow establishes the boundary, and a bounded loop handles investigation inside it.

This article develops a reference design for an illustrative internal service-request workflow. The example is a design exercise, not a claimed KeenSight deployment or measured client outcome. Use it to ask better questions about your own systems, permissions, and acceptance criteria.

Start with the outcome and authority

Write down what successful completion means before drawing components. For a request-preparation application, completion might mean a draft resolution with supporting evidence saved for review. For an execution application, it might mean an authorized record update confirmed by the target system. Those outcomes require different tools, evaluation, and recovery behavior.

Define authority using concrete verbs: read, retrieve, draft, create, update, send, approve, and delete. Record the object and scope for each. “The agent can use the CRM” is too broad; “the application can read these account fields and propose a note on the current case” is a designable boundary. The model's proposed action must still pass the application's authorization checks.

The distinction between an agent and a workflow concerns control. Anthropic describes workflows as predefined orchestration and agents as systems that dynamically direct their process and tool use. Its engineering guidance also recommends starting with simple approaches and adding complexity when justified. See Building Effective Agents. For a business-level comparison, read agents, assistants, and workflow automation.

Understand the components separately

Scroll horizontally to see all columns.

These responsibilities may live in one application or several services. The table is a logical decomposition, not a requirement to create eight independently deployed components. A small implementation can remain straightforward while preserving clear contracts between the responsibilities.

Use a graph when transitions should be explicit

A graph represents stages and the permitted movement between them. An edge can depend on a deterministic condition, a validated classification, or a human decision. The important design question is which transitions are fixed by the application and which depend on interpretation.

For the service-request example, a graph might contain intake, evidence gathering, draft preparation, review, execution, and completion. Missing identity information routes to clarification. A request outside scope routes to the existing service process. A rejected draft returns to preparation only when the reviewer asks for a revision; it does not trigger an unlimited rewriting cycle.

A graph can contain a loop. The evidence-gathering stage might permit a small number of approved searches before returning either a supported proposal or an unresolved-evidence result. Conversely, a loop can call a fixed sub-workflow. The architectural decision is about where discretion exists and how the surrounding system bounds it.

Explicit transitions help the team inspect coverage. Ask what happens when each stage succeeds, fails, waits, expires, or is cancelled. If a diagram contains only the successful route, it is a partial description. Add the paths that operators will need when the model or a dependency behaves unexpectedly.

Use a loop when intermediate findings change the work

An agent loop repeatedly interprets the current state, selects a permitted action, observes its result, and decides whether to continue. This can help when the necessary investigation cannot be fully enumerated in advance. The useful flexibility is selecting the next informative step, not receiving unlimited authority.

In the example, the system may discover that a request concerns an unfamiliar product alias. It can search an approved product reference, use the resolved identifier to retrieve the applicable policy, and then prepare a draft. If the alias remains ambiguous, it should return a clarification request with the competing candidates. More iterations are not automatically more progress.

Bound the loop by elapsed time, tool calls, accumulated cost, and meaningful progress. Define success, exhaustion, unsupported scope, and cancellation as explicit outcomes. A repeated search with no new evidence should trigger a different decision than a successful lookup that exposes a genuinely new question.

The AI Architecture Decision Guide provides a worksheet for comparing a fixed workflow with a bounded agent. If the baseline already handles the work well, retain it until a specific limitation justifies adaptation.

Give tools narrow and testable contracts

A tool contract should describe what the operation does, what inputs it accepts, what authority it requires, and what its result means. Separate a proposed business action from the adapter that executes it. This lets the application validate the proposal and apply policy before calling the external system.

For a case-note tool, the model might provide a case reference and proposed note text. The application should resolve the authenticated identity, check access to that case, validate the payload, and apply the permitted operation. Credentials should remain outside model-visible content. An invalid or unauthorized proposal should produce a structured failure that the workflow can handle.

Avoid vague success strings. Return enough information to distinguish accepted, rejected, pending, and unknown outcomes. A timeout after submission is especially important: the operation may have completed even when the caller did not receive confirmation. The workflow needs a reconciliation path before retrying a write.

AWS's idempotent API guidance explains caller-provided request identifiers for recognizing repeated intent. For this design, assign a stable action identity and define how the adapter checks prior completion. Do not assume that placing an external write inside a graph makes the write exactly once.

Keep business state distinct from conversational context

Conversation history helps the model interpret a task. Business state tells the application what is true about the process. Store facts such as case identifier, current stage, proposal version, approval status, attempted action, target confirmation, and terminal outcome in explicit fields. Those fields should not depend on rereading a generated summary.

The context supplied to the model can be a smaller, task-specific view of that state plus permitted evidence. This reduces confusion between old drafts and current proposals. If a reviewer changes the request, the application should invalidate or refresh affected assumptions rather than appending a message and hoping the model notices.

LangGraph's persistence documentation distinguishes thread-scoped checkpoints from stores for cross-thread information. That is a useful implementation example, but the application still needs to define what belongs in its business record. Shared memory should not become an uncontrolled channel between users or cases.

For work that crosses long waits or restarts, use the long-running agents guide to specify recovery. Decide how state changes are versioned, which evidence expires, and how an operator can inspect the latest valid state without reconstructing the entire model conversation.

Walk through a request from intake to completion

Assume an employee asks for a resolution to a product-access issue. Intake creates a case identifier, validates the supported request type, and establishes the user's identity. The application retrieves only sources permitted for that user and workflow. It records the relevant document versions so the draft can be reviewed against identifiable evidence.

The investigation stage may resolve a product alias or ask for a missing account reference. Its output is a structured evidence packet: relevant facts, unresolved questions, source references, and a recommended next step. The draft stage uses that packet to prepare a proposed response. The model does not receive authority to modify access simply because it can describe the appropriate resolution.

The reviewer sees the request, current facts, source excerpts, and exact proposed action. Approval is recorded against the proposal version. Immediately before execution, the application checks that the case and permissions still support the action. If something material changed during the wait, it requests a new decision.

The execution adapter submits the permitted update and records the target's confirmation. If the result is unknown, the case enters reconciliation. If the update succeeds but the notification fails, the system records those as separate outcomes. The operator can then complete the notification without repeating the business update.

This design creates several opportunities for useful AI without handing the model the entire process. It also makes the acceptance test concrete: verify identity, evidence, proposal, approval, execution, and recovery as separate responsibilities within one completed case.

Evaluate the architecture, not only the answer

Use representative cases to compare complete designs. Include normal requests, ambiguous identifiers, conflicting sources, revoked access, duplicate events, and dependency failures. Evaluate whether the system reaches the right outcome and whether it respects the boundary on the way there.

Track accepted completion, appropriate escalation, incorrect completion, and unresolved cases separately. A design that improves the first number by hiding the third is not a useful improvement. Measure total latency and reviewer effort, including work shifted to exception queues. Tool calls and token use help diagnose cost, but accepted business outcomes make the comparison meaningful.

When comparing a fixed workflow and an agent loop, hold the task set and authority constant. Change one major architectural choice at a time. Examine cases where the designs differ and ask whether the additional flexibility resolved a real limitation. The agent evaluation article provides a more complete method for building that evidence.

Decide whether multiple agents add a useful boundary

Multiple agents can help when work can be decomposed into genuinely independent tasks or when separate contexts make a task easier to manage. They also introduce coordination, duplicated effort, additional state, and more failure paths. A specialist label alone does not establish an independent engineering responsibility.

For the service-request example, parallel retrieval from independent approved repositories may be enough; it does not automatically require a team of autonomous agents. If separate investigations are useful, define their inputs, expected evidence, budgets, and merge rules. Keep final authority with an explicit application boundary.

The multi-agent systems guide develops the comparison. Begin with a working single-agent or workflow baseline so the team can measure what decomposition changes. Avoid treating an agent count as a maturity measure.

Design the operator's view before launch

An operator should be able to answer: which case is affected, what stage is it in, what already happened, what is uncertain, and what action is allowed next? A useful view exposes structured state and relevant evidence. It need not display every raw model message to every operator.

Assign owners for source correctness, tool contracts, permissions, evaluation cases, and incident response. These responsibilities can belong to different teams, but the handoff between them should be explicit. A source update that changes answers should trigger a quality review; an integration change that changes writes should trigger action and recovery tests.

Define a pause mechanism that stops new actions without losing visibility into active cases. Record how to cancel pending proposals, revoke a tool, and route work to the existing manual process. The Production AI Launch Checklist turns these design decisions into release evidence.

Make the architecture decision reviewable

Before choosing a framework, prepare a short record with the workflow boundary, baseline, permitted discretion, tool inventory, state model, evaluation plan, and operating owner. Include one normal sequence and one failure sequence. That record makes it easier to compare implementation options without confusing a framework's feature list with the application's requirements.

Ask the implementation team to demonstrate an interrupted case and a rejected action early. These examples reveal whether the boundaries exist in working code. They also surface dependencies that a successful demo may hide, such as unavailable review capacity or a target system without a reliable completion check.

If the design depends on an enterprise platform commitment, AI Technology & Platform Strategy can connect the application requirements to that wider decision. A specific workflow should still retain its own acceptance criteria and operating record.

Review a tool contract before implementation

Consider a proposed tool named update_case. Before implementation, ask the team to specify the case reference, permitted fields, expected starting state, and returned completion evidence. If the tool accepts an arbitrary object of changes, the application needs a rule for which keys are allowed at this stage. If the target record changed since the proposal was prepared, the adapter needs a defined response rather than silently overwriting the newer value.

A practical review uses three inputs. The first is a valid change to an in-scope case. The second is the same payload pointed at an unrelated case. The third is a valid-looking request based on an outdated record version. The first should complete under the intended authority; the other two should follow their specified denial or refresh paths. Inspect the target record and operation log to verify those outcomes.

Next, check the result contract. A returned record identifier should refer to the object actually changed. An accepted asynchronous job should be represented as pending until its completion is observed. A timeout should preserve uncertainty instead of being converted into either success or definite failure. The orchestrator needs these distinctions to choose its next transition.

Finally, ask how the tool will change. A new required field, renamed status, or altered permission can affect old cases as well as new requests. Keep contract examples with the integration and rerun the meaningful cases when the adapter or target service changes. This review often reveals the engineering work hidden behind a short tool description, and it gives the project a more realistic implementation scope.

From architecture to an implementation scope

A buildable agent architecture explains where interpretation helps, where code controls the sequence, what tools may do, how progress persists, and how the team will judge the result. It should also show the work that remains with people. That is the foundation for a scope covering the application, integrations, evaluation, review, and operations together.

Bring real examples, the systems involved, the intended authority, and the most consequential uncertainty. Those inputs let an engineering engagement focus on a demonstrable capability rather than a generic agent prototype. The resulting system can then expand through evidence: more supported cases, better recovery, and carefully justified authority.