AI Architecture Decision Guide: Workflow, Agent, RAG, or GraphRAG?

Choose the smallest architecture that can complete the workflow under its real constraints. This guide separates four questions that are often combined: how work moves, where interpretation is needed, how evidence is retrieved, and which actions the system may execute. A workflow can use retrieval without being an agent, and an agent can operate inside a controlled workflow.

Use the guide after describing the process with the Workflow Discovery Template. Record a baseline option, the specific limitation it cannot address, and the evidence that would justify a more complex design. The result is a decision record you can take into an AI Workflow Blueprint.

Compare patterns by the decision they solve

Scroll horizontally to see all columns.

Anthropic distinguishes workflows with predefined orchestration from agents that dynamically direct their own process. That vocabulary helps describe control flow, but does not determine a whole application design. See Building Effective Agents.

Decision 1: who controls the next step?

Write the major steps in order. If the route is known, keep those transitions explicit. A model might classify an incoming request while code chooses the appropriate handler from a validated label. That is a useful AI workflow even when the model cannot invent new steps.

If useful work requires adaptive investigation, define the area in which the system may choose. For example, an agent could select additional approved sources to resolve a discrepancy, while a fixed boundary prevents it from updating customer records. Record maximum attempts, stop conditions, and the handoff when the evidence remains insufficient.

Compare options on the same cases. A more flexible architecture should earn its additional operating cost through better accepted outcomes, fewer unresolved cases, or another observable benefit. The agent architecture article explains graphs, loops, tools, and control in more depth.

Decision 2: what knowledge must be retrieved?

Identify the evidence needed for a correct answer and where it lives. If the task is a lookup, an exact database query or search interface may be sufficient. If the task needs synthesis from text, compare keyword, vector, and hybrid retrieval with source-aware answer generation. Retrieval is not a replacement for source ownership or permission checks.

GraphRAG is a family of approaches, not a universal upgrade setting. Microsoft's implementation includes entity-focused local search and corpus-oriented global search over community reports. Those modes address different query needs. Use Microsoft's query overview to understand the implementation and the RAG vs. GraphRAG article to frame an application comparison.

Decision 3: what authority does the application need?

Separate reading, drafting, recommending, creating, updating, sending, and approving. Write one row for each proposed action. Record its target system, acting identity, required evidence, permitted scope, reviewer, and confirmation of completion. The architecture should enforce these rules outside generated text.

An answer-generation task may need no write tools. A draft workflow may need a place to save a proposal but no permission to send it. An execution workflow needs both authorization and recovery design. Narrowing authority can make a first release useful while the team gathers evidence for a broader scope.

Decision 4: what must survive an interruption?

Record the longest expected wait and the external actions that can occur before it. If the workflow waits for a reviewer or another system, define durable business state, expiration, cancellation, and resumption. A transcript alone may not establish whether a business operation completed.

For writes, decide how duplicate requests and unknown outcomes are resolved. A request that times out may already have changed the target system. Identify the reconciliation mechanism before enabling automatic retries. The long-running agents guide develops this design question.

Complete the architecture decision record

Scroll horizontally to see all columns.

Worked choice: assembling a service response

Suppose a team receives internal service requests and must assemble an answer from approved product documentation. Start with a fixed intake, retrieval, draft, and review sequence. Evaluate whether the retrieved passages actually support the answer and whether reviewers can resolve exceptions efficiently.

If a meaningful subset of requests requires investigating multiple sources in an unpredictable order, test a bounded retrieval agent for that subset. Keep the review boundary fixed. If questions instead concern relationships across products, incidents, and dependencies, test whether graph-assisted retrieval improves those specific cases. These are two different reasons to change the baseline.

The decision record should state what improved, what remained difficult, and which operating cost was added. Avoid adopting both an agent team and a knowledge graph in one experiment if that prevents the team from understanding which change helped.

Take the decision into delivery

Use Custom AI Development when the workflow and architecture are sufficiently defined for an implementation scope. If the architecture depends on a wider platform commitment, AI Technology & Platform Strategy helps connect that decision to the enterprise environment.

Already have a concrete initiative? Discuss an AI project with the completed decision record. It gives the conversation a clear starting point: the work to complete, the boundary to preserve, and the uncertainty to resolve.