The Short Answer
Use the least complex pattern that solves the problem you have measured, and climb one rung only when you can name the failure that forces you to.
Most teams get this backwards. They read about multi-agent systems, sketch a supervisor with five specialists, and spend a month debugging hand-offs on a task that one well-prompted call with retrieval would have handled. Anthropic's guidance on building agents makes the same point: find the simplest solution possible and add complexity only when it demonstrably improves outcomes. Every pattern in this catalog buys capability with cost, latency, and debugging surface. This guide helps you decide when the purchase is worth it.
The guide has four parts:
- The complexity ladder: the order in which to try patterns.
- The decision questions: five questions that tell you which rung you need.
- The symptom table: "my system does X wrong" mapped to the pattern that fixes it.
- Multi-agent topologies: if you really do need several agents, which shape to use.
The Complexity Ladder
Think of the patterns as rungs. Each rung adds one new capability and one new class of failure. Start at the bottom.
| Rung | What you add | Patterns | New cost | New failure mode |
|---|---|---|---|---|
| 0 | One well-written call | Few-Shot Prompting, Chain-of-Thought | Baseline | Wrong format, shallow reasoning |
| 1 | External knowledge | Basic RAG, Grounded Generation | Retrieval infra, larger prompts | Retrieving the wrong passages |
| 2 | Fixed multi-step flow | Prompt Chaining | 2–5x calls | Errors compounding across steps |
| 3 | Branching by input | Semantic Router, Model Router | A classifier in the hot path | Misrouted requests |
| 4 | Model decides the steps | Tool Calling, ReAct Loop | Unbounded calls per request | Loops, wrong tool, runaway cost |
| 5 | Explicit plan, then execute | Plan-and-Execute | A planning call plus replanning | Stale plans |
| 6 | Several agents | Multi-Agent Collaboration | Several times the tokens of one agent | Lost context between agents |
Rungs 0 to 3 are workflows: your code decides the control flow and the model fills in the steps. Rungs 4 to 6 are agents: the model decides what to do next. Workflows are predictable, cheap, and easy to test. Agents handle open-ended tasks that a workflow can't anticipate, but every run can take a different path, so you need to test them statistically.
The cost jump at the top is real. Anthropic reported that its agents use about 4x the tokens of a chat interaction, and its multi-agent research system about 15x. On Anthropic's internal research evaluation, that system beat a single agent by a wide margin, so the spend can pay off. It pays off for broad, parallelizable research, which is not a reason to make it your default.
The Five Decision Questions
Ask these in order. Each question either tells you to stop at the rung you are on or sends you to the next question.
1. Does a single, well-prompted call fail on real inputs?
Collect a few dozen real inputs before you design anything (30 to 50 is a reasonable start). Write the best single prompt you can, using Few-Shot Prompting for format and Chain-of-Thought for multi-step reasoning, and score the outputs. If it passes, ship it. If you can't say how it fails, you aren't ready to choose a more complex pattern. Build the evaluation first with LLM-as-Judge.
2. Does the model lack information it needs?
If failures come from missing, private, or recent facts, the answer is retrieval, not more reasoning. Start with Basic RAG. If retrieval returns the wrong passages, improve it with Hybrid Retrieval and Retrieval Refinement before you touch the generation side. If answers must be traceable, add Grounded Generation. The production search pipeline guide shows the full stack. Then continue to question 3: retrieval combines with every rung above it.
3. Can you write the steps down in advance?
If a person could draw the flowchart before seeing the input, use a workflow:
- Steps are always the same: Prompt Chaining.
- Steps depend on the type of input: Semantic Router in front of several chains.
- Steps are independent of each other: run them at the same time and merge the results with Parallelization. Self-Consistency is the voting form of this idea.
- Output must meet a bar before it ships: a generate-then-critique loop with Reflection.
If you can draw the flowchart, stop here. You only need an agent when you cannot, because the next step genuinely depends on what the last step found.
4. Does one agent with good tools get the job done?
When you do need an agent, start with one. Give it a small, well-described tool set via Tool Calling or MCP, run it as a ReAct Loop, and set a hard cap on iterations. If it wanders on long tasks, move up to rung 5, Plan-and-Execute, so the plan is explicit and you can inspect it. If it needs procedural know-how rather than more tools, package that as Agent Skills. If one agent meets your quality bar, stop here.
5. Is the single agent failing for a reason more agents would fix?
There are only three good reasons to split into multiple agents:
- Context overflow. The task needs more material than one context window holds well. Sub-agents can each read a slice and return a compressed summary.
- Parallel breadth. The task splits into independent branches, such as researching ten companies, so parallel agents cut wall-clock time.
- Conflicting configurations. Parts of the task need incompatible system prompts, tool sets, or models.
"It would be cleaner as separate roles" is not on the list. Splitting work across agents means splitting context, and each agent makes implicit decisions the others can't see. Cognition's essay on multi-agent systems builds its argument on exactly this: agents should share full context, because every action carries implicit decisions. If the work is tightly coupled and sequential, as most coding tasks are, keep it in one agent and manage the context instead. The context engineering guide covers how.
The Symptom Table
Start from what is going wrong, not from what pattern looks interesting.
| Symptom | Try first | Then |
|---|---|---|
| Output is in the wrong format or tone | Few-Shot Prompting | Structured output via Tool Calling |
| Wrong answers on multi-step reasoning | Chain-of-Thought | Self-Consistency |
| Makes up facts | Basic RAG | Grounded Generation, Self-Check |
| Retrieval misses the right document | Hybrid Retrieval | Retrieval Refinement |
| Questions need several lookups | Agentic RAG | Deep Search |
| Long task falls apart halfway | Prompt Chaining | Plan-and-Execute |
| Needs to act in other systems | Tool Calling | MCP |
| Calculations or data analysis are wrong | Code Execution | — |
| First drafts are mediocre | Reflection | LLM-as-Judge as the critic |
| Forgets earlier turns | Conversation Memory | — |
| Forgets the user between sessions | Long-Term Memory | — |
| Unsafe input or output reaches users | Guardrails | Self-Check |
| Agent takes risky or irreversible actions | Human-in-the-Loop | Guardrails on every tool call |
| Too expensive | Prompt Caching | Model Router, Cascading |
| Too slow | Prompt Caching | Small Language Models, Inference Optimization |
| Hand-tuning prompts doesn't scale | Prompt Optimization | — |
| Can't tell if a change helped | LLM-as-Judge | — |
Most rows are fixed by knowledge, format, or cost patterns rather than by giving the model more autonomy.
If You Need Multiple Agents: Pick a Topology
Multi-agent systems come in a small number of shapes, covered in depth in Multi-Agent Topologies. Most of them are a single-agent pattern from this catalog with an agent at each node instead of a prompt. Pick the shape from the structure of the task.
| Topology | Shape | Use when | Single-agent equivalent |
|---|---|---|---|
| Sequential | A → B → C | Fixed stages with different expertise, like research → draft → edit | Prompt Chaining |
| Parallel | Fan out, then merge | Independent subtasks, or several attempts at the same task | Parallelization, Self-Consistency |
| Router | One classifier picks one agent | Distinct request types needing different tools | Semantic Router |
| Aggregator | Many inputs → one agent | Combining outputs from several sources into one answer | The synthesis step of Deep Search |
| Loop | Producer ⇄ critic | Quality has a checkable bar | Reflection |
| Hierarchical | Supervisors delegate to workers, possibly through further supervisors (one level: Orchestrator-Workers) | Task decomposes at run time into unknown subtasks | Orchestrator-Workers for one level; Plan-and-Execute is the single-agent analogue |
| Network | Agents call each other freely | Rarely the right default; open-ended collaboration only | — |
Some rules of thumb for picking among them:
- Prefer sequential, parallel, and router while they fit. They are workflows with agents at the nodes, so you can still trace and test them like workflows.
- Use hierarchical when subtasks are only known at run time. A supervisor plans, delegates to workers with clean contexts, and synthesizes their compressed results. This one-level form is Orchestrator-Workers and the shape Anthropic used for its multi-agent research system. The Multi-Agent Collaboration pattern covers the supervisor and hierarchical variants in depth.
- Avoid the network topology unless you have a strong reason. When any agent can call any other, debugging means reconstructing a conversation graph after the fact.
- Put a human at the irreversible step. Whatever the topology, actions that send, pay, delete, or deploy should pause for approval with Human-in-the-Loop. Approval gates sit naturally at the edge of the graph, between the agent that proposes and the tool that acts.
- Shared state beats message passing for facts. When several agents need the same facts, put them in a shared store (Shared Memory, or Long-Term Memory across sessions) rather than relaying them through messages, where details get dropped at each hop. When control itself must move to a specialist, use Agent Handoffs with a summary payload.
Common Stacks
Real systems combine patterns. Three combinations cover most of what teams ship.
Knowledge assistant. Semantic Router → Hybrid Retrieval → Retrieval Refinement → Grounded Generation, wrapped in Guardrails, with Prompt Caching on the system prompt and Conversation Memory for follow-ups. No agent required.
Task agent. A single ReAct Loop with Tool Calling through MCP, Code Execution for anything numeric, Guardrails on every tool call, and an approval step before irreversible actions.
Research agent. An orchestrator using Orchestrator-Workers spawns parallel workers running Agentic RAG, then synthesizes with Grounded Generation. LLM-as-Judge scores the final report. This is where multi-agent pays for itself.
Mistakes to Avoid
- Choosing a pattern before you have an eval. Without a scored test set you can't tell whether a rung up the ladder helped. Build the eval first.
- Starting with a framework. Frameworks make rung 6 look as cheap as rung 0. Write the first version against the model API directly so you can see every call.
- Fixing a retrieval problem with reasoning. If the right document never reaches the prompt, no amount of chain-of-thought or reflection will find it.
- Using agents for workflows. If you can draw the flowchart, code the flowchart. A workflow that always takes the same five steps is cheaper and more reliable than an agent that usually does.
- Splitting agents by job title. "Researcher, writer, editor" sounds tidy but often splits context the agents needed to share. Split by context boundary instead.
- Uncapped loops. Every agent and every reflection loop needs a maximum iteration count and a cost budget.
How to Use the Rest of This Catalog
Once you've picked a rung, read the pattern pages for it in order. Each one covers when to use it, how it fails, and what it costs. When two candidates look close, the side-by-side comparisons settle it:
- Prompt Chaining vs ReAct Loop: workflow or agent?
- Prompt Chaining vs Parallelization: sequential or parallel?
- ReAct Loop vs Plan-and-Execute: which agent loop?
- Multi-Agent Collaboration vs Plan-and-Execute: one agent or many?
- Orchestrator-Workers vs Agent Handoffs: supervisor or handoff?
- Basic RAG vs Agentic RAG: fixed retrieval or agent-driven?
- Model Router vs Cascading: which cost strategy?
References
- Anthropic, "Building Effective Agents" (December 2024). Workflows versus agents, and the case for starting simple.
- Anthropic, "How we built our multi-agent research system" (June 2025). Token cost of agents and multi-agent systems, and when parallel sub-agents pay off.
- Cognition, "Don't Build Multi-Agents" (June 2025). Why splitting context across agents causes failures.