Multi-Agent Topologies

Multi-Agent Topologies is a pattern that chooses how several agents connect before deciding what each one does. Seven shapes cover most systems: sequential, parallel, router, aggregator, loop, hierarchical, and network. Each decides who calls whom, who sees which context, and where control flow lives. You read the shape off the structure of the task, then compose shapes like functions.

At a Glance

• Problem: Teams split work across agents by job title and wire them together ad hoc, so nobody can say how many calls a request makes, which agent saw which facts, or why the run stopped where it did.
• Forces: Control flow fixed in code versus control flow chosen by a model; isolated contexts that keep each agent focused versus shared context that keeps their decisions consistent; latency saved by fanning out versus tokens spent repeating the input.
• Solution: Match the task to a shape: fixed stages run sequentially, independent subtasks run in parallel, distinct request types go through a router, and subtasks known only at run time go to a supervisor. Compose shapes and cap the total steps.
• Consequences: Gains: a predictable call count and cost per request, shapes you can test like workflows, and a shared vocabulary for design reviews. Costs: a fixed shape can misfit unusual requests, and composed shapes multiply calls in ways that are easy to underestimate.

What problem does Multi-Agent Topologies solve?

Once a team decides it needs more than one agent, the next question is usually "which agents?" The answer is a list of roles, and the wiring between them is left to a framework default. That is backwards. The connections decide three things that dominate cost and reliability.

The first is call count. A chain of four agents makes four calls; a supervisor that delegates until satisfied makes an unknown number, and you cannot budget for that. The second is context flow. In a chain, each agent sees only what the previous one passed on; parallel workers never see each other's output. Most multi-agent failures are context failures: a fact one agent found never reached the agent that needed it, or two agents made conflicting assumptions. Cognition's "Don't Build Multi-Agents" makes this argument at length. The third is who owns control flow. If your code decides which agent runs next, you can test the system like a workflow. If a model decides, every run can take a different path.

Multi-Agent Collaboration covers how specialist agents coordinate. This page is the step before: choosing the shape from the task, so the coordination you build is the least you need.

How does Multi-Agent Topologies work?

Each topology is a single-agent pattern from this catalog with an agent at each node instead of a prompt.

Sequential (A → B → C) runs fixed stages in order, each consuming the previous output. Use it when the stages are known in advance and need different instructions or tools. It is Prompt Chaining with agents at the steps. Latency adds up, and errors compound along the chain.

Parallel (fan out, then merge) sends the same input to independent agents at once. Use it for independent subtasks, or for several attempts at the same task that you vote on. It is Parallelization at the agent level. Latency is the slowest branch; input tokens multiply by the number of branches.

Router (one classifier picks one agent) sends each request to exactly one specialist. Use it when requests fall into distinct types needing different tools or models. It is Semantic Router with agents as the targets. Each request touches one specialist, and the classifier can run on a small model.

Aggregator (many inputs → one agent) combines several outputs into one answer, usually as the merge end of a parallel fan-out, like the synthesis step of Deep Search. It is where conflicting findings get reconciled, so it often deserves the strongest model.

Loop (producer ⇄ critic) repeats until a check passes. Use it when quality has a checkable bar, such as passing tests. It is Reflection at the agent level, and it needs a hard iteration cap because a critic that never approves will otherwise run forever.

Hierarchical (supervisors delegate to workers, possibly through further supervisors) lets a model decide the subtasks at run time. Its one-level form is Orchestrator-Workers: a lead agent splits the task, hands each piece to a worker with a clean context, and synthesizes the reports. That one-level form is the shape of Anthropic's multi-agent research system. Add further levels only when a subtask is itself too broad for one worker.

Network (agents call each other freely) has no fixed control flow at all. It is rarely the right default, because debugging it means reconstructing a conversation graph after the fact.

The first four shapes keep control flow in your code, and so does a loop whose exit condition is a check you wrote. In the router, the classifier's label picks the branch, but only among branches you listed. Hierarchical and network hand control to a model. That split is the most important property of a topology, which is why you prefer the code-controlled shapes while they fit.

Composition. Real systems nest shapes. A router sends refund requests to a sequential chain and bug reports to a parallel fan-out with an aggregator. Because each shape takes an input and returns an output, they compose like functions, and the call count of the whole follows from the parts: a sequence adds its children, a parallel block adds its branches plus the merge, and a router adds the classifier plus one branch. The loop and the supervisor are the only shapes whose count is not fixed, which is why both need a cap.

When should you use Multi-Agent Topologies?

Ask these questions about the task, in order, and stop at the first that fits:

• Are the stages known in advance and ordered? Use sequential.
• Do several subtasks need the same input but not each other's output? Use parallel, with an aggregator if the results must become one answer.
• Do requests fall into distinct types that need different tools or models? Put a router in front.
• Is there a check that says when an output is good enough? Wrap that step in a loop with a cap.
• Are the subtasks only knowable after reading the request? Use hierarchical.
• Must agents negotiate with no fixed order? Only then consider network, and log every handoff.

When not to use it: if one well-prompted agent passes your evaluation, you need no topology; use a single ReAct Loop or Prompt Chaining and move up only when it fails. Once you have settled on a supervisor or peer design and need the details of roles and handoffs, read Multi-Agent Collaboration instead.

Implementation

# Using OpenAI SDK for illustration — swap client for any provider
import threading
from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI

client = OpenAI(timeout=30, max_retries=2)
MAX_STEPS = 10   # hard cap on agent calls per request, across all branches

class StepBudget:
    def __init__(self, limit: int = MAX_STEPS):
        self.left, self.lock = limit, threading.Lock()
    def spend(self) -> None:
        with self.lock:   # parallel branches share one budget
            if self.left <= 0:
                raise RuntimeError("step budget exhausted")
            self.left -= 1

def agent(instructions: str, model: str = "gpt-4o-mini"):
    def run(text: str, budget: StepBudget) -> str:
        budget.spend()
        response = client.chat.completions.create(
            model=model,
            messages=[{"role": "system", "content": instructions},
                      {"role": "user", "content": text}])
        return response.choices[0].message.content or ""
    return run

def sequential(*steps):   # A -> B -> C: each output feeds the next step
    def run(text, budget):
        for step in steps:
            text = step(text, budget)
        return text
    return run

def parallel(*branches, merge=None):   # fan out on one input, then fan in
    def run(text, budget):
        with ThreadPoolExecutor(max_workers=len(branches)) as pool:
            outputs = list(pool.map(lambda b: b(text, budget), branches))
        joined = "\n\n".join(f"[{i}] {out}" for i, out in enumerate(outputs, 1))
        return merge(f"Request:\n{text}\n\nFindings:\n{joined}", budget) if merge else joined
    return run

def router(classify, routes: dict, fallback):   # one classifier picks one branch
    def run(text, budget):
        label = classify(text, budget).strip().strip(".").lower()
        return routes.get(label, fallback)(text, budget)   # unknown label -> fallback
    return run

# Usage
# triage = agent("Reply with exactly one word: refund, bug, or other.")
# refund = sequential(agent("Extract order ID and amount."), agent("Draft a refund reply."))
# bug = parallel(agent("List likely causes."), agent("List workarounds."),
#                merge=agent("Write one reply to the customer from these findings."))
# support = router(triage, {"refund": refund, "bug": bug}, fallback=agent("Answer briefly."))
# print(support("The app crashes when I upload a photo.", StepBudget()))

Every node, a single agent or a composed shape, takes text and a budget and returns text. That is what lets router take a sequential chain as a branch and parallel take an agent as its aggregator, with no framework in between. One StepBudget travels through the whole request, so the cap covers every call in every branch, and its lock keeps the count correct when parallel branches spend at once. The router normalizes the classifier's reply and sends any label it does not recognize to an explicit fallback instead of raising a KeyError. A failed branch or an exhausted budget raises, so the request fails loudly rather than returning a partial answer that looks complete.

Worked Example: Cost and Latency

A product team answers 1,000 analyst questions a day. Each question comes with about 3,000 tokens of context and needs three analyses (pricing, competitors, reviews) of about 400 tokens each, plus a 600-token final answer. All agents run on the large model.

Assumptions: each analyst reads the context plus a 200-token instruction; in the chain, the second and third analysts also read the earlier analyses (400 and 800 tokens). The writer reads a 200-token instruction plus the 1,200 tokens of analyses. The router row covers questions needing one analysis: a small-model classifier reads 3,100 tokens and writes 5, then one specialist reads 3,200 and writes 600.

The single agent costs 3,200 × $3 per million + 1,800 × $15 per million = $0.0366 and takes 0.5 + 1,800 / 60 = 30.5 s. Each parallel analyst costs $0.0096 + 400 × $15 per million = $0.0156 and takes 0.5 + 400 / 60 = 7.17 s; the writer costs 1,400 × $3 per million + 600 × $15 per million = $0.0132 and takes 0.5 + 600 / 60 = 10.5 s. Parallel totals 3 × $0.0156 + $0.0132 = $0.060, with latency 7.17 + 10.5 = 17.7 s, since the analysts run at once. The chain pays for its growing input ($0.0156 + $0.0168 + $0.018 + $0.0132 = $0.0636) and adds every latency: 3 × 7.17 + 10.5 = 32.0 s. The router's classifier costs 3,100 × $0.15 per million + 5 × $0.60 per million = $0.000468 (0.53 s), and the specialist $0.0186 (10.5 s).

The same four agents cost about the same in sequence or in parallel, but parallel answers in 17.7 s instead of 32.0 s because the analyses are independent. Both cost 1.6 to 1.7 times the single agent, so the split only pays if the separate analyses are measurably better. When a question needs only one analysis, a router in front avoids running all three: it spends $19.07 per 1,000 questions instead of the $60.00 the parallel block would spend on the same questions, and answers in 11.0 s instead of 17.7 s.

What are the common pitfalls?

Choosing the shape by job title. "Researcher, writer, editor" becomes a sequential chain whether or not the stages depend on each other. Draw the data dependencies first: stages that only need the original input belong in a parallel block.

Uncapped loops and supervisors. A critic that never approves and a supervisor that keeps delegating make the call count unbounded. Give every request one step budget that all nodes draw from, and treat hitting it as a failure to investigate, not a normal exit.

A router without a fallback. Classifiers return labels you did not list, extra punctuation, or a sentence. If an unknown label raises or defaults to the most expensive branch, a classifier slip becomes an outage or a cost spike. Normalize the label, route unknowns to a cheap general agent, and log how often that happens.

Starting at the network. Free-form handoffs feel flexible in a demo; in production, nobody can say which agent decided what. Most networks turn out to be a supervisor or a router once the handoffs are written down; build that instead.

Forgetting that compositions multiply. A router whose branches are parallel blocks of loops can make dozens of calls per request. Compute the worst-case call count before shipping and set the step budget from it.

What are the trade-offs?

The code-controlled shapes (sequential, parallel, router, aggregator) give you a bounded call count, a cost you can compute in advance, and traces that look the same on every run. You give up handling requests whose structure you did not anticipate: a router can only send a request to a branch that exists.

The model-controlled shapes (hierarchical and network) adapt to each request, at the price of a variable call count, harder debugging, and statistical evaluation. Anthropic reported its multi-agent research system using about 15 times the tokens of a chat interaction, worthwhile for broad research that parallelizes well, but not a default.

The loop sits between the two, adapting its iteration count while keeping the exit condition in code. Parallel shapes trade tokens for latency; sequential shapes trade latency for building on earlier output. Isolated contexts keep agents focused but risk inconsistent decisions; shared state such as a blackboard (Shared Memory; Hayes-Roth, 1985) or Long-Term Memory keeps facts consistent at the cost of a store every agent depends on.

Goes Well With

Multi-Agent Collaboration is the next step once you have chosen a multi-level hierarchy or a network shape: specialist roles, supervisor and peer coordination, and handoffs that keep context. For a one-level hierarchy, start with Orchestrator-Workers.

Orchestrator-Workers is the hierarchical shape at one level, worked out in detail: a structured plan validated and capped in code, workers bounded by a semaphore and a timeout, and a synthesizer that names failed subtasks instead of filling them in.

Agent Handoffs covers the edges of the router and network shapes when the conversation itself moves to another agent: what the receiving agent is given, and how a cap on handoffs stops two agents from passing a user back and forth.

Shared Memory replaces message relay between nodes with one versioned store. Any shape can use it when several agents need the same facts, and it records who wrote each one and whether it was checked.

Parallelization covers fan-out mechanics for single calls, such as bounding concurrency and handling partial failures. The same rules apply when the branches are agents.

Human-in-the-Loop fits at the edges of any topology: put the approval gate between the agent that proposes an irreversible action and the tool that carries it out.

Reflection is the single-agent form of the loop topology, and its advice on writing a critic that can say "done" carries over to critic agents.

References

• Anthropic. (2024). Building Effective Agents. Describes prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer workflows, and recommends the simplest one that works.
• Anthropic. (2025). How we built our multi-agent research system. A lead agent spawns parallel subagents; reports token use relative to chat.
• Cognition. (2025). Don't Build Multi-Agents. Argues that splitting work across agents with isolated contexts leads to conflicting decisions, and favors sharing full context.
• OpenAI. Agents SDK documentation: Handoffs. One agent delegating a conversation to another.
• Google. (2025). Announcing the Agent2Agent Protocol (A2A). An open protocol for agents built by different vendors to communicate.
• Hayes-Roth, B. (1985). A Blackboard Architecture for Control. Artificial Intelligence.
• Wu, Q., et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv:2308.08155.

Further Reading

• Wu et al., "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation" (2023) — A framework in which customizable agents converse to solve tasks, with examples of two-agent loops and group chats that correspond to the loop and network shapes. arXiv:2308.08155
• Hong et al., "MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework" (2023) — Encodes standard operating procedures as a fixed sequence of role agents passing structured documents, an example of the sequential shape at scale. arXiv:2308.00352