How do they differ?
Both patterns replace one overloaded prompt with several focused LLM calls. They differ in how those calls connect. Prompt Chaining runs them in a line: extract, then classify, then summarize, with each step's output becoming the next step's input. Parallelization runs them side by side: a splitter sends the same input to several independent workers, and an aggregator merges what they return.
The choice comes down to one question about your task: does a step need another step's output before it can start? If it does, those two steps belong in a chain, and nothing you do will make them run at the same time. If it does not, running them in sequence makes the user wait for work that could have overlapped. Latency and cost follow from that answer, and so does the way errors spread.
Side-by-side
| Dimension | Prompt Chaining | Parallelization |
|---|---|---|
| Topology | Sequential: A → B → C | Fan out to independent workers, then fan in |
| Use when | Each step needs the previous step's output | Pieces need the same input but not each other's results |
| Latency | Sum of all steps | Slowest worker plus the merge |
| Input tokens | Each step reads only what it is passed | The shared input is sent to every worker |
| Main failure | An early error flows into every later step | Workers disagree or repeat each other |
| Consistency between pieces | Later steps see earlier output, so they can build on it | Workers never see each other |
| Extra code | Format contracts between steps, partial-failure handling | Concurrency limits, timeouts, retries, an aggregator |
Latency: sum versus max
A chain's latency is the sum of its steps. The Prompt Chaining page's compliance example runs extract, classify, and summarize on the large model in 7.2 s, 3.0 s, and 3.0 s, for about 13.2 s per document. Those steps cannot overlap, because classification needs the extracted phrases and the summary needs the classifications.
A parallel block's latency is the slowest worker plus the merge. The Parallelization page's contract review runs three checks (liability, payment, termination) on the small model. Each takes 0.5 s to the first token plus 300 output tokens at 150 tokens per second, or 2.5 s. Run in parallel, the request takes 2.5 s. Run them one after another and it takes 7.5 s for identical output. Look for that case in your own code: calls written as a sequence that do not depend on each other.
Cost: what each call reads
A chain's input grows only with what you pass forward. In the compliance example only the first step reads the document (2,100 tokens with its instruction); the later steps read 500 and 700 tokens of intermediate output.
Parallelization in its sectioning form sends the shared input to every worker. In the contract example each worker reads the 6,000-token contract plus its 100-token instruction, so the three checks read 18,300 tokens between them and cost $3.29 per 1,000 contracts. One combined prompt covering all three checks reads the contract once and costs $1.47 per 1,000, but takes 6.5 s because it writes all 900 output tokens in one stream. Parallel checks cost about 2.2 times the combined prompt in exchange for about 2.6 times lower latency. With long shared inputs, Prompt Caching on the common prefix reduces that repetition.
When to use Prompt Chaining
Use a chain when the task has stages you could draw as a flowchart and each stage consumes the one before it.
Dependent transformations. Extract facts, classify them, then summarize the classifications. No step can begin before the previous one finishes.
Gates and branches. If step two should only run when step one finds a certain condition, the chain gives you a place to inspect the intermediate output and route it. A parallel block runs every worker regardless.
Different models per stage. The compliance example's chain costs $20.40 per 1,000 documents on the large model, and $8.66 when extraction moves to the small model, with latency of about 9.2 s. Each stage can be tuned and retried on its own.
Outputs that must build on each other. A summary written from the extracted facts stays consistent with them, which two independent workers cannot guarantee.
The Prompt Chaining page's own advice applies here. If the next step depends on what the model discovers at runtime rather than on a flow you can draw in advance, a chain is the wrong tool; that calls for an agent loop (see Prompt Chaining vs ReAct Loop). And if independent steps are pushing a user-facing response past its time limit, run them in parallel instead.
When to use Parallelization
Use a parallel block when several pieces of work need the same input but not each other's results.
Independent checks on one input. Security, style, and test coverage on one pull request. Liability, payment, and termination on one contract. Each check has a short, focused prompt, which often does better than one prompt asked to check everything.
Per-item or per-chunk processing. One call per chunk or list item, with the count known in advance.
Several independent judgments. Voting runs the same task several times and takes the majority or the most conservative answer. Disagreement between workers is a confidence signal a single call cannot give you. Self-Consistency is this form applied to reasoning.
Latency matters more than tokens. The contract example's 2.5 s against 7.5 s is the case where parallelism earns its extra input cost.
The Parallelization page lists three reasons not to use it: steps that depend on each other's output, pieces that need to see each other to stay consistent (such as sections of one essay that must not repeat each other), and running close to your provider's rate limit, since every worker hits the limit at the same moment.
Error propagation versus consistency
The two topologies fail in opposite ways.
In a chain, every step trusts the previous one. If extraction misses an entity, classification never sees it and the summary is incomplete, with no error raised anywhere. The defense is validation between steps: check the intermediate output against a schema or a simple rule before passing it on, and retry the failing step rather than the whole chain.
In a parallel block, an error in one worker stays in that worker's section. With return_exceptions=True a failed worker becomes a marked gap instead of discarding the rest. The cost of that isolation is consistency. Workers never see each other, so a summary worker and a key-risks worker reading the same document can disagree, or repeat the same point in different words. The defense lives in the aggregator: deduplicate, and surface disagreements rather than hiding them. If the pieces truly have to agree, they are not independent, and they belong in a chain.
Can they work together?
Yes, and this is the common case. A chain can fan out at one step and fan back in at the next, giving a workflow that is sequential where it must be and parallel where it can be. In the vocabulary of Multi-Agent Topologies, a parallel block is one node in a sequence, and its merge is the aggregator.
Take the contract review and add a chained step after the fan-in: the large model reads the three checks' findings (900 tokens) plus a 200-token instruction and writes a 300-token risk summary. That step costs 1,100 × $3.00 per million + 300 × $15.00 per million = $0.0078 and takes 0.5 + 300 / 60 = 5.5 s. The whole workflow (three parallel checks, then the summary) costs $11.09 per 1,000 contracts and takes 2.5 + 5.5 = 8.0 s. Running the checks in sequence instead would take 7.5 + 5.5 = 13.0 s for the same cost. The summary has to wait for all three checks, so it stays in the chain; the checks do not need each other, so they run at once.
Orchestrator-Workers has the same shape, a plan, a parallel dispatch, and a synthesis in sequence, except that a model writes the list of workers on each request.
Common mistakes
Chaining steps that do not depend on each other. The most common case is code written as a sequence of await calls because that is how the author thought about the task. Draw the data dependencies before you write the calls. Any step that reads only the original input can move into a parallel block.
Parallelizing steps that share hidden dependencies. Two workers look independent until their outputs have to fit together. If you find yourself writing an aggregator that reconciles contradictions on every request, move those steps into a chain so the later one can see the earlier output.
Dropping context between chain steps. Each step sees only what you pass it. If step three needs a fact from the original input that step one did not carry forward, it is gone. Decide per step what context it needs and pass it explicitly.
Unbounded fan-out. Processing a list of 200 items with one unbounded gather sends 200 requests at once and trips rate limits, after which the retries arrive together as well. Use a semaphore, a per-call timeout, and jittered backoff.
Choosing based on your constraints
If latency is the constraint, find the steps that do not depend on each other and run them in parallel. Wall-clock time drops to the slowest of them.
If token cost is the constraint and the inputs are long, prefer a chain or a single combined prompt, since sectioning repeats the shared input once per worker. Cache the shared prefix if you parallelize anyway.
If correctness depends on the pieces agreeing with each other, chain them so later steps can read earlier ones.
If you are near your provider's rate limit, a chain spreads calls out over time while a parallel block sends them together.
If you are unsure, draw the dependency graph of the task. Its edges are your chain, and the nodes with no edge between them are your parallel block.
References
- Wu, T., et al. (2022). AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. CHI 2022.
- Anthropic. (2024). Building Effective Agents. Describes parallelization as a workflow with two variants, sectioning and voting.
- OpenAI. Function calling documentation. Parallel tool calls in a single model response.