Agent Orchestration Patterns: From Chains to Swarms

🗺️The pattern map and the one rule that governs it

Agent orchestration is the discipline of deciding how work flows between models, tools, and control logic. Before the pattern catalogue, one rule governs all of it: coordination cost grows superlinearly with the number of autonomous decision-makers. A chain has zero coordination cost — the flow is code. A swarm of peer agents negotiating over a task board has coordination as its dominant failure mode. Everything in between is a trade you should make with open eyes.

The second rule: an agent is only worth adding when it has a distinct context, a distinct capability, or a distinct permission boundary. Splitting one agent into three because it feels more "agentic" multiplies cost and latency while dividing nothing that needed dividing. The best multi-agent systems we have seen in production were single agents until a measurable limit forced the split — context overflow, genuinely parallel work, or a security boundary.

A third observation before the details: orchestration patterns compose. A router whose destinations are orchestrator-worker systems, or a chain with a fan-out stage in the middle, is normal. Learn the primitives and their failure signatures; the composite systems fail in ways that are just the sum of their parts.

What follows is ordered from least to most coordination. That ordering is also, not coincidentally, the order you should try them in.

Agent orchestration, defined in the glossary→

🔗Sequential chains: the pattern nobody brags about

A chain is a fixed pipeline: step A output feeds step B, feeds step C, with each step an LLM call, a tool call, or plain code. It is the least glamorous pattern on this list and the one that ships the most business value. If you can write the flowchart of your task before you run it, you have a chain — and you should write it as one, in deterministic code, with the model doing only the steps that actually need a model.

The engineering virtues are underrated. Chains are testable end to end with recorded fixtures. They fail loudly at a specific step, so retries are targeted rather than global. Latency and cost are predictable because the number of model calls is fixed. And quality is tunable per step — a cheap model for extraction, a strong model for the judgment step — which is a lever you lose the moment flow control moves inside the model.

The failure mode is over-rigidity. When inputs genuinely branch — a support ticket that might be billing, might be fraud, might be both — a single chain either grows if-statements until it is untestable, or forces every request down a path that fits it poorly. That is the signal to add exactly one dynamic step: a router at the front, feeding chains behind it. The chains do not disappear; they get chosen.

Resist the temptation to "upgrade" a working chain into an agent loop because agents are the current fashion. An LLM in a loop re-deciding what to do next is strictly less reliable than a chain whose next step is known. Autonomy is a cost you pay for flexibility you have actually demonstrated you need.

🚦Router / dispatcher: one model decision, then determinism

The router pattern puts a single classification step in front of a set of specialized handlers: the model reads the request and picks a route — a chain, a tool, a specialist agent — and everything after the pick is deterministic. Customer support intake, document processing queues, and any "many request types, one front door" system converge on this shape because it concentrates all the model uncertainty into one cheap, auditable decision.

The design work is in the route taxonomy, not the model call. Routes must be mutually exclusive enough that the classifier can be right, collectively exhaustive enough that nothing falls through, and few enough that the choice fits comfortably in context. Systems with forty routes and no "other" bucket misroute constantly; systems with six routes and an explicit fallback misroute rarely and recover gracefully. When two routes overlap heavily, that is a taxonomy bug — merge them or add a disambiguation step, do not hope the model splits the difference.

Confidence handling separates production routers from demos. The classifier should return a route and a confidence, and the dispatcher should have a policy: above the threshold, dispatch; below it, route to a generalist handler or a human queue rather than guessing. A router that always commits converts classification uncertainty into downstream errors that are much harder to trace.

Measure routing as its own subsystem — per-route accuracy against labelled traffic, drift as request mix changes, and the cost of a misroute per route (some are harmless, some are expensive). The router is one model call, but it is the highest-leverage call in the system.

👷Orchestrator-workers: the workhorse of real multi-agent systems

Orchestrator-workers is the pattern most people mean when they say "multi-agent" and get it right. A lead agent decomposes a task into subtasks, dispatches them to worker agents — each with its own focused context, tools, and prompt — and synthesizes the results. Research agents, coding agents that fan edits across a codebase, and report-generation systems all live here.

The pattern pays when two conditions hold: subtasks need different contexts (different slices of a codebase, different document sets) that would blow a single window, or subtasks are independent enough to run in parallel for wall-clock savings. If neither holds, the orchestrator is overhead. This is the pattern where context isolation genuinely earns its complexity — each worker sees only its subtask brief and returns a distilled result, so the lead context stays small and the total token spend is usually lower than one giant monolithic run.

The coordination costs are real and specific. Briefing quality determines everything: a worker given a vague subtask returns a vague result, and the lead cannot tell the difference. Result contracts matter — workers should return structured output (findings, confidence, what was not checked) rather than prose the lead must re-interpret. And the synthesis step is itself an LLM call that can silently drop a worker finding; for high-stakes runs, merge deterministically where the structure allows it and let the model write only the narrative.

Reliability hinges on worker independence. When workers share mutable state — the same files, the same database rows — you get interleaving bugs that look exactly like distributed-systems bugs because they are distributed-systems bugs. Partition work so workers write to disjoint surfaces, or serialize the writers. The models are nondeterministic enough without adding races on top.

🌐Parallel fan-out / fan-in: concurrency without autonomy

Fan-out/fan-in is orchestrator-workers with the intelligence moved out of the fork: code splits the input into independent units (a list of documents, a batch of records, N sections of a report), runs the same or templated operations concurrently, and a join step aggregates. The model does not decide what the units are — the map is deterministic, which is precisely why it scales and tests well.

Use it whenever the same judgment must be applied many times and units do not interact: classification over a queue, extraction over a document set, review comments over files. Compared to orchestrator-workers you give up adaptive decomposition (the split is fixed) and gain everything operations teams care about: bounded concurrency, per-unit retries, rate-limit-aware scheduling, and cost you can compute before you run.

The join step deserves design attention. Three joins cover most systems: reduce (concatenate or aggregate structured results in code — cheapest, most reliable), adjudicate (a model call merges conflicting results, needed when units genuinely overlap), and vote (N samples per unit, majority or weighted — buys accuracy on noisy judgments at N-times cost). Vote in particular is underused: for classification-style units where errors are independent, three samples with majority voting is a cheap, measurable accuracy bump that needs no new architecture.

Operationally, fan-out is where rate limits and cost explode if nobody did the arithmetic. A thousand units times a three-call chain is three thousand model calls; budget it, cap concurrency, and make the join tolerate partial failure so one bad unit does not void the batch.

🏛️Hierarchical teams: when the lead runs out of context

Hierarchical orchestration nests the orchestrator-workers pattern: a top-level lead delegates to sub-leads, each running their own workers. It emerges naturally when task scope outgrows what one lead can brief and synthesize within a context window — multi-department process automation, large migrations, research programmes spanning domains. Frameworks that model "teams of teams" are encoding exactly this.

Understand what you are buying, because the price is steep. Every hierarchy level adds a decomposition round-trip and a synthesis round-trip, so latency and token cost multiply by depth. More subtly, each level is an information bottleneck: the sub-lead summarizes for the lead, who summarizes for the top, and the telephone-game distortion compounds. Requirements that must survive intact — compliance constraints, exact figures, hard exclusions — have to be pinned in shared state every level reads directly, not passed down through summaries.

The failure signature of hierarchy is coherent nonsense at the top: every level did something locally reasonable and the aggregate quietly drifted from the original ask. Mitigations are the ones from real organizations — explicit charters per level, written contracts between levels, and escalation paths — plus one the org chart does not need: evals at every level boundary, because "the sub-lead misunderstood the brief" is only catchable if you test the brief.

Our honest advice: treat two levels as the default ceiling. Three levels and above is occasionally right and usually a sign the scope should be split into separate systems with a deterministic integration, not one deeper agent tree.

⚖️Market and auction patterns: real, rare, and mostly premature

In market-style coordination, peer agents bid for tasks based on self-assessed capability and cost, and an allocator assigns work to winners. It is the most intellectually interesting pattern on this list — it borrows from decades of multi-agent systems research that predates LLMs — and it has genuine appeal for heterogeneous fleets: agents with different models, tools, and price points, where the best assignee varies by task in ways nobody can hard-code.

It is also, as of writing, rare in business production systems, and the reasons are structural. Bidding requires agents to assess their own competence on a task before doing it, and self-assessment is exactly what LLMs do worst — bids cluster near "yes, I can do that" regardless of ground truth, which collapses the market into a lottery with extra steps. Then there is the economics: every task pays a bidding round before work starts, so coordination overhead is paid on the cheap tasks as well as the hard ones. And debugging a misallocation means reconstructing why a market cleared the way it did, which is strictly harder than reading a routing table.

Where the pattern genuinely fits: genuinely open marketplaces (third-party agents with unknown capabilities), adversarial or game-theoretic settings, and research systems exploring emergence. If your fleet is homogeneous — same model, same tools, different prompts — a router with load awareness gives you most of the benefit at a fraction of the coordination cost.

If you do build toward market dynamics, build the boring version first: a scored allocator where bids are computed by deterministic heuristics (queue depth, historical success per task type, cost per call) with the model only providing a capability self-report as one scored input. You get adaptive assignment without trusting the agent to know what it is good at.

🧭Choosing: coordination cost vs what it buys

The decision procedure we run on real engagements is deliberately boring. Start at the bottom of the ladder: can you write the flowchart? Then it is a chain. Do requests arrive in distinct, enumerable types? Add a router in front of chains. Does the task decompose into subtasks needing separate contexts or parallel execution? Orchestrator-workers. Many identical judgments over independent units? Fan-out/fan-in. Only when a lead demonstrably cannot hold the scope do you add hierarchy, and market patterns when assignment itself is unpredictable — which, in most businesses, it is not.

The counter-pressure to name honestly is demo gravity. Multi-agent systems demo beautifully and fail expensively. Every pattern above router level multiplies your debugging surface, your token spend, and the number of prompt interactions nobody has evals for. The teams running agent systems reliably in production are, almost without exception, running simpler topologies than the conference talks suggest — and investing the difference in evals, retries, and observability.

Whichever pattern you choose, the reliability fundamentals do not change: per-component timeouts and retries, idempotent side effects so retries are safe, a trace that follows work across agent boundaries, and an eval suite that tests the seams between agents, not just the end output. Orchestration is distributed systems engineering with a nondeterministic component in every node. Respect both halves of that sentence and any pattern on this list can carry production traffic.

Single-agent vs multi-agent: when the complexity pays→LangGraph vs CrewAI: which framework fits which pattern→Talk to us about agent architecture→