# Building a Multi-Agent System > When one agent juggling too much starts dropping things, the fix is splitting work across several narrower agents. The real orchestration patterns, and the failure modes that come with them. --- # Building a Multi-Agent System One agent, one job, works fine until the job stops being singular. Ask a single agent to research a topic, write it up, and fact-check its own writing, and you'll watch it lose the thread - the research context crowds out the writing instructions, the writing voice bleeds into what should be a neutral verification pass. The usual fix people reach for is "add more agents." Sometimes that's right. Often it trades one problem (a confused agent) for three (coordination overhead, unclear failure attribution, a bigger bill). This guide is for people who've already built a single working agent - see [Building an AI Agent](/guides/building-an-ai-agent) if you haven't - and are deciding whether to split it into several. It assumes you know what an agent is ([AI Agents, Explained](/guides/ai-agents-explained)), how context windows work ([Context Engineering](/guides/context-engineering)), and how to design a self-correcting loop ([Loop Engineering](/guides/loop-engineering)). None of that gets re-explained here. What follows is specific to the moment you add a second agent to the picture: why you'd do it, the shapes that orchestration actually takes, and what breaks that didn't break before. Three phases. First, the clear-eyed case for and against splitting up the work - the coordination problem multi-agent systems solve, and the coordination tax they charge for solving it. Second, three concrete orchestration patterns with nameable shapes: pipeline, fan-out/fan-in, and supervisor-worker, each with a worked scenario. Third, the part that actually determines whether your system survives contact with production: what happens when a sub-agent fails, lies, or loops, and the guardrails - timeouts, retries, output validation, budget caps - that keep one bad sub-agent from taking down the whole run. By the end you'll know when adding an agent is the right call, which shape fits your problem, and how to keep a multi-agent system from silently burning your budget on a Tuesday afternoon. --- # Why One Agent Isn't Enough Give one agent three unrelated jobs - research a topic, write it up, then check the write-up for accuracy - and watch what happens around turn 20. The system prompt for "be a careful fact-checker" is competing with 4,000 tokens of research notes and a half-finished draft, all in the same context window. The agent doesn't announce that it's confused. It starts citing a source that says something slightly different from what it wrote, or verifying its own claim by re-reading its own draft instead of the source. Nothing crashes. The output is worse in ways that are individually hard to catch. This is the coordination problem in miniature: one context window, one set of instructions, three jobs that pull in different directions. [Context Engineering](/guides/context-engineering) covers why a crowded window degrades output in general. Multi-agent design is the specific fix when the crowding comes from *distinct concerns* - not just "too much text" but "too many different jobs stacked in one instruction set." ## What "distinct concerns" actually means Not every long task needs to be split. A single agent that reads five documents and summarizes them is doing one job with a lot of input - that's a context-size problem, not a specialization problem. Solve it with retrieval and summarization, not more agents. The split earns its keep when the *jobs themselves* conflict: - **Different voice or stance.** A writer agent should sound confident and readable. A verifier checking the writer's claims should sound skeptical and literal. One system prompt can't hold both without one leaking into the other. - **Different context needs.** A researcher wants the raw sources in its window. A summarizer wants the researcher's *conclusions*, not the raw sources - feeding it both dilutes the summary with material it should already have distilled past. - **Different failure modes you want to isolate.** If the research step hallucinates a source, you want that caught before it reaches the writing step, not tangled into a draft that's already half-written around it. If none of those apply - if it's genuinely one concern that runs long - a longer, better-managed context on a single agent beats splitting it. Specialization fixes conflicting jobs, not big jobs. ## The fix: one agent, one concern Split along the seams above and each agent gets a narrow, coherent job: a researcher that only researches, a writer that only writes from a research summary, a verifier that only checks claims against sources. Each one's system prompt fits in a paragraph. Each one's context holds only what that job needs. A verifier with a clean, skeptical framing and no attachment to the prose it's checking catches things a self-checking writer won't - it isn't defending its own sentences. ```text single agent, three hats three agents, one hat each ┌─────────────────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ │ research + write + │ → │ Research │→ │ Write │→ │ Verify │ │ verify, one context │ └──────────┘ └────────┘ └──────────┘ │ (instructions compete)│ each: narrow job, clean context └─────────────────────┘ ``` That's the appeal, and it's real. It's also only half the story. ## The tax: coordination overhead is not free Three agents means three places for things to go wrong instead of one, plus a fourth new problem that didn't exist before: getting them to work together. - **Handoff design becomes a real task.** What exactly does the researcher pass the writer - raw notes, a structured summary, a citation list? Get the handoff format wrong and the writer is missing what it needs, or drowning in what it doesn't. This is a new design surface that a single agent never had. - **Latency stacks up.** Three sequential agent calls are three round trips, not one. If each takes 15 seconds, the pipeline takes 45-plus, not 15. - **Cost multiplies.** Every agent has its own system prompt, its own context, its own model call. Three agents doing a job one agent used to do is roughly three times the token spend, sometimes more once retries and handoff overhead are counted in. - **Debugging gets harder, not easier.** When the final output is wrong, which agent caused it - did research miss something, did writing misstate it, or did verification wave it through? A single agent's mistake is at least in one place. A multi-agent mistake requires tracing across a handoff boundary, which is exactly the kind of manual work [Loop Engineering](/guides/loop-engineering) is trying to get an agent to do *for itself* - you don't want to reintroduce it for yourself as the human debugging the system. None of this means multi-agent is wrong. It means the decision is a real trade, not a free upgrade. A rough rule of thumb: split when the concerns genuinely conflict and the task is valuable enough to absorb the coordination cost. Don't split because "multi-agent" sounds more sophisticated than "agent." A single well-scoped agent with a clean context is still the right answer for most tasks - multi-agent earns its complexity on the tasks where one agent's context and voice can't hold every job at once. ```quiz [ { "q": "What kind of task actually benefits from splitting into multiple agents?", "choices": [ "Any task involving more than one document", "A task where the sub-jobs need different voices, different context, or isolated failure modes", "Any task that takes more than 10 turns to finish", "Tasks that need to run faster" ], "answer": 1, "explain": "Splitting pays off when the jobs conflict in stance, context needs, or failure isolation - not merely because a task is long or multi-step. A long single-concern task is usually a context problem, not a specialization problem." }, { "q": "Which of these is a real cost of moving from one agent to three?", "choices": [ "Nothing changes except better output", "Coordination overhead: handoff design, added latency, multiplied cost, and harder debugging", "Multi-agent systems are always cheaper because each agent does less work", "Multi-agent systems eliminate the need for context management" ], "answer": 1, "explain": "Three agents means three round trips, three context windows to manage, a new handoff-format design problem, and a harder question of which agent caused a bad final result." }, { "q": "A single agent summarizing five long documents into one report is struggling. What's the better first fix?", "choices": [ "Split it into five separate agents, one per document", "Improve retrieval and summarization within one agent's context before assuming it needs to be multiple agents", "Add a supervisor agent to manage it", "Increase the step budget" ], "answer": 1, "explain": "This is one concern (summarization) with a lot of input - a context-management problem, not a case of conflicting jobs. Multi-agent splitting solves a different problem than context overload." } ] ``` --- # Orchestration Patterns Once you've decided a task genuinely needs more than one agent, the next decision is shape: how do the agents hand work to each other? Most real systems are built from three patterns, often combined. Each has a distinct data flow, a distinct failure profile, and a scenario it fits naturally. ## Pattern 1 - Linear pipeline Agent A's output becomes agent B's input, in a fixed sequence. No agent decides what runs next - the order is hardcoded by you. ```text [Researcher] → notes → [Writer] → draft → [Editor] → final ``` **Scenario:** a content pipeline. A researcher agent pulls facts and sources into a structured note. A writer agent turns that note into a draft, seeing only the note - not the raw sources, which would dilute its focus. An editor agent tightens the draft for voice and length. Each stage has one clean input and one clean output; nothing loops back. Pipelines are the easiest pattern to reason about because the flow is linear and each handoff is a fixed contract - the writer always gets "a note," never "sometimes a note, sometimes raw sources." Failure is also straightforward to localize: check the stage outputs in order, and the first bad one is the culprit. The limitation is rigidity - a pipeline can't react to something the researcher finds mid-run by changing what the writer is asked to do; the sequence is fixed before it starts. ## Pattern 2 - Fan-out / fan-in Several agents work the same problem independently and in parallel, then a synthesis step combines their outputs. No agent sees another's work until the fan-in step. ```text ┌→ [Agent A: cost angle] ──┐ [Problem] ──fan──┼→ [Agent B: risk angle] ──┼─→ [Synthesizer] → combined answer └→ [Agent C: timeline angle]┘ ``` **Scenario:** evaluating a vendor proposal from three angles at once - one agent assesses cost, one assesses risk, one assesses delivery timeline - running in parallel rather than one agent trying to hold all three lenses in its head sequentially. A synthesis agent then reads all three assessments and produces a single recommendation, explicitly weighing where they agree and where they conflict. Fan-out is the pattern for genuinely independent sub-problems, and its main payoff is speed - three parallel calls instead of three sequential ones - plus each agent staying narrowly focused on its one lens. The cost is at the fan-in step: the synthesizer has to reconcile viewpoints that may disagree, and a synthesizer with a vague brief will average disagreements away instead of surfacing them, quietly hiding the most useful signal the pattern produced. ## Pattern 3 - Supervisor-worker One agent plans and dispatches; it doesn't do the work itself. Worker agents each execute one narrow, well-defined task and report back. The supervisor decides what happens next based on what comes back - the pattern with real branching, unlike the fixed sequence of a pipeline. ```text ┌──────────────┐ │ Supervisor │ ← plans, dispatches, decides next step └──────┬───────┘ dispatch │ │ │ dispatch ┌──────┘ │ └──────┐ ▼ ▼ ▼ [Worker: search] [Worker: code] [Worker: test] │ │ │ └──────report────────────┘ ▼ back to Supervisor ``` **Scenario:** a coding task where the supervisor breaks "add a feature" into steps, dispatches a search worker to find the relevant files, dispatches a code-writing worker once it knows where to edit, then dispatches a test worker to check the result - deciding after each report whether to proceed, retry, or dispatch a different worker. Unlike a pipeline, the next dispatch depends on what the previous worker returned. This pattern handles dynamic, branching work a fixed pipeline can't - the supervisor can react to a worker failing by trying a different worker, or skip a step that turns out to be unnecessary. That flexibility is also its cost: the supervisor itself becomes a single point of failure and a single point of complexity. A supervisor with a vague dispatch policy will spin - reassigning the same failed step to the same worker with the same result, a variant of the loop problem covered in [Building an AI Agent](/guides/building-an-ai-agent). The supervisor's decision logic needs the same rigor as any single agent's loop, plus the added job of interpreting worker reports correctly. ## Picking one Match the shape to how the work actually branches, not to which pattern seems most sophisticated: | Your task looks like... | Use | |---|---| | A fixed sequence of transformations, each stage depending only on the last | Pipeline | | Several independent angles on the same input, none needing the others' work | Fan-out/fan-in | | Work whose next step depends on what just happened, with real decisions to make | Supervisor-worker | Real systems mix these - a supervisor might dispatch a fan-out step as one of its workers, or a pipeline stage might internally be a supervisor-worker loop. Start with the simplest shape that fits, and only add branching complexity where the task actually branches. ```quiz [ { "q": "In a fan-out/fan-in pattern, where does most of the design risk sit?", "choices": [ "In dispatching the parallel agents, which is trivial", "In the synthesis step, where a vague brief can average away real disagreement between the parallel results", "There is no risk once agents run in parallel", "In choosing which model each worker uses" ], "answer": 1, "explain": "The parallel agents are usually the easy part. A synthesizer without a clear brief for handling disagreement will smooth conflicting findings into a bland average instead of surfacing the useful signal." }, { "q": "What distinguishes supervisor-worker from a linear pipeline?", "choices": [ "Supervisor-worker is always faster", "The supervisor decides the next step based on what a worker just reported, rather than following a fixed sequence", "Pipelines can't have more than two stages", "Workers don't report results back" ], "answer": 1, "explain": "A pipeline's order is fixed in advance. A supervisor makes a real decision after each worker's report - retry, proceed, or dispatch something different - which a fixed pipeline cannot do." }, { "q": "You need a fixed sequence: extract data, transform it, load it into a database, with no branching decisions needed. Which pattern fits best?", "choices": [ "Supervisor-worker, for maximum flexibility", "Fan-out/fan-in, to parallelize the stages", "Linear pipeline, since each stage depends only on the previous stage's fixed output", "None - this doesn't need multiple agents" ], "answer": 2, "explain": "A fixed, non-branching sequence is exactly what a pipeline is for. Supervisor-worker's branching logic and fan-out's parallelism aren't needed when the steps and order never change." } ] ``` --- # Failure Recovery and Guardrails Every guardrail a single agent needs - step budgets, tool validation, approval gates, covered in [Building an AI Agent](/guides/building-an-ai-agent) - still applies to each agent in a multi-agent system. What's new here is what happens *between* agents: a sub-agent failing, returning something wrong, or looping costs more than that agent's own budget. It can propagate into every downstream agent that trusts its output. The orchestrator's job is to make sure it doesn't. ## Failure 1 - A sub-agent hangs or times out A worker agent gets stuck mid-tool-call, or its model call never returns. In a single-agent system, that's the whole program stalled. In a multi-agent system, it's worse in a specific way: a supervisor waiting on one worker blocks every step that depended on that worker's result, and if nothing bounds the wait, the whole run hangs indefinitely. The fix is a timeout on every sub-agent dispatch, with an explicit decision for what happens when it fires - not just "the call ends." ```text dispatch("search-worker", task, timeout=30s) → no response in 30s → orchestrator marks this dispatch FAILED, not "still pending" → supervisor decides: retry, reassign to a different worker, or abort this branch ``` A timeout without a defined next step converts a hang into a silent skip instead of a fix. Decide up front whether a timed-out worker gets retried, replaced, or treated as a hard failure for that branch of work. ## Failure 2 - A sub-agent loops The same "can't tell when to stop" problem from single-agent systems ([Loop Engineering](/guides/loop-engineering)) shows up at the orchestration level too, in a form that's easier to miss: a supervisor re-dispatching the same failed task to the same worker, expecting a different result. ```text Attempt 1: dispatch(search-worker, "find the config file") → not found Attempt 2: dispatch(search-worker, "find the config file") → not found ← same task, same worker Attempt 3: dispatch(search-worker, "find the config file") → not found ← still looping ``` A **retry limit** caps this per task, and it should trigger a change in strategy, not just a stop: ```text MAX_RETRIES = 2 if retries_for(task) >= MAX_RETRIES: escalate_to_supervisor("search-worker failed twice on: find the config file") # supervisor tries a different worker, a different approach, or gives up cleanly ``` A retry cap that halts silently is only half a fix - the useful version tells the supervisor to change tactics, since retrying an identical failed dispatch a third time has the same odds as the first two. ## Failure 3 - A sub-agent hallucinates a wrong answer This is the dangerous one, because a hallucinated result doesn't look like a failure - it looks like a normal, confident output. A research worker reports a source that doesn't say what it claims. A code worker reports "tests pass" when it didn't actually run them. Nothing times out, nothing errors. The orchestrator has no reason to distrust it unless something checks. The guardrail is the same principle as [Guardrails That Hold](/guides/prompt-injection-and-guardrails): treat a sub-agent's output as untrusted input to the orchestrator, not as ground truth. Validate before the next agent builds on it. ```text worker_result = dispatch("code-worker", "fix the failing test") # don't trust worker_result.claims_tests_pass on its own actual_result = run_tests() # orchestrator verifies independently if not actual_result.passed: reject(worker_result, reason="claimed pass, tests still failing") ``` Where independent verification isn't possible - a research claim you can't automatically re-check - the fallback is asking a *different* agent to check it, the same way a human editor doesn't proofread their own writing. A verifier agent with no stake in the original answer catches things the original agent, primed to see its own work as correct, will not. The security angle on this - a sub-agent's output potentially carrying adversarial or manipulated content, not just a good-faith mistake - is covered in [Prompt Injection and Guardrails](/guides/prompt-injection-and-guardrails); the orchestration guardrail here handles the good-faith-mistake case, that guide handles the adversarial one. ## Failure 4 - Runaway cost across the whole system A single agent's cost grows with its turn count. A multi-agent system's cost grows with turn count *per agent, multiplied by the number of agents*, and a supervisor that retries liberally or fans out too wide can burn a budget fast without any single step looking expensive in isolation. ```text 3 workers × up to 2 retries each × growing context per call = worst case, several times a single agent's cost, for one "run" ``` Two caps make this bounded instead of open-ended: - **A per-agent step budget**, same as a single agent needs - no worker or supervisor turn runs unbounded. - **A total run budget** the orchestrator tracks across every agent combined, in tokens or dollars, that halts the entire run when hit - not just one agent's slice of it. ```text RUN_BUDGET = 500_000 # tokens, across all agents combined if total_tokens_used >= RUN_BUDGET: abort_run("Hit total budget cap") # partial results so far are still returned - not silently discarded ``` A cap per agent alone misses the case where every agent individually stays under its limit but the sum across a wide fan-out or a retried supervisor loop is what actually blows the budget. Track the total, not just the parts. ## Putting it together None of these guardrails are exotic - timeouts, retry limits, independent validation, and budget caps are ordinary engineering discipline applied to a system where the "code path" is chosen by models at runtime instead of by you at write time. The pattern across all four: don't let one agent's confidence stand in for verification, and don't let any single failure mode run unbounded. Build these before the first real multi-agent run, not after the first surprising bill. ```quiz [ { "q": "A worker agent times out mid-task. What's the guardrail beyond setting a timeout?", "choices": [ "Nothing else is needed - the timeout alone fixes it", "An explicit decision for what happens next: retry, reassign, or abort that branch", "Automatically restart the entire multi-agent run from the beginning", "Ignore the timeout and let the run continue without that worker's result" ], "answer": 1, "explain": "A timeout that ends the call without a defined next step converts a hang into a silent skip. The orchestrator needs a decision - retry, reassign to a different worker, or abort - for what a timeout means." }, { "q": "Why is a hallucinated sub-agent result more dangerous than a timeout or an error?", "choices": [ "It costs more tokens than a timeout", "It looks like a normal, confident, successful output, so nothing flags it unless something independently checks it", "It always crashes the orchestrator", "It only happens with fan-out patterns" ], "answer": 1, "explain": "A timeout or error is visibly a failure. A hallucinated result looks like success - a confident claim with no error signal - so the orchestrator needs to independently validate it rather than trust it by default." }, { "q": "Why isn't a per-agent step budget alone enough to control multi-agent cost?", "choices": [ "Per-agent budgets don't work at all", "Every agent could individually stay under its own budget while the combined total across all agents and retries still runs away", "Step budgets only apply to supervisor agents", "Cost is unrelated to the number of agents" ], "answer": 1, "explain": "Cost in a multi-agent system is roughly per-agent cost multiplied by the number of agents and retries. A per-agent cap doesn't catch a wide fan-out or a retry-heavy supervisor where each piece stays under its own limit but the sum doesn't." } ] ```