Multi Agent Architecture: Patterns for Production
Most multi agent systems fail in production for reasons that never show up in a demo.
Blake Aber · Predicate Ventures
A single agent that plans, calls tools, and checks its own work is enough for most tasks. Teams reach for multiple agents when a single context window can no longer hold the job, when different steps need different tools or permissions, or when parallel work beats sequential work. Those are the reasons that justify the added cost. Everything else is architecture for its own sake.
This article covers what changes when you run more than one agent at a time, and the patterns that survive contact with real traffic.
When more than one agent earns its cost
Every agent you add multiplies tokens, latency, and failure surface. A two-agent handoff can double your cost per task before you write a line of application logic. The default should be one agent until you have a specific reason to split.
Three reasons hold up in practice.
The first is context pressure. When one task requires reading more material than a model can hold and reason over at once, splitting the work across agents with separate contexts beats stuffing everything into one call.
The second is permission boundaries. An agent that reads customer data and an agent that writes to production should not be the same process. Separation lets you scope credentials per role and audit each one independently.
The third is genuine parallelism. If three subtasks have no data dependency, running them concurrently cuts wall-clock time. Fan-out only helps when the work is actually independent.
Orchestration patterns that hold up
Orchestrator-worker
One agent owns the plan and delegates discrete subtasks to workers. The orchestrator never does the subtask work itself; it decomposes, dispatches, and assembles results. This keeps the planning context clean and lets workers stay narrow.
The common mistake is letting the orchestrator also hold every worker's full output. Return summaries, not transcripts. The orchestrator needs enough to decide what happens next, not the entire working memory of each worker.
Sequential pipeline
Fixed stages, each with a defined input and output contract. Extraction feeds classification feeds drafting. This is the least agentic pattern and often the right one. When the steps are known in advance, a pipeline is cheaper and more predictable than a planner that rediscovers the same sequence every run.
Use agents inside a stage only where the stage genuinely requires judgment. Hard-code the rest.
Debate and review
One agent produces, another critiques. This catches a class of errors a single pass misses, but it roughly doubles cost and can converge on confident agreement rather than correctness. Reserve it for outputs where the cost of being wrong exceeds the cost of a second model call.
State and handoffs
The hard part of multi agent work is not the agents. It is what passes between them.
Every handoff is a serialization boundary. One agent's conclusion becomes another agent's input, and anything implicit in the first agent's context is lost unless you write it down. Underspecified handoffs are where these systems quietly degrade.
Define explicit contracts for what each agent receives and returns. Treat the message between agents like an API payload, with required fields and validation, not a free-form note. When a handoff fails validation, you want it to fail loudly at the boundary, not three steps later as a confused final answer.
Keep a shared record of what happened. Each agent should append its decision and its reasoning in a form the next agent and your logs can both read. That record is what you replay when something breaks.
Evaluation before deployment
You cannot ship what you cannot measure. Multi agent systems are harder to evaluate than single agents because a bad final answer can originate at any step.
Evaluate at two levels. Per-agent evaluation checks whether each role does its own job given correct input. End-to-end evaluation checks whether the assembled system produces the right result. A system can pass every per-agent test and still fail end-to-end when handoffs lose information.
Build a fixed set of cases with known-good outcomes and run the full system against them on every change. Track where failures originate, not just that they happened. When the orchestrator routes to the wrong worker, that is a different fix than a worker returning a malformed result, and your evaluation should tell them apart.
Guardrails and permissions
Give each agent the narrowest tool set and credentials its role requires. A research agent needs read access and nothing more. A writing agent needs scoped write access to one system, not blanket keys.
Validate tool inputs before execution, especially anything irreversible. An agent proposing a delete or a payment should pass through a check that a human or a deterministic rule approves. The model's confidence is not a safety control.
Bound the loops. Set a hard cap on steps, tool calls, and total spend per task. Agents that can call themselves or each other will, under the wrong conditions, run until something stops them. Make that something a limit you chose, not a bill you discover.
Cost and failure modes
Cost in these systems compounds. Each agent adds its own tokens, retries multiply those tokens, and orchestration overhead sits on top. Measure cost per completed task, not cost per call, because retries and dead ends hide in the gap between the two.
The failure modes worth planning for:
Error propagation. A wrong conclusion early becomes an authoritative input later. Validate at handoffs so errors surface where they start.
Runaway loops. Agents retry or hand off indefinitely without a stopping rule. Cap everything.
Lost context. Information held implicitly in one agent never reaches the next. Make handoffs explicit.
Confident agreement. Review agents ratify bad output instead of catching it. Keep critics adversarial and measure whether they change outcomes.
Cost drift. Token spend grows faster than value as agents multiply. Watch cost per task over time, not just at launch.
What to build first
Start with one agent. Add a second only when context pressure, permission boundaries, or real parallelism forces it. Define handoff contracts before you write orchestration logic. Build evaluation at both levels before you deploy. Cap loops and spend from the first commit.
The systems that last are the ones with the fewest agents that still do the job.