When one agent is not enough, and when it already was

Start with the case against it

A great deal of what gets described as agent orchestration is a decision table with a language model sitting in the middle of it. If the routing rules can be written down as a list of conditions, write them as a list of conditions. Rules are cheaper, faster and testable, and they behave the same way on Tuesday as they did on Monday.

A model call earns its place where the input is genuinely unstructured or the judgement genuinely varies. Reading a message somebody typed in a hurry. Deciding what a scanned document is. Drafting a reply that has to sound like a person wrote it. Almost everything around those moments can be plain logic, and plain logic is what you will want at two in the morning when something has gone wrong.

Every agent added to a system adds somewhere for it to be wrong, a cost per run, a delay and one more thing to evaluate. A system that is mostly agents is usually a system whose designer enjoyed building it.

• The decision can be written as a table, which means it should be a table
• The same input has to produce the same output every time, which a model does not guarantee
• There is one tool to call and one obvious thing to do with the result
• Nobody has written down what a correct outcome looks like, so nothing can be measured yet
• The volume is low enough that a person doing it is both cheaper and better

What routing between specialists means in practice

Orchestration starts to earn its keep when one agent needs too many tools and too many instructions to do its job well. An agent with six tools and a page of instructions behaves predictably. The same agent with forty tools and eight pages starts reaching for the wrong tool, and the instructions written for one task begin contaminating another.

The usual shape is a router feeding a set of narrow specialists. The router reads the incoming work, decides which specialist it belongs to, and hands it over with only the context that specialist needs. Each specialist gets a small tool set, a short brief and an explicit statement of what it is not allowed to do.

The router should be the boring part. Cheap to run, easy to test against a fixed set of examples, and allowed to answer that it does not know, which routes the work to a person instead of guessing. Its accuracy gets measured separately from the specialists, because the two multiply. A router that is right nine times in ten feeding a specialist that is right nine times in ten produces a system that is right about eight times in ten, and eight is the number the customer feels.

• One router, cheap and testable, with a fixed set of destinations
• Specialists with small tool sets and short briefs
• An explicit destination for work the router cannot classify, which is a person
• Routing decisions logged alongside the input, so misroutes can be found and fixed
• Accuracy measured at each hop rather than only at the end of the run

Shared state, and who is allowed to write to it

Two agents working the same job have to agree on the facts. The tempting approach is to pass the whole conversation along so each agent sees everything that happened. It works for a while, then it becomes the largest line on the bill, because every step is re-reading every step before it.

Agents should share facts rather than transcripts. A structured record holding the customer, the order, what has been established and what is still unknown is smaller, cheaper, easier to inspect and far less likely to distract the next agent with a stray sentence from three turns ago.

Write access needs to be narrower than read access. Most agents should be able to read the shared record and propose changes while the orchestrator commits them. That one rule prevents the failure where two steps update the same field with different values and the final state depends on which finished first.

• One structured record per job, holding facts rather than conversation
• Read widely, write narrowly, with the orchestrator committing every change
• Every field traceable back to the step that set it
• Anything unknown recorded as unknown rather than left absent, since absent reads as false
• A size limit on what any single step receives, enforced rather than hoped for

When step four of six fails

A run books an appointment, sends a confirmation, updates the CRM, raises an invoice, notifies the team and files a note. If the invoice step fails, the appointment is already booked and the confirmation is already sitting in the customer's inbox. Rerunning the whole thing books a second appointment and sends a second confirmation.

There are three honest ways out and they can be mixed. Make every step safe to repeat, so the whole run can be retried without consequence. Write a compensating action for each step with a side effect, so a partial run can be undone. Or hold the side effects until the end, doing all the reading and deciding first and committing everything in one short stretch at the finish.

Some actions cannot be compensated at all. A message that has reached a customer cannot be unsent, and a payment taken can be refunded but not erased. Those steps belong last, behind whatever is most likely to fail, and anything that goes wrong after them belongs in a queue a person works through rather than in a retry loop.

Cost, latency and the loop that runs all night

Cost per run belongs in an estimate before the first line is written rather than in a bill at the end of the month. The estimate is rough arithmetic: how much context each step receives, how many steps a typical run takes, how many runs a day, and which model does which step.

Most systems use one expensive model everywhere out of habit. Routing, extraction and classification usually work fine on a small one, and the expensive model is worth reserving for the step where the judgement is genuinely hard. Splitting a system along that line often changes the running cost more than any amount of prompt tuning will.

Two limits should be enforced in code rather than trusted to good behaviour. A maximum number of steps per run, so two agents cannot hand work back and forth until morning. And a spend ceiling per run and per day that halts the system and tells somebody, rather than continuing quietly and being discovered later.

Latency is a design constraint anywhere a human is waiting. Somebody in a live chat will wait a few seconds and will not wait a minute, which usually means fewer hops and a faster model at the front. A batch running overnight has a completely different budget and can afford to be careful.

• Cost per run modelled before the build, then measured against the model afterwards
• Small models for routing, extraction and classification
• A hard cap on steps per run, enforced by the orchestrator
• A spend ceiling that stops the system and alerts a person
• A stated latency budget for anything a human is waiting on

When a person is still the better answer

If the job cannot be described to a new employee on a single page, it cannot be orchestrated either. Ambiguity that a person resolves by leaning over and asking a colleague becomes an agent inventing an answer and moving on.

If the tools underneath are unreliable, orchestration multiplies the unreliability instead of absorbing it. Six steps that each work almost every time still fail often enough together to be noticed by the person on the other end.

If nobody has defined what a correct outcome looks like, there is nothing to evaluate, and an agent system without evaluation is an opinion with a subscription. Writing down twenty real cases and the right answer for each is the cheapest work in the project and the most reliably skipped.

And if the volume is small, a person is better. Orchestration pays back on repetition. A process that runs four times a week is a process worth documenting, not one worth handing to a fleet of agents.

How long before it runs, who can turn it off, and when to keep it inside

Orchestration is the part of this field where a build can disappear for a quarter with nothing to show, so the shape is fixed. The audit takes the first week and produces a written map of the steps, the handoffs and every place state gets written. A single specialist doing one step end to end is live inside a fortnight. Routing between two of them comes afterwards, and only once the single one has proved it holds.

Every orchestrated system needs a stop. When a step fails twice, when two agents write conflicting state, or when a run costs more than the job is worth, it halts and waits for a person rather than looping. The queue shows the run, the step that failed and the state at the moment it failed, so somebody can decide rather than guess. Nothing orchestrated should be able to spend all night retrying without anyone hearing about it.

Ownership is why self-hosting matters more here than on a simple workflow. The graph definition, the prompts, the tool credentials and the run history are files, and they end up in your own accounts on infrastructure you control. If you leave, you take a running system rather than a description of one.

Ibrahim owns build and workflows at Wobble, and his usual advice on this is to build less. Doing it in-house is realistic if you have an engineer who will own the state model, because the state model is the whole difficulty and it is not an AI problem. Wobble stays accountable for orchestration it operates, and will say on the call when one agent was already enough.

Common questions

What is AI agent orchestration?

Coordinating several AI agents so work moves between them in a controlled way. A router decides which specialist should handle a piece of work, each specialist has a narrow set of tools, a shared record holds the facts they all need, and the orchestrator decides what happens when a step fails. The coordination itself is usually ordinary code or a workflow tool rather than another model.

When do I actually need more than one agent?

When a single agent needs so many tools and so many instructions that it starts choosing the wrong ones, or when two parts of the job need genuinely different knowledge and permissions. Before that point, one agent with a small tool set and a short brief is easier to build, cheaper to run and far easier to debug.

What is the difference between an AI agent and an automation workflow?

A workflow follows a path decided in advance. An agent decides what to do next based on what it finds, using the tools it has been given. Most working systems are a workflow with an agent at the one or two points where judgement is needed, and any system that is mostly agents is usually more expensive and less predictable than the job requires.

How do you keep the cost of a multi-agent system under control?

Estimate cost per run before building. Use small models for routing and extraction and reserve the expensive one for the hard judgement. Pass structured facts between steps rather than whole transcripts. Then enforce two limits in code: a maximum number of steps per run, and a spend ceiling that stops the system and alerts somebody.

What happens when one agent in a chain fails?

That depends on what the earlier steps already did. Either every step is safe to repeat, so the run can be retried whole, or each step with a side effect has a compensating action that undoes it, or side effects are held to the end and committed together. Actions that cannot be undone, such as a message already sent, should be ordered last, and failures after them belong in a human queue.

Do we need a dedicated agent orchestration platform?

Usually not at first. A workflow tool the team already runs will coordinate agents perfectly well, and keeping the coordination readable matters more than the label on the software. A dedicated framework starts to pay off with many agents, complicated shared state or a real need for detailed tracing, and by then the requirement will be obvious rather than theoretical.