The Model Wasn't the Hard Part: My Learnings After Shipping Multi-Agent Systems

Every agent demo works. That is the problem with them.

You write a prompt, wire up three or four tools, ask a question, and get back something that looks like magic. Then you put the same thing in front of real users and start finding out what the demo was hiding. A queue delivers the same message twice, so you create the same ticket twice. A connector times out after it has already done the thing you asked it to do. A conversation gets long enough that the one fact the model actually needed is buried under forty turns of noise. Somebody tweaks a prompt to fix one model and quietly makes another one worse. And every so often a tool call is perfectly well-formed, perfectly valid, and does something the user was never allowed to do.

I'm Nabh Patodi, a software engineer fairly early in my career. I have been building agentic systems for about two and a half years now — some as open-source projects I own end to end, some as internal platforms for companies where I was one engineer among several. Most of what follows I learned by getting it wrong first, which is the only reason I trust any of it.

When I started, nearly all of my time went into models and prompts. Now almost none of it does. It goes into context, permissions, durable state, evaluations, retries, traces and recovery. The model was never the hard part.

One agent or several?

This is usually the first real decision you make, and it is the one I see people overthink the most.

My early instinct was to count tools. Too many tools on one agent, split them across specialists. That was never the right question, and it has aged badly. Models have got much better at picking from large toolsets, and honestly I don't have a clean number for where the ceiling sits today — it moves every few months and it depends far more on your schemas than on the count. Twelve tools with fuzzy, overlapping descriptions are harder to use correctly than thirty that are obviously distinct.

The question I ask now is where context, authority, ownership and failure need to be kept apart.

If most of your tasks draw on the same information, run under the same permissions, happen in sequence, and can share one retry policy and one measure of quality, you almost certainly want one agent. The moment any of those diverge — a step needs a large or sensitive context that nothing else should see, a step writes to a system others must not touch, work can genuinely run in parallel, or one component needs its own validation and its own quality bar — you have a boundary worth drawing.

On one internal platform I worked on, users could look things up and take actions across several business systems, internal and external. I could have given a single general-purpose agent the whole conversation, every source, and every write-capable tool. I chose not to, for three reasons that had nothing to do with elegance.

The first was cost and latency. I wasn't using frontier models. I was deliberately running cheaper, faster tiers, and the only way to keep those working well was to keep each agent's job small and its context short.

The second was information flow. That platform touched a lot of systems, and I did not want one agent holding data from all of them at once. One agent per system meant I could see exactly what went where, and it meant a document retrieved for one purpose didn't sit in the context window while an unrelated action was being planned.

The third was blast radius, and it mattered most. Giving one agent the ability to read, create and edit across every connected system means a single confident hallucination can modify something critical or write nonsense into a system of record. Narrow agents with narrow toolsets meant a bad decision had somewhere much smaller to land.

That architecture did what I wanted. Costs stayed predictable, traces became readable, and permissions became something I could reason about per user and per query instead of guessing at.

But I want to be precise about what the split did and did not buy me, because this is where I see the most confident wrong advice.

Multiple agents do not prevent information leakage. They do not enforce permissions. They do not correct each other's hallucinations by existing. Every one of those properties came from something deterministic — scoping the context in code, checking authorisation in the application, validating tool arguments, requiring approval for the risky writes, and having actual recovery logic. The topology made those controls easier to place. It was not itself a control.

And multi-agent has a tax. Every agent you add is another prompt to maintain, another handoff to get wrong, another model call, another context boundary, more latency, and one more thing that can fail. My rule now: add an agent when it creates a real context, authority, ownership, parallelism or recovery boundary. Otherwise you have not built an architecture, you have just added a model call.

The harness is where the product actually lives

Once you have decided on the topology, the interesting question is what is actually running the system. This is the part people were slow to name, and I think it is the single most important layer in an agentic product.

For anyone who hasn't come across the term: the agent harness is everything wrapped around the model. It decides what context gets passed and when, which tools are exposed and whether this particular user is allowed to invoke them, what happens before and after each loop, how outputs are validated, what the loop and budget limits are, where authoritative state is written, how timeouts and retries behave, and what gets traced, evaluated, approved or escalated to a human.

A better model reasons better. It cannot fix an application that hands it the wrong context, hands it too much authority, re-runs a write after a crash, or loses its state entirely. That is the harness's job, and no amount of model progress removes it.

The model proposes. The harness decides what is allowed, records what happened, and cleans up when it goes wrong.

I got very satisfactory results from small, cheap models on systems where the harness was built properly, and I have seen strong models produce a mess inside a weak one. Frameworks help here — LangGraph, and the various tool-loop and middleware and persistence primitives, are genuinely useful — but they hand you mechanics, not decisions. Where your permission boundaries sit, what your authoritative state is, which retries are safe, and what evidence you need before shipping a change are yours to work out.

Context is a budget, not a bucket

My first instinct with context was to treat the window as storage. Put in the conversation, the retrieved documents, the tool responses, and anything else that might conceivably be useful. It felt generous. It was mostly counterproductive.

Irrelevant context costs tokens, adds latency, buries the thing that mattered, and gives the model more raw material for connecting facts that were never related. I now treat the context window as two things at once: a quality budget, and a data-minimisation control.

For every step, I try to answer four questions. What does this component need in order to make this decision? What must it not be shown? What should be fetched only when it turns out to be needed? And what belongs in durable application state rather than in a prompt at all?

It helps to stop thinking of “the context” as one thing. In practice there are several kinds, and they have different lifetimes and different audiences. There is the conversation. There is the current task, its constraints and its expected output. There is retrieved evidence, which should carry its provenance. There is policy and identity — who the user is, what tenant they are in, what they are permitted to do, what has been approved. And there is execution state: which steps are done, what the tools returned, where the checkpoints are, what has already happened out in the world.

The last two should mostly live outside the prompt. Identity, permissions, approvals and receipts belong in your application, and the model should receive only the minimum derived constraint it needs right now.

In practice that means three agents in the same request see three very different things:

user-facing agent
  ├─ last N turns, compacted
  └─ the current request

documentation specialist
  ├─ the current question only
  └─ 5 retrieved passages, each with its source id
     (no conversation history, no other domain's data)

write-capable specialist
  ├─ the proposed action, as typed arguments
  └─ authorised: true
     (not the user's role, not the policy, not the documents
      retrieved twenty minutes ago)

That last one is the one people find counter-intuitive. The write agent doesn't get told why it's allowed to act, or what the user's permissions are — it gets a boolean the application computed. There's nothing there for a persuasive message or a hostile retrieved document to argue with, because the reasoning happened somewhere the model can't reach.

Related to that, and I want to be blunt about it: conversation history is not your record of what happened. A model can be told that an action completed, but your application still needs its own authoritative record of that action, its approval, its result and any external receipt. Asking the model to remember is not state management, it is hoping.

As conversations grow, summarisation helps, but a summary is derived context and not ground truth. The underlying facts should stay retrievable from real storage, and provenance has to survive compaction — a summary that has quietly lost track of which source said what is worse than no summary.

Good context is not the most information you can fit. It is the smallest set you can trust for the step you are on.

Evaluate the path, not just the answer

I stopped trusting one good response the day I watched a small rewording send the same workflow down a completely different route and still produce a plausible-sounding answer.

Monitoring tells you what happened. Evaluation tells you whether that was acceptable, and whether the change you just made improved it. And if you only ever grade the final paragraph, you miss almost everything that can go wrong: whether the right route was chosen, whether the right tool was picked, whether the arguments were valid, whether the user's permissions were respected, whether the evidence used was the evidence retrieved, whether an external action got repeated, whether the system recovered from a bad step without throwing away completed work, and whether it escalated when it should have simply stopped.

I think about this in three layers now.

Component evals test the bounded pieces: retrieval, classification, routing, extraction, tool selection, argument generation. They are the easiest to act on, because when one fails you know exactly what broke.

Trajectory evals look at the path. Did the agent call permitted tools in a sensible order? Did it use the source it claimed to use? Did it ask for approval before doing something expensive? Did it recover from a malformed tool response without losing the four sections it had already written?

End-to-end evals ask the only question the user cares about: did this actually get done, accurately, in acceptable time, at acceptable cost.

Your eval set needs more than the happy path. Ambiguous requests, missing information, permission failures, malformed tool output, duplicate delivery, timeouts, and every uncomfortable edge case production has handed you. Because outputs vary run to run, the cases that matter should be run more than once — a single pass tells you very little. And slice the results by task type, model, prompt version and risk level, because a model that writes excellent summaries can be quietly unsuitable for planning a write.

Production traces are the best source of new regression cases you will ever have. Sanitise them. An eval suite is not an excuse to keep every private prompt and document forever.

The goal is simple: turn “it did something weird that one time” into a test that makes sure it doesn't do it again.

Ship prompts and models like software

Once failures are measurable, the next thing you need is the ability to undo a change.

I don't let production prompts change silently any more. A prompt is a versioned artefact with an owner, a stated purpose, eval results attached, and a rollback path. This is the single cheapest habit on this list and the one I adopted latest, which still annoys me.

But versioning the prompt alone isn't enough, because the behaviour you are actually shipping is a bundle:

prompt version
+ pinned model snapshot, where the provider offers one
+ tool schemas
+ retrieval and index configuration
+ policy version
+ runtime and harness configuration

Change any one of those and the behaviour can change. A trace that only records the model name will not tell you why yesterday was fine and today isn't.

Different models do respond better to different instructions — I have found genuine gains from model-specific tuning. But maintaining a completely separate prompt per provider turns into a maintenance problem faster than you expect, because they drift and you end up with three subtly different products. What has worked better for me is one shared behavioural contract with small, deliberate per-model adaptations layered on top.

Any new prompt-and-model combination goes through the same regression set before it goes out. For anything higher risk, roll it out gradually with enough tracing to compare it against what it replaced. And rollback should mean selecting a known-good release, not asking somebody to reconstruct yesterday's configuration from memory at eleven at night.

On using more than one model

I used to think that having several kinds of task automatically justified having several models. I am more conservative now: start with one default model you understand well, and add routing when your evals show a real difference in quality, latency or cost. Not before.

That said, once you do measure it, the difference is usually there. In the systems I have built, most steps ran perfectly well on mid-tier models — classification, extraction, formatting, simple routing, constrained tool use. Something in the region of 85 to 90 per cent of the work, in my systems, never needed a frontier model. What did need one was ambiguous planning, hard research, complex code, and synthesis where a weak answer would be expensive. Paying frontier prices to reformat a date is just a larger bill for the same outcome.

Two things I would add. Prefer deterministic routing before you reach for an LLM router — if you already know from the entry point that this is a document lookup, you do not need another model call to discover that. And when you escalate to a stronger model, trigger it on something observable: a validation failure, an unsupported capability, a failed cheaper attempt. Not on the model's own claim that it feels uncertain.

The target is not the cheapest model. It is the cheapest model-and-harness combination that reliably clears the bar for that task.

A prompt can ask for restraint; only code can enforce it

If there is one thing on this list I would keep at the cost of everything else, it is this one.

There is a difference between asking a model to follow a rule and making it impossible to break. The first shapes behaviour. Only the second changes what your system can actually do.

You can tell a model not to perform unauthorised actions, and it will mostly comply. It is still not your authorisation layer. A user can phrase a request persuasively. Retrieved content can contain instructions written specifically to be read by your agent. The model can simply misunderstand. None of those should be able to widen what the system is permitted to do.

The way I approach it: separate reads from writes first, because they deserve completely different levels of paranoia. Then for each write, work out whether it is reversible, how much damage it can do, what authority the user actually holds right now, what narrower slice of that authority has been delegated to the agent, and whether it needs explicit approval.

The effective permission for a call is something your application computes from those facts. It is not something the model infers:

def authorise(tool_call, user, session):
    tool = TOOLS[tool_call.name]              # the model can only name tools we registered

    if tool.is_read:
        return Decision.ALLOW if user.can_read(tool.scope) else Decision.DENY

    # Writes: the intersection of what the user holds and what the agent
    # was delegated — never the union, never just one of them.
    effective = user.permissions & session.delegated_scopes

    if tool.required_scope not in effective:
        return Decision.DENY
    if tool.is_irreversible or tool.blast_radius > session.approval_threshold:
        return Decision.NEEDS_APPROVAL

    return Decision.ALLOW

Nothing in that function reads the prompt, the conversation, or the model's stated reasoning. It takes the tool name and the identity, and it answers from application state. The model's role was to propose tool_call; everything after that is arithmetic on facts it doesn't get a vote on.

Check it again immediately before execution, too. Plans are made at one moment and executed at another, and permissions change in between.

Write-capable tools also need strict schemas, bounded inputs, receipts, and a deliberate answer to “what do we do if we don't know whether this already happened” — because some external writes genuinely cannot be made idempotent and you need to decide in advance. Treat tool output and retrieved documents as data, never as new instructions. And if you have a human handoff, give it a real state model: who owns the request now, what context is shared, whether the agent may keep acting in the meantime, and what marks it resolved. A “talk to a human” button is a UI element, not a safety mechanism.

This changed how I think about specialists, too. A specialist is not safe because it has a narrow name and a carefully worded system prompt. It is safe when the application exposes a narrow set of capabilities to it and enforces that boundary outside the model.

Prompts guide behaviour. They do not grant permission.

Retries are not the simple part

Even with the right context and the right permissions, a workflow will hurt you if it forgets what it has already done.

Picture a worker that successfully creates a ticket and then dies before it can record that it did. From your side, the step timed out. Retry it blindly and you have two tickets. A timeout tells you that you stopped waiting. It tells you nothing about whether the work happened.

Long-running agent workflows need the same machinery as any other distributed system: durable jobs instead of one long HTTP request, checkpoints after real progress, ownership leases and heartbeats, bounded retries with backoff, idempotency keys or deduplication, receipts for external effects, and a defined answer for cancellation and for workers that go stale.

I learned most of this the hard way on Research-AI, my open-source deep research system. A single research run could take long enough that tying it to one HTTP request and one process was obviously fragile once I looked at it properly. Moving it into durable jobs meant a worker could disappear without taking the whole report with it, and another could pick the work up without stomping on what was already finished.

Tool-calling makes this considerably worse than normal retry logic, because an assistant message requesting a tool and the tool result answering it form a protocol exchange. If a failure leaves that exchange half-finished, appending another message or replaying the history blindly can produce a request the provider rejects outright — or worse, one it accepts and acts on wrongly. So I now separate warm retries, where the conversation is still trustworthy and can be continued after repair, from cold retries, where I keep the task and throw the conversation away.

Recovery should also be granular. One failed section of a research report should not discard the six that succeeded. One unavailable connector might still allow a partial answer with an honest note about what couldn't be checked. Critical writes are the exception — when their state is uncertain, they should fail closed.

There is enough in this to fill its own article, and it does: Retrying an AI Agent Is Not the Same as Retrying an API Call goes through the leases, checkpoints and transcript repair in detail.

The short version is that a retry is only safe when the operation is idempotent or deduplicated, when you can prove the first attempt didn't commit, or when you can find the committed result and reuse it.

Observability is part of the product

When something fails across several agents, a couple of queues, and three services, a final error message is close to useless.

On one evaluation pipeline I worked on, a single request crossed an API, a queue, cloud infrastructure and services written in different languages. Every component could report itself perfectly healthy while the user watched a run fail. Carrying one correlation ID across all of those boundaries was the change that turned four separate healthy-looking logs into one readable story.

A trace worth having should let you answer: which route and which specialist ran, which prompt and model and tool schema and policy versions were active, which tools were called and what kind of effect each one had, where the latency actually went, which validation or retry or fallback or approval path was taken, and what durable state existed at the moment it broke. Token and cost dashboards are worth having, but they will never tell you why the thing misbehaved.

Correlation has to survive asynchronous hops, too. If your API enqueues a job, a worker calls a specialist, and that specialist calls an external service, it should still be one run when you go looking for it.

The counterweight is that observability must not slide into surveillance. Prompts, retrieved documents, user content, credentials, signed URLs, tool payloads and model outputs are all capable of holding things you should not be storing. Prefer structured event names, opaque identifiers, bounded summaries, timings, state transitions and sanitised error categories over dumping raw payloads and sorting it out later.

Also, monitor more than the model. Agent health depends on database connectivity, queue depth, connector credentials, token expiry, third-party APIs, browser workers and background schedulers. A model endpoint returning 200 OK says nothing about whether your product works.

And some of this is user-facing. Long-running work should show progress derived from actual completed steps or durable checkpoints. An animated bar with no relationship to reality is a small lie you are telling on every single run.

My test for whether the tracing is good enough: can I explain a failed run without reading anyone's raw prompts and payloads? If not, it is either too thin or too invasive.

What I check before calling something production-ready

Deploying gives a system a URL. Shipping means you know how it behaves on a bad day — duplicate events, expired credentials, a backed-up queue, a worker that dies mid-run, a schema change, a provider outage.

Before I am willing to call an agent workflow production-ready, I want real answers to these:

• Can I explain why this needs more than one agent?
• Is each component's context scoped to the minimum it needs?
• Is authority enforced outside the model?
• Can a duplicate delivery cause a duplicate external action?
• What happens if a worker dies halfway through?
• Can the workflow resume from durable state?
• Are prompts, models, tool schemas and policies versioned together?
• Do the evals cover trajectories, not just final answers?
• Can I reconstruct a failure without storing sensitive content?
• Is there an explicit approval and human-escalation path?
• Can I roll back in one step?

If those don't have answers, the system might be deployed. It isn't dependable yet.

Where I've ended up

The biggest shift in how I work has been giving up on the question I started with, which was some version of “how do I make the agent smarter?”

The question I ask first now is what has to stay true when the agent is wrong. Not if — when. If I can answer that in terms of context, authority, durable state, evaluations and recovery, I am probably building something that will hold up. If I can't, then adding another agent has not made the system more capable. It has just given the uncertainty one more place to go.