Multi-Agent System Design: Lessons From Running Fifteen Agents on One Founder Memory

Multi-agent system design is mostly the discipline of putting every agent on one pipeline and refusing exceptions. We run fifteen specialized agents for early-stage founders on a single per-company memory, and nearly every production bug we have shipped came from an agent that took a shortcut around that pipeline, a lookup that matched nothing, or a model call nobody counted. This is the list of lessons we would hand a CTO before they wire their second agent.

One invoke pipeline, and every agent runs through it

Every model-calling agent must pass through the same invoke path, because the pipeline is where the shared behaviour lives. Ours does five things in order: build the founder context (company profile, agent memories, recent history), inject a ranked block of shared facts, check the caller's tier, call the model through a fallback ladder, and hand the output to a background extractor that writes new facts back into the store.

The failure mode is the bespoke endpoint. Three of our agents had been given dedicated views with their own model calls, for reasons that were sensible at the time: a guided flow, a document pipeline, a drafting tool. Each one lost the shared context read and the write-back, and each one lost the post-run nudge that tells a free-tier founder what was just saved. Nothing errored. The agents answered well. They simply did not take part in the system. The fix was to route them through the same two halves the pipeline uses, and the rule now is that anything new that reaches a model goes through the invoke pipeline unless there is a stated reason. Our overview of what shared context is and how the handoff works covers the founder-facing side of the same design.

Read and write are separate halves, and both must be wired

An agent takes part in shared context only when it both reads the store before its model call and writes facts after it. One half without the other looks fine in a demo and fails in production. An agent that writes but never reads enriches the store without getting smarter.

We found this by audit rather than by alarm. The read half had two call sites in the whole codebase while our own documentation said it fired for all fifteen agents. The dedicated views had been wired to write forward and never to read back. Once both halves went through one helper (a prefix builder before the call and a record-and-extract step after it), every model-calling agent had both, and a test now asserts the mirror between the backend allowlist and the frontend list of agents that claim to save forward. The claim a founder can check, "saved to your shared context", must be true for exactly the agents that make it.

Extraction is fire-and-forget, so stamp its outcome on the request

A background extraction that fails silently is indistinguishable from one that never ran, so every extraction must leave a record of whether it produced anything. Extraction runs after the response has already been returned, behind a broad exception handler, because a failed extraction must never break a founder's answer.

Our HR agent's write-back was broken for the entire life of the feature by one wrong dictionary key: hr where every caller sent hr-agent. A key that matches nothing does not raise, log or fail a request. The allowlist read like coverage while nothing was written. Fixing the key alone changed nothing, because the tool name listed under it did not exist either. Two other agents had tools listed under the wrong agent id, and a third used an id that matched neither of the two spellings the system dispatches. Four bugs, one shape: a lookup that matches nothing.

Two changes closed it. A test asserts that every key in the extraction allowlist is an id the system actually dispatches and every tool listed under it is one that agent has. A health command reports attempted-versus-produced per agent from the stamped outcome on each request log, so a row that reads "attempted, wrote nothing" is an incident rather than a mystery. Do not quote a coverage number you have not re-derived from that report.

The fallback ladder: tripwires, definitive errors and no repeated prompts

A fast model that degenerates under load needs a ladder, and the ladder needs rules or it becomes a cost multiplier. Our fast model degenerates roughly one call in three on very heavy prompts: it loops, pads or trails off. So every agent call carries a tripwire on output size, and a response that trips it is re-sent to a second model. Three rules keep that ladder honest:

• A 400, 403 or 404 from the provider is definitive. A bad request, a permission failure or a missing model will fail identically on the next rung, so it is never retried on the fallback.
• The ladder never sends the same prompt to the same model twice. A retry that repeats an identical call pays twice for the same failure.
• The tripwire costs at most two calls. The worst case for one agent invoke is the fast model plus one fallback, and a test asserts that count.

The ladder also covers the empty-response case: a grounded search call that returns no text twice gets one rung without grounding rather than an endless loop of grounded retries.

Function-calling loops: test the second turn

A tool-calling loop is only proven when the turn after the tool call succeeds, and nothing in a normal test suite checks that turn unless you write it. Our public lead chatbot captured zero leads for seven weeks. It was not disinterest. Every conversation that called a tool failed on the very next turn, because we sent the tool result back on a role the model API does not accept, and pricing questions always called a tool. The highest-intent question a visitor asks was the broken path the whole time.

Two lessons came out of that. First, test the second turn after every tool call in every loop, including the loop you copied from the one that already works. Second, re-baseline before drawing a conclusion from a number: seven weeks of zero leads measured the bug, not the funnel. The loop now appends the model's response parts exactly as returned, so any provider metadata rides along, and the tool result goes back on the role the API expects.

The loop also stops when the same tool with the same arguments errors twice, and when its call cap is spent, because a model that keeps calling a broken tool is spending money to confirm the tool is broken. If you want the wider architecture argument, our piece on the shift from point-solution SaaS to multi-agent AI sets it out.

Every model call is counted, capped and cached

A model call that is not counted is a cost that is not capped, so the rule is that every new model call ships with a test that asserts its call count. The rest follows from that rule:

| Concern | What we do | |---|---| | Uncapped organizations | An organization with no explicit budget row gets a daily floor of five hundred tool calls, not unlimited | | Extraction cost | Extractors run on the cheapest model with a length guard through a bounded background runner | | Repeated prompts | Deterministic prompts (document autofill, resource lists) go through a result cache | | Runaway crons | A prospect is parked after three failed drafts, and every scheduled job stops on one flag | | Frontend double fires | Any effect that triggers a model call checks an in-flight reference first, because development mode mounts twice |

The frontend row surprised us. A generated result that is not persisted is regenerated on the next open, and an effect that fires twice in development runs two paid calls in production. Cost control is a full-stack concern.

Autonomy stops where an action leaves the system

The moment the next step is an email to a real person or a public post, a hard guard or a human sits in the path. Our daily social calendar drafts posts but the drafting code has no way to post. A separate publisher can post an approved row behind four guards: a kill switch that defaults to off, a daily cap per platform, an idempotency key (the platform's own returned post id) and a heartbeat row per run. A reviewer model scores every draft out of 100 and only 90 or above skips the human tap.

The guard must live on the process that acts. Our outbound email program runs on a separate cron service with its own environment variables, and we once set the "require human approval" flag on the API service instead. It looked like it worked, and the cron sent a cold email to a program director who was supposed to be reviewed first. You verify a kill switch by reading it back from the process it governs. Our article on human-in-the-loop AI governance for startups goes through the full guard set.

What to build first if you are starting this week

Build the health report before the second agent. Before you add a third agent, make sure these five exist: one invoke pipeline every agent runs through; a store with both read and write halves wired and a test that the allowlist keys match real dispatch ids; a per-request stamp of whether extraction produced anything, with a command that reports it per agent; a fallback ladder with a tripwire, definitive errors and a maximum call count per invoke; and a test on every model call that asserts how many times it fires. About 1,140 backend tests run on every push from a local pre-push hook that fails closed: if the collected test count drops, the push is blocked, and every production bug becomes a test before its fix ships. That gate is the thing that makes the other four durable. If you would rather see the finished shape than build it, the agent library shows all fifteen on one memory, and our AI Red Team article shows what the store looks like when an agent reads it to argue with you.

Frequently Asked Questions

What is the most common multi-agent system design mistake?

Letting one agent bypass the shared pipeline. A bespoke endpoint answers well in isolation and silently loses the shared context read, the write-back and any cost controls that live in the pipeline.

How do you detect an agent that silently stops writing to shared memory?

Stamp every extraction outcome on the request log and run a per-agent report of attempted-versus-produced. A lookup keyed on the wrong id matches nothing and raises nothing, so only the report can catch it.

When should a fallback model be tried and when should it not?

Try it on a tripped output tripwire, a server error, a safety block or an empty response. Never try it on a 400, 403 or 404, which will fail identically, and never send the same prompt to the same model twice.

What should a function-calling loop test cover?

The turn after the tool call. The first turn returns a tool request and passes in any test; the second turn is where a wrong role or a dropped part fails, and it is the turn that carries the user's real answer.

How do you cap LLM spend across many agents?

Give every organization a daily budget with a floor when none is set, run extractors on the cheapest model behind a length guard, cache deterministic prompts, and ship a call-count test with every new model call.