Orchestrating Multi-Agent Swarms with Rust

Our first scheduler was 900 lines of Python and it worked beautifully until the fleet crossed roughly four hundred concurrent tasks. Then the failure mode changed character: not crashes, but slow, correlated drift.

The scheduler is the one component in an agent system that must never be probabilistic. It decides which task runs, against which budget, with which tools — and it has to answer in single-digit milliseconds while holding a consistent view of the fleet.

Backpressure you can reason about

Rust gave us bounded channels with explicit capacity per depot queue. When a downstream tool degrades, the queue fills, the planner sees the refusal synchronously, and the fleet sheds load in a defined order rather than quietly accumulating latency.

let (tx, rx) = mpsc::channel::<Task>(512);
match tx.try_send(task) {
    Ok(_) => metrics.enqueued(),
    Err(TrySendError::Full(t)) => policy.shed(t),
}

Tail latency at p99 fell from 1.9s to 240ms on identical hardware. The more valuable outcome was diagnostic: every shed decision now carries a reason code, so a bad week has an explanation instead of a hypothesis.

What we would not repeat

We rewrote the tool adapters at the same time. Do not do this. Ship the scheduler alone, hold the adapters on the old runtime behind a stable interface, and you keep the ability to bisect a regression to one system.