🏟️ Arena — the capacity sandbox
This release adds the Arena: a drag-and-drop capacity sandbox where you compose an agent architecture on a canvas, pour a population of users onto it, and watch a pure, deterministic model tell you where it breaks — which box saturates first, the end-to-end turn latency, and the LLM bill.
The Simulator shows how one request flows; the Arena shows what happens at scale. It sits beside the Simulator as a new top-level page (toggle next to Learn).
Honest by construction (constitution §3): the Arena is an analytical model, not a live load test. It never sends traffic anywhere and emits no
TraceEvents — it's frontend-only and persists tolocalStorage. The numbers are order-of-magnitude teaching benchmarks (the LLM figures are anchored to published Azure OpenAI quota tables), meant to make relative bottlenecks legible — chiefly that the rate-limited LLM is the wall an agent hits long before the databases do.
✨ Highlights
- Drag-drop capacity canvas — a palette of component kinds, a wired canvas, and a live readout of per-node QPS · utilization · latency · status plus a highlighted bottleneck. Recomputes on every edit; designs persist to
localStorage. - Real capacity model — offered load propagates through the wired graph (Kahn topological order):
min(inbound, capacity)with the excess reported as an honest 429 shed rate. Routers split load1/N; caches forward only their miss fraction; cycles read unreachable. - Load framed in users (Little's Law) — set users + think time; the bar converts to req/s and shows the conversion. A closed-loop equilibrium self-throttles the rate when latency backs the population up (demanded vs effective req/s).
- Scaling both ways — vertical (instance size) and horizontal (replicas), with a per-kind scaling vocabulary (LLM units are deployments with a quota, not containers).
- ReAct fan-out + Agent Harness — a configurable calls-per-request models the 2–5 model calls a real turn makes; an always-on Agent Harness box makes that fan-out visible while staying transparent to the model (every number is byte-identical with or without it).
- Architect readouts — end-to-end turn latency and the two LLM bills: provisioned (billed even idle) + usage (served calls only).
- AI gateway, routing tax & regions — a backend wired directly to N LLM deployments pays a routing tax; an AI gateway / load balancer is exempt and aggregates a fleet. Per-node regions share a regional quota and cross-region hops add latency.
- Caches, payload & the connection wall — key-value + semantic caches forward only misses; a global workload payload (tokens in/out) moves capacity/latency/cost together; a second status signal tracks streams held in flight against a per-node connection budget.
- Eight example scenarios with bilingual callouts (Simple RAG agent → Scale the LLM → RAG with a cache → Agent with tools → Semantic cache → Production shape → 100k users → Regional quota → Escape across regions), plus a default sample on first visit.
- Builder UX — snap-to-connect handles, auto-wire on drop, Backspace edge removal, auto-arrange, per-node/per-edge annotations, contextual nudges.
- Fully bilingual EN/PT and covered by unit + integration tests (Vitest, 963 green).
📚 Docs
- New "Arena — the capacity sandbox" section in
README.md/README.pt-BR.md(what it is, why/how to use it, what the model captures). CHANGELOG.md[1.2.0]entry.
Full changelog: v1.1.0...v1.2.0