Skip to content

v1.2.0 — Arena

Latest

Choose a tag to compare

@reginaldosilva27 reginaldosilva27 released this 23 Jul 22:30

🏟️ Arena — the capacity sandbox

This release adds the Arena: a drag-and-drop capacity sandbox where you compose an agent architecture on a canvas, pour a population of users onto it, and watch a pure, deterministic model tell you where it breaks — which box saturates first, the end-to-end turn latency, and the LLM bill.

The Simulator shows how one request flows; the Arena shows what happens at scale. It sits beside the Simulator as a new top-level page (toggle next to Learn).

Honest by construction (constitution §3): the Arena is an analytical model, not a live load test. It never sends traffic anywhere and emits no TraceEvents — it's frontend-only and persists to localStorage. The numbers are order-of-magnitude teaching benchmarks (the LLM figures are anchored to published Azure OpenAI quota tables), meant to make relative bottlenecks legible — chiefly that the rate-limited LLM is the wall an agent hits long before the databases do.

✨ Highlights

  • Drag-drop capacity canvas — a palette of component kinds, a wired canvas, and a live readout of per-node QPS · utilization · latency · status plus a highlighted bottleneck. Recomputes on every edit; designs persist to localStorage.
  • Real capacity model — offered load propagates through the wired graph (Kahn topological order): min(inbound, capacity) with the excess reported as an honest 429 shed rate. Routers split load 1/N; caches forward only their miss fraction; cycles read unreachable.
  • Load framed in users (Little's Law) — set users + think time; the bar converts to req/s and shows the conversion. A closed-loop equilibrium self-throttles the rate when latency backs the population up (demanded vs effective req/s).
  • Scaling both waysvertical (instance size) and horizontal (replicas), with a per-kind scaling vocabulary (LLM units are deployments with a quota, not containers).
  • ReAct fan-out + Agent Harness — a configurable calls-per-request models the 2–5 model calls a real turn makes; an always-on Agent Harness box makes that fan-out visible while staying transparent to the model (every number is byte-identical with or without it).
  • Architect readoutsend-to-end turn latency and the two LLM bills: provisioned (billed even idle) + usage (served calls only).
  • AI gateway, routing tax & regions — a backend wired directly to N LLM deployments pays a routing tax; an AI gateway / load balancer is exempt and aggregates a fleet. Per-node regions share a regional quota and cross-region hops add latency.
  • Caches, payload & the connection wall — key-value + semantic caches forward only misses; a global workload payload (tokens in/out) moves capacity/latency/cost together; a second status signal tracks streams held in flight against a per-node connection budget.
  • Eight example scenarios with bilingual callouts (Simple RAG agent → Scale the LLM → RAG with a cache → Agent with tools → Semantic cache → Production shape → 100k users → Regional quota → Escape across regions), plus a default sample on first visit.
  • Builder UX — snap-to-connect handles, auto-wire on drop, Backspace edge removal, auto-arrange, per-node/per-edge annotations, contextual nudges.
  • Fully bilingual EN/PT and covered by unit + integration tests (Vitest, 963 green).

📚 Docs

  • New "Arena — the capacity sandbox" section in README.md / README.pt-BR.md (what it is, why/how to use it, what the model captures).
  • CHANGELOG.md [1.2.0] entry.

Full changelog: v1.1.0...v1.2.0