Skip to content

Releases: reginaldosilva27/AgentSimulator

v1.2.0 — Arena

Choose a tag to compare

@reginaldosilva27 reginaldosilva27 released this 23 Jul 22:30

🏟️ Arena — the capacity sandbox

This release adds the Arena: a drag-and-drop capacity sandbox where you compose an agent architecture on a canvas, pour a population of users onto it, and watch a pure, deterministic model tell you where it breaks — which box saturates first, the end-to-end turn latency, and the LLM bill.

The Simulator shows how one request flows; the Arena shows what happens at scale. It sits beside the Simulator as a new top-level page (toggle next to Learn).

Honest by construction (constitution §3): the Arena is an analytical model, not a live load test. It never sends traffic anywhere and emits no TraceEvents — it's frontend-only and persists to localStorage. The numbers are order-of-magnitude teaching benchmarks (the LLM figures are anchored to published Azure OpenAI quota tables), meant to make relative bottlenecks legible — chiefly that the rate-limited LLM is the wall an agent hits long before the databases do.

✨ Highlights

  • Drag-drop capacity canvas — a palette of component kinds, a wired canvas, and a live readout of per-node QPS · utilization · latency · status plus a highlighted bottleneck. Recomputes on every edit; designs persist to localStorage.
  • Real capacity model — offered load propagates through the wired graph (Kahn topological order): min(inbound, capacity) with the excess reported as an honest 429 shed rate. Routers split load 1/N; caches forward only their miss fraction; cycles read unreachable.
  • Load framed in users (Little's Law) — set users + think time; the bar converts to req/s and shows the conversion. A closed-loop equilibrium self-throttles the rate when latency backs the population up (demanded vs effective req/s).
  • Scaling both waysvertical (instance size) and horizontal (replicas), with a per-kind scaling vocabulary (LLM units are deployments with a quota, not containers).
  • ReAct fan-out + Agent Harness — a configurable calls-per-request models the 2–5 model calls a real turn makes; an always-on Agent Harness box makes that fan-out visible while staying transparent to the model (every number is byte-identical with or without it).
  • Architect readoutsend-to-end turn latency and the two LLM bills: provisioned (billed even idle) + usage (served calls only).
  • AI gateway, routing tax & regions — a backend wired directly to N LLM deployments pays a routing tax; an AI gateway / load balancer is exempt and aggregates a fleet. Per-node regions share a regional quota and cross-region hops add latency.
  • Caches, payload & the connection wall — key-value + semantic caches forward only misses; a global workload payload (tokens in/out) moves capacity/latency/cost together; a second status signal tracks streams held in flight against a per-node connection budget.
  • Eight example scenarios with bilingual callouts (Simple RAG agent → Scale the LLM → RAG with a cache → Agent with tools → Semantic cache → Production shape → 100k users → Regional quota → Escape across regions), plus a default sample on first visit.
  • Builder UX — snap-to-connect handles, auto-wire on drop, Backspace edge removal, auto-arrange, per-node/per-edge annotations, contextual nudges.
  • Fully bilingual EN/PT and covered by unit + integration tests (Vitest, 963 green).

📚 Docs

  • New "Arena — the capacity sandbox" section in README.md / README.pt-BR.md (what it is, why/how to use it, what the model captures).
  • CHANGELOG.md [1.2.0] entry.

Full changelog: v1.1.0...v1.2.0

v1.1.0

Choose a tag to compare

@github-actions github-actions released this 24 Jun 02:48

AI Agent Simulator v1.1.0

This release adds a real network edge — the production ingress chain an agent request crosses before the backend, running as actual Docker containers — and a third LLM provider, Google Vertex AI.

⚠️ The network edge runs as containers. Bring it up with docker compose up. Local dev (uvicorn + npm run dev) talks to the backend directly and does not exercise the chain.

🌐 Network edge — the real ingress chain

A request now genuinely travels through a chain of real network appliances, each reporting real evidence (not a diagram):

Hop Container Role
DNS CoreDNS Resolves the upstream service
CDN / cache Varnish Browser-facing front door (:8090); cache HIT / BYPASS
TLS / load balancer HAProxy Terminates TLS 1.3 (single decryption point), load-balances
WAF ModSecurity + OWASP CRS Inspects every request; real attacks get a 403
API gateway Kong Path routing + real rate limiting (a burst returns 429)
  • Click the frontend→backend arrow or any appliance to inspect forwarded headers, cache status, LB pool/algorithm, WAF paranoia level + anomaly threshold, and gateway route + rate-limit policy.
  • WAF block visualization — a blocked request lights the path up to the WAF with a 403 badge and an explanation of the matched OWASP CRS rule, plus a bilingual chat note.

✨ Google Vertex AI provider

A real, opt-in third LLM provider alongside OpenAI and Ollama — contributed by new contributor @elizeureisl in #4. 🎉

  • Bind an agent to Vertex AI, configure GCP project/location + a service-account key (persisted, masked on read, with a step-by-step help tooltip), pick a curated Gemini model, and run the agentic loop against real Gemini — no OpenAI key required for that run.
  • Save & test validates the credentials with a live call. Amends constitution §2 (OpenAI + Ollama + Vertex AI). Bilingual EN/PT throughout.

🐛 Fixed

  • WAF blocked the app's own REST calls — the OWASP CRS default allowed_methods returned 403 for PATCH/PUT/DELETE, breaking agent rename, provider switch, settings save and agent delete through the chain. The WAF now allows those verbs.

🔒 Security

  • Scoped WAF exclusion for secret-carrying settings endpoints/api/settings/* carry opaque secrets (service-account JSON private key, API keys) that false-positive the CRS LFI family (e.g. the credentials field matching rule 930120). A narrow, path-scoped exclusion drops only those rules on those endpoints; the rest of the API keeps full CRS coverage and real attacks stay blocked.

🧩 Also in this release

  • Chunk overlap highlighting — the chunk full-text view highlights the carried overlap prefix; the recursive chunker sub-splits oversized paragraphs.
  • Playwright integration tests — browser E2E driving the live Docker stack through the network chain (manual workflow).

Full changelog: v1.0.1...v1.1.0

v1.0.1

Choose a tag to compare

@github-actions github-actions released this 22 Jun 14:23

What's Changed

Full Changelog: v1.0.0...v1.0.1

v1.0.0

Choose a tag to compare

@reginaldosilva27 reginaldosilva27 released this 22 Jun 11:53

What's Changed

New Contributors

Full Changelog: https://github.com/reginaldosilva27/AgentSimulator/commits/v1.0.0