Skip to content

v0.2.0 — /compare endpoint + React frontend + rate limiting

Choose a tag to compare

View all tags
@raulmn00 raulmn00 released this 05 Jun 14:54
v0.2.0
1f63959

Demo layer release: a side-by-side comparison endpoint and a production React frontend that consumes it.

Live

URL
Frontend (Vercel) https://agent-router-five.vercel.app
Backend (Cloud Run) https://agent-router-909428365094.us-central1.run.app
Model release https://github.com/raulmn00/agent-router/releases/tag/v0.1.0

Highlights

  • POST /compare endpoint — runs all three routers (DistilBERT, LLM zero-shot via gpt-4o-mini, embeddings + LogReg) on the same input and returns one row per router with measured latency_ms, derived confidence, computed cost_per_1k_usd, and either an intent or an error string. Each router is isolated behind try/except — a single failure leaves the others' rows intact and the endpoint still returns 200.
  • React + TypeScript frontend (Vite, hand-written CSS, no UI library, strict: true, no any). Two modes: Roteamento (single /route call → IntentBadge + ConfidenceGauge + PathFlow + collapsible Trace; the Trace auto-expands when intent is complex_task to show Planner → Executors → Critic) and Comparação (three columns revealing in the order of the measured latency).
  • Confidence everywhere — LLMRouter.classify_with_confidence derives a [0, 1] score from the OpenAI logprobs of the first output token; EmbedRouter.classify_with_confidence returns predict_proba.max(). OpenAIProvider.complete_with_logprobs is a new optional method on the provider abstraction.
  • Rate limiting via slowapi — 30 req/min/IP on /route, 10 req/min/IP on /compare (it burns real tokens). 429 returns a friendly message; the frontend handles it gracefully.
  • Persisted embeddings LogReg — python -m eval.fit_embed_router saves the fitted classifier to eval/models/embed_router.joblib (~25 KB, committed). The backend loads it once at startup; cold-start no longer pays the training cost.
  • ProviderUnavailableError — typed exception in agents/providers.py mapped to 503 by the backend with a generic message that never leaks the env var name.

Migration from v0.1.0

  • New backend dependency: slowapi==0.1.* (in backend/requirements.txt).
  • Optional env var: EMBED_ROUTER_PATH (defaults to <repo>/eval/models/embed_router.joblib).
  • The Dockerfile now copies eval/ into the image and adds it to PYTHONPATH. Cloud Run deploys must be re-built — the v0.1.0 image will start but /compare would 500 with ModuleNotFoundError: eval.
  • /route is behavior-compatible: same request, same response, same 422/503/500 error codes. The only addition is 429 on overuse.
  • Frontend is brand-new (frontend/ subdir of the monorepo). Deployed independently on Vercel; VITE_API_URL points at the Cloud Run backend.

Test counts

router/tests/    11  schema, balance, stratified split, classifier softmax/argmax
agents/tests/    15  orchestrator happy path + critic-reproves-and-retries
backend/tests/   19  /route 4 dispatch paths + /compare cases + 422/429/503/500
eval/tests/      17  accuracy, f1_macro, cost_per_1k pinning + sklearn cross-check
                 ──
                 62

npm run build in frontend/ runs tsc -b first — type-check failures break the build.

Assets 2
Loading