Skip to content

v0.2.0 — Production Release: From Demo to Product

Choose a tag to compare

View all tags
@1304674612 1304674612 released this 10 Jul 02:15
v0.2.0
40c74f2

AgentBench v0.2.0 — Production Release

This release transforms AgentBench from an alpha prototype into a production-grade product. 12 work streams delivered simultaneously across 182 files (21,179 additions).

🔴 P0: Critical Foundations

Authentication (S1) — Full NextAuth.js v5 integration

  • Credentials provider (email + password) + GitHub + Google OAuth
  • Route protection middleware, sign-in/sign-up pages, registration API
  • Real API key validation against database (replaces mock ab-* prefix)
  • Seed admin user: admin@agentbench.dev

Testing (S2) — 342 unit tests across 14 test files (was 51 in 3)

  • Every core module now tested: runner, tracer, assertion DSL, evaluators, snapshot, diff, replay, experiment, coverage, reporter
  • 31 expanded API integration tests

Streaming (S3) — SSE stream capture

  • New StreamCapture class: OpenAI + Anthropic SSE parsing
  • Tracer.traceLLMCallStream() — end-to-end streaming trace
  • SDK streaming with tee'd streams for simultaneous user + trace consumption

🟡 P1: Ecosystem Expansion

SDK Adapters (S4) — New @agentbench/langgraph package

  • Real LangGraphAdapter with run() and stream() support
  • Duck-typing design — no hard dependency on LangChain

Production Hardening (S5)

  • GET /api/health — DB + Redis health check
  • Token bucket rate limiter — 100 req/min per API key/IP
  • Centralized Zod schemas for all API inputs
  • CORS middleware, unified ApiError handler, request logging

Examples (S6) — 3 complete example projects (21 files)

  • Customer Support Agent, Code Review Agent, Research Agent
  • 1066-line seed script with demo data

Python SDK (S7) — pip install agentbench

  • 15 files: Runner, Tracer, Assertion DSL, CLI, HTTP client
  • OpenAI + Anthropic wrappers for automatic tracing
  • 49 tests, all passing

🟢 P2: Polish & Delivery

CLI Modularization (S8) — 734 lines → 16 files

  • 11 command modules + 4 shared libraries
  • New agentbench dev command with file watcher
  • Progress spinners, --json on all commands, --debug flag

Feature Refactoring (S9)

  • 8 shared UI components + 7 feature components
  • Loading skeletons, barrel exports from @/features

Email Notifications (S10)

  • nodemailer + 4 HTML templates + notification API
  • In-app notification dropdown with unread badge
  • Auto-notify on run completion + regression detection

Performance & SEO (S11)

  • Full metadata on all 12+ pages, OG image API, sitemap, robots.txt
  • PWA manifest, bundle optimization

Publishing (S12)

  • All 5 packages build to dist/ ready for npm
  • Multi-stage Dockerfile + docker-compose.prod.yml
  • Changesets + GitHub Release workflow

📊 Stats: v0.1.0 → v0.2.0

Metric v0.1.0 v0.2.0
Tests 51 391+
Test files 3 17
Packages 8 9
API Endpoints 18 20
CLI Commands 8 11
Shared UI Components 3 11
Examples 0 3
Python Support None Full SDK (49 tests)
Auth Mock NextAuth.js + OAuth
Streaming No Full SSE capture
Docker DB only Full production

Full Changelog: https://github.com/1304674612/agentbench/blob/main/CHANGELOG.md

Assets 2
Loading