Skip to content

v0.2.0

Choose a tag to compare

@shiftu shiftu released this 18 May 04:40
· 53 commits to main since this release

Theme: Make the gateway deployable by a second team in under 5 minutes while deepening the three moats (AI-native MCP-only, Chinese-provider-first, single-binary).

Tasks shipped (T1-T8)

  • T1 — Provider API key encryption at rest (XChaCha20-Poly1305 AEAD; env > ~/.config/llm-gateway/master.key)
  • T2 — Multi-stage Docker image (scratch, ~10MB) + docker-compose
  • T3 — /healthz + opt-in Prometheus /metrics
  • T4 — Routing rules with conditional model dispatch (prefix / equals / regex; team-scoped or global; match_field=model|kind)
  • T5 — Admin audit log with prune + MCP tools, instrumented on every mutating tool
  • T6 — Opt-in SQLite-backed RPM bucket persistence (LLM_GATEWAY_PERSIST_RPM=1)
  • T7 — mcp_auditor scope with tools/list handshake filtering
  • T8 — README v0.2 + docs/UPGRADE-v0.1-to-v0.2.md

Post-T8 hardening

  • Docker HEALTHCHECK using the new llm-gateway health subcommand (works on scratch image without curl)
  • Dockerfile.local variant for local arm64 cross-compile testing
  • /healthz and /metrics no longer require bearer auth (was a T3 oversight blocking probes + Prometheus scraping)
  • XDG_CONFIG_HOME=/data in Docker so state.db AND encryption.key both land under the persistent volume
  • LLM_GATEWAY_ADDR=0.0.0.0:7421 default in compose so port mapping works
  • LLM_GATEWAY_PERSIST_RPM env wired through compose

Upgrading from v0.1

See docs/UPGRADE-v0.1-to-v0.2.md. TL;DR: backup state.db → set master key → start v0.2 binary (migration is automatic, additive only) → re-add provider entries via MCP for encryption.

What's next

v0.3 'Cognitive Gateway' plan is on disk at docs/v0.3-plan.md — 22 tasks across 4 pillars (Cognitive Routing, Unified AI Semantics, MCP-native Runtime, Chinese-provider-first) plus Trust must-haves (OTel, budgets, multi-key rotation, Helm).