Repository navigation
v0.2.0
Theme: Make the gateway deployable by a second team in under 5 minutes while deepening the three moats (AI-native MCP-only, Chinese-provider-first, single-binary).
Tasks shipped (T1-T8)
- T1 — Provider API key encryption at rest (XChaCha20-Poly1305 AEAD; env >
~/.config/llm-gateway/master.key) - T2 — Multi-stage Docker image (scratch, ~10MB) + docker-compose
- T3 —
/healthz+ opt-in Prometheus/metrics - T4 — Routing rules with conditional model dispatch (prefix / equals / regex; team-scoped or global; match_field=model|kind)
- T5 — Admin audit log with prune + MCP tools, instrumented on every mutating tool
- T6 — Opt-in SQLite-backed RPM bucket persistence (
LLM_GATEWAY_PERSIST_RPM=1) - T7 —
mcp_auditorscope withtools/listhandshake filtering - T8 — README v0.2 +
docs/UPGRADE-v0.1-to-v0.2.md
Post-T8 hardening
- Docker
HEALTHCHECKusing the newllm-gateway healthsubcommand (works on scratch image without curl) Dockerfile.localvariant for local arm64 cross-compile testing/healthzand/metricsno longer require bearer auth (was a T3 oversight blocking probes + Prometheus scraping)XDG_CONFIG_HOME=/datain Docker sostate.dbANDencryption.keyboth land under the persistent volumeLLM_GATEWAY_ADDR=0.0.0.0:7421default in compose so port mapping worksLLM_GATEWAY_PERSIST_RPMenv wired through compose
Upgrading from v0.1
See docs/UPGRADE-v0.1-to-v0.2.md. TL;DR: backup state.db → set master key → start v0.2 binary (migration is automatic, additive only) → re-add provider entries via MCP for encryption.
What's next
v0.3 'Cognitive Gateway' plan is on disk at docs/v0.3-plan.md — 22 tasks across 4 pillars (Cognitive Routing, Unified AI Semantics, MCP-native Runtime, Chinese-provider-first) plus Trust must-haves (OTel, budgets, multi-key rotation, Helm).