v0.4.0 — TypeSafe System One 与 Responses API

新功能

• TypeSafe System One：新增 POST /v1/systemone，支持 noul、choice、score 结构化评估；通过 MCP 配置 typesafe_base_url，支持模型别名、团队用量与协议隔离。
• Responses API：新增面向 Codex 的 /v1/responses 桥接，支持工具调用与流式输出；修复并行工具调用、流中断和 reasoning token 统计等边界情况。
• 缓存计费和用量汇总：增加缓存命中 token 计费，以及按周/月查询日用量汇总的 MCP 工具。
• MCP 管理：增加团队删除工具；改善健康探测鉴权、OpenAI base URL 路径处理与 SSE 用量解析内存占用。

升级说明

• 数据库在首次启动时自动升级至 schema v13。升级前请备份 SQLite 数据库、配置、加密密钥材料及旧二进制；活动数据库请使用 SQLite 一致性备份，勿仅复制可能仍有 WAL 写入的主文件。
• 替换二进制后需重启 gateway 服务和已有 MCP stdio 子进程。launchctl 用户需确认服务实际使用的二进制路径，并检查新 PID 和 /healthz。
• TypeSafe 需要自行配置 API key；建议添加独立 provider 和 evaluate → jev-latest 别名，保留已有聊天默认 provider。TypeSafe 不支持聊天或流式请求。
• 本版本包含 release 方案文档，但安装脚本、CLI 自更新与自动 launchctl 部署工具尚未实现。当前发布资产沿用既有归档和 SHA-256 校验和格式。
• 若回滚，先停止写库进程，恢复匹配的旧二进制、迁移前数据库和密钥配置；恢复旧数据库会丢弃备份之后的写入。

验证

• 本地 go test ./...、go vet ./... 通过。
• TypeSafe mock 测试覆盖鉴权、别名与大整数保真、用量、错误、故障切换及数据库迁移；真实 API key 联调尚未执行。
• 发布资产的交叉编译不代表已在每个目标系统完成运行验收。

接入文档：https://github.com/shiftu/llm-gateway/blob/v0.4.0/docs/typesafe.md

完整变更：v0.3.0...v0.4.0

v0.3.0

v0.3.0 — Cognitive Gateway

Theme: The gateway no longer just proxies tokens — it understands reasoning, tools, and agent workflows.

What's new

Cognitive Routing

• Per-provider health probes with SLO rolling window
• Composite scoring (cost × latency × quality × health) with per-team weight overrides
• Cognitive alias mode: gateway picks the highest-scoring eligible provider automatically
• Policy-based fallback on HTTP 429/5xx with configurable chain depth
• Route-decision explain trace: explain_route_trace shows exactly why a request went where it did

Unified AI Semantics

• Reasoning IR: extract and re-emit reasoning content across DeepSeek, Anthropic, GLM, Qwen
• Tool-calls translation: OpenAI ↔ Anthropic function call formats
• Unified finish_reason mapping across all providers

MCP-native Runtime

• MCP resources via lgw:// URI scheme (teams, requests, providers, audit)
• 6 preset MCP prompts for ops workflows (traffic investigation, cost review, provider health, etc.)
• Provider capability registry for eligibility filtering

New Providers

• Qwen / DashScope (OpenAI-compatible, reasoning + tool use confirmed)
• Moonshot / Kimi (OpenAI-compatible)

Trust must-haves

• OpenTelemetry tracing — opt-in via LLM_GATEWAY_OTEL_ENDPOINT (OTLP gRPC)
• Per-team USD budget limits (set_budget, check_budget, list_budgets)
• Multi-key rotation: KEK-wrapped master keys, zero-data-loss rekey_providers
• Helm chart + raw k8s manifests at deploy/helm/llm-gateway/

CEO cherry-picks

• Prompt caching pass-through (cache_control headers forwarded verbatim)
• Tamper-evident audit log: SHA-256 hash chain, verify_audit_chain MCP tool

Migration from v0.2

Drop-in upgrade — schema auto-migrates to v10 on first start. See UPGRADE-v0.2-to-v0.3.md for the full guide.

Stats

• 22 tasks shipped across 4 pillars
• ~120 automated tests, 11 packages
• Binary size: ~12 MB (static, CGO_ENABLED=0)

v0.2.0

Theme: Make the gateway deployable by a second team in under 5 minutes while deepening the three moats (AI-native MCP-only, Chinese-provider-first, single-binary).

Tasks shipped (T1-T8)

• T1 — Provider API key encryption at rest (XChaCha20-Poly1305 AEAD; env > ~/.config/llm-gateway/master.key)
• T2 — Multi-stage Docker image (scratch, ~10MB) + docker-compose
• T3 — /healthz + opt-in Prometheus /metrics
• T4 — Routing rules with conditional model dispatch (prefix / equals / regex; team-scoped or global; match_field=model|kind)
• T5 — Admin audit log with prune + MCP tools, instrumented on every mutating tool
• T6 — Opt-in SQLite-backed RPM bucket persistence (LLM_GATEWAY_PERSIST_RPM=1)
• T7 — mcp_auditor scope with tools/list handshake filtering
• T8 — README v0.2 + docs/UPGRADE-v0.1-to-v0.2.md

Post-T8 hardening

• Docker HEALTHCHECK using the new llm-gateway health subcommand (works on scratch image without curl)
• Dockerfile.local variant for local arm64 cross-compile testing
• /healthz and /metrics no longer require bearer auth (was a T3 oversight blocking probes + Prometheus scraping)
• XDG_CONFIG_HOME=/data in Docker so state.db AND encryption.key both land under the persistent volume
• LLM_GATEWAY_ADDR=0.0.0.0:7421 default in compose so port mapping works
• LLM_GATEWAY_PERSIST_RPM env wired through compose

Upgrading from v0.1

See docs/UPGRADE-v0.1-to-v0.2.md. TL;DR: backup state.db → set master key → start v0.2 binary (migration is automatic, additive only) → re-add provider entries via MCP for encryption.

What's next

v0.3 'Cognitive Gateway' plan is on disk at docs/v0.3-plan.md — 22 tasks across 4 pillars (Cognitive Routing, Unified AI Semantics, MCP-native Runtime, Chinese-provider-first) plus Trust must-haves (OTel, budgets, multi-key rotation, Helm).

v0.1.0

REVISION-3 FINAL multi-tenant rewrite. Tasks T1–T25 all delivered on main.

Delivered scope

Schema / Store (T4-rework + collapsed migration T24): teams, api_keys, providers, model_aliases (composite PK with team_id), request_logs, usage_counters, quotas, model_costs. Migrations v1→v3 collapsed to single final-shape schema.

Inbound (T1 / T1.5 / T6 / T6.5): OpenAI Chat Completions /v1/chat/completions, Anthropic Messages /v1/messages, Bearer auth middleware + structured errors, lgw_ API key path + 60s TTL cache.

Routing (T2a / T3 / T5 / T18): Pass-through preferred over IR translation, DeepSeek provider (byte-fidelity body, URL+header swap), GLM/Zhipu provider, Router resolves model → (provider, upstream_model), Team-scoped alias resolution.

MCP (T7 / T8 / T9 / T22 / T23): MCP stdio server + declarative ACL registry, Provider + model-alias tools, Tenancy tools (team CRUD, key mgmt, quota, usage, logs), Team-scoped alias tools, Streamable HTTP MCP transport at /mcp.

Operations (T10 / T11 / T11.5 / T12 / T13 / T14 / T19 / T20 / T21 / T25): Quota middleware (RPM/day/month), init subcommand + persistent token + boot banner + mcp-config, DeepSeek + GLM model_costs seeded at startup, goreleaser + VERSION ldflags + Makefile + CI workflows, list_request_logs MCP tool, Usage counter write-back + cost tracking, Error reference table + SDK config + MCP tool listing in docs, GET /v1/models with sizing + pricing metadata, README v0.1 with quick start, 5 MCP client snippets, env vars.

Pre-work

• Task 0 wizard-of-oz validation (docs/task-0-wizard-of-oz.md)
• DeepSeek protocol spike (docs/spikes/2026-05-13-deepseek-protocols.md) drove REVISION-2 (pass-through preferred over IR)
• Three plan revisions: REVISION-1 (dual-protocol), REVISION-2 (pass-through), REVISION-3 FINAL (multi-tenant)

Outcome

v0.1.0 is the multi-tenant baseline that v0.2 (Launchability) and v0.3 (Cognitive Gateway) build on top of.