v1.8.0

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut.

[1.8.0] - 2026-07-27

Trust tier becomes live policy input, config authority grows a command
line, and the admin console gains the pages it was missing. This release
also moves the vendored Pingora fork onto upstream 0.8.1, which carries
security fixes; see Security below.

Security

• Pingora updated to upstream 0.8.1. The vendored fork was based on
  0.8.0 and has been rebased onto 0.8.1, picking up an HTTP/2 server
  limit bound that mitigates a memory-exhaustion vector, plus the fixes
  for RUSTSEC-2026-0098 and RUSTSEC-2026-0099. Every deployment
  terminating HTTP/2 should take this release. SBproxy's three local
  patches (dynamic rustls cert resolver, the
  upstream_response_decision retry hook, and the refusal to retry once
  response bytes have reached the client) are unchanged.

Added

• The admin console reports context compression. A Compression page
  lists the sessions whose history has been externalized to a summary,
  with tokens covered, summary size, and the resulting ratio. Summary
  text is never listed, only its size and provenance.
• The admin console reports who can sign in. A Users page lists each
  account and its role over a new read-only GET /api/admin/users.
  Accounts remain config (admin.username, admin.operators), so the
  route reports and does not mutate, and passwords are never included in
  the response.
• Spend links through to the requests behind it. Origin rows in the
  spend breakdown open the request log filtered to that origin.
• Trust tier is now live policy input. The request path combines
  authentication and agent-detection evidence into suspicious, strong,
  named, or anonymous; CEL expression and assertion policies can read
  request.trust_tier, and sbproxy_trust_tier_requests_total reports the
  closed-set distribution. Verified Web Bot Auth resolves to strong.
• Operate a config authority from the command line. Running one used
  to mean hand-rolled curl. sbproxy config authority init generates
  the Ed25519 signing key owner-only, writes the verifying-key file
  subscribers install, and prints what to copy where; it refuses to
  overwrite an existing key, and --force rotates by adding the new
  verifying key beside the old one so subscribers keep verifying while
  they are updated. publish runs the same three validation steps the
  authority runs, through the same code, so a payload that would be
  refused is refused locally before a revision number is spent on it.
  status shows the current revision, the key id, and every subscriber's
  last-seen revision, which is fleet drift visible from a terminal.
  rollback republishes the previous revision's payload under a new
  revision number, because a subscriber's anti-replay cursor refuses
  anything that does not move forward. subscriber add | list | revoke
  manages credentials, and add prints the credential exactly once and
  says so. Every command that changes what the fleet sees goes over the
  admin API and reports what the server returned, and an unreachable
  authority is a distinct non-zero exit rather than something local that
  looks like success. New admin route:
  POST /admin/config-authority/rollback.
• Preview the configuration an authority would push, before it lands.
  sbproxy config pull --dry-run runs a real subscriber cycle up to the
  point of applying: conditional fetch, signature and digest and replay
  verification, the merge over the local document, and the
  unresolved-${VAR} screen. Then it prints the plan diff and stops. The
  bundle cache is not written, the replay cursor is not advanced, and
  nothing reloads.
• Subscribe to signed configuration from an upstream authority. A new
  proxy.config_authority.upstream block points a node at an authority
  that publishes signed configuration bundles. The node polls, verifies
  the signature against the keys it trusts, merges the payload over its
  own file, and applies the result through the same reload transaction a
  SIGHUP takes, so a bad bundle is rejected before anything is published
  and the previously applied configuration keeps serving. Paths that
  describe the box rather than the fleet are refused outright: listeners,
  TLS material, the admin surface, secret backends, cluster identity, and
  the authority block itself. A monotonic cursor refuses a replayed or
  rolled-back revision, including across a restart, and the verified
  bundle is cached so an unreachable authority costs nothing but a
  climbing staleness gauge. mode: overlay merges over the local file;
  mode: replace treats the bundle as the configuration and will not
  start without one. Bundles that still reference an environment
  variable the node does not set are refused rather than applied as
  literal text, because nobody is reading the log on a hundred machines
  at once. New metrics: sbproxy_config_bundle_revision,
  sbproxy_config_bundle_age_seconds,
  sbproxy_config_bundle_fetch_total,
  sbproxy_config_bundle_applied_total, and
  sbproxy_config_bundle_applied_degraded_total.
• A response-cache store you can pick. The response cache has had
  four storage backends for a while, but only one of them was reachable:
  nothing in the pipeline built the others, so no config could ask for
  them. The new top-level proxy.response_cache_store block selects
  memory, file, memcached, or redis and the pipeline builds what
  it names. file gives you a cache that survives a restart and can be
  shared by replicas pointed at one directory; memcached gives you a
  shared cache without standing up Redis. The block sits under proxy
  rather than on an origin because one store serves the whole process,
  and every origin with response_cache.enabled shares it. Leave it out
  and nothing moves: the store is still Redis when l2_cache_settings
  is configured and an in-process map otherwise. See
  docs/configuration.md.
• Encryption at rest for cached responses. An encryption block
  under proxy.response_cache_store seals cached headers and bodies
  with AES-256-GCM on the way to whichever backend you chose, so a
  cache directory or a shared memcached is no longer a plaintext copy
  of everything your upstreams returned. The key is a secret reference
  like any other in the config, so it stays out of the config file, and
  it should be 32 random bytes rather than a passphrase. previous_keys
  covers rotation: new writes seal under the active key while retired
  keys keep opening older entries. There is no plaintext fallback. A key
  that cannot be resolved stops startup instead of quietly caching in
  the clear, and an entry that fails its integrity check is evicted
  rather than served. Runnable example in
  examples/response-cache-encrypted/.
• Local classifier-based routing. A type: classifier input guardrail
  embeds a prompt with a verified local ONNX model, chooses the nearest
  configured class centroid, and publishes the label to
  ai.guardrails.labels. CEL can turn that label into
  route_to:<model>, so the gateway routes on request intent without
  sending the prompt to a classifier service. Invalid or unresolved
  classifier artifacts remain inert, and score and margin thresholds prevent
  ambiguous labels. See
  docs/ai-gateway.md and the
  runnable
  examples/ai-classifier-routing/.

Changed

• A reload that fails now really does change nothing. Reloading a
  config installed a dozen pieces of process state (log redaction,
  cardinality caps, log sinks, the AI provider catalog, the key plane,
  detection singletons, Lua sandbox limits) before it got to the two
  steps most likely to reject the config. So a config that parsed but
  failed to build left the box running the new redaction rules and the
  new AI catalog against the old pipeline, while the log line said the
  previous config was still serving. Everything that can refuse a config
  now runs first, and nothing installs until every one of those checks
  has passed. POST /admin/reload also reports what happened rather
  than only whether it worked: the response carries fully_applied and,
  when a subsystem loaded with stale state, a degraded list naming it.
  A handful of subsystems are still allowed to fail without refusing the
  reload, because a stale AI catalog beats a proxy pinned on an old
  config, but they can no longer fail silently.
• Changing proxy.secrets is refused instead of ignored. The secret
  resolver owns live connections to Vault, AWS, GCP, or Kubernetes and
  is built once at startup, so a reload never actually rebuilt it. The
  change was dropped on the floor and the first reference to a
  newly-declared backend then failed at handler construction with an
  error naming the reference rather than the cause, long after the
  reload had reported success. Such a reload is now rejected outright
  with a message saying a restart is required, the way a cluster
  identity change already was. Rotating a secret inside your vault still
  needs no restart; only changing where SBproxy looks does. See
  docs/secrets.md.
• The admin server no longer boots wide open on default credentials.
  admin / changeme exists so a first run works, but nothing stopped
  it from being the credential on an admin API bound to 0.0.0.0 with a
  private-range allowlist and no TLS, which is a published password in
  front of key minting and config writes. Validation now refuses the
  default password when the surface is reachable from another host,
  meaning bind is not a loopback address or allow_ips contains an
  entry outside loopback, and the error names which of the two tripped.
  Loopback with the defaults is untouched, since that is the local
  development path. Three related soft spots went with it: an empty
  allow_ips denied nothing at the type level (the safe loopback-only
  default lived in an if at the one call site, so the filter itself
  was fail-open), loopback was matched by comparing text so an
  IPv4-mapped peer such as ::ffff:127.0.0.1 was turned away from a
  loopback-only server, and an unparseable bind silently fell back to
  127.0.0.1 rather than failing, which made a typo in a wide bind look
  like it had worked. sbproxy plan also stops describing
  proxy.admin.** as a reload: AdminConfig is read once at startup, so
  a rotated admin password or a swapped certificate needs a restart, and
  the plan now says so. See docs/admin.md.
• Accepted configuration now has an accountable runtime owner. GraphQL
  depth, introspection, and syntax controls are enforced before upstream
  dispatch; configured CEL feature flags publish atomically across reloads;
  concurrent limits can be keyed by client, API key, header, or route; and AI
  shadow requests run through a bounded, drop-on-saturation lane that cannot
  delay the primary response. Enabling the reserved HTTP/3 listener now fails
  configuration compilation instead of logging and continuing without QUIC.
  A build-time schema audit rejects future keys that have neither a production
  reader nor an exact reviewed ConfigOnly justification.
• Workspace rate-budget behavior now has one owner. The
  rate_limit_budget policy module owns the soft, throttle, and auto-suspend
  state machine and its tests. The previously ignored per_route_rps field is
  now a config error; use rate_limiting for a per-route ceiling. The
  headers.include_ratelimit_policy switch now controls the corresponding
  response header.

Fixed

• GET /admin/drift no longer invents drift after a hot reload. The
  baseline it compares against was recorded at startup and by
  POST /admin/reload, but not by the file watcher or by SIGHUP. So
  editing the config file and letting the watcher pick it up left the
  running config correct and the baseline stale, and drift reported a
  difference that did not exist until the next admin reload or restart.
  Every path that loads a config now records the baseline.
• Saving config from the admin console no longer leaks health probes.
  Validating a config meant building the whole pipeline to see whether
  every module would construct, and that construction spawned the active
  health-check probes for any load-balancer target configured with
  health_check. The pipeline was then thrown away, but the probes were
  not: each one held the discarded pipeline alive and kept issuing real
  requests at the upstream on its own timer, forever. Every save in the
  admin console's config editor started another full set. An operator
  iterating on a config could leave a target being probed by a dozen
  generations of dead pipelines at once. Validation now constructs
  without starting anything that outlives the check, and the admin write
  path asks for a validation pipeline rather than a live one. The
  validate and plan subcommands were never affected, because they
  run outside an async runtime where the spawn was already a no-op.
• Memcached cache keys are hashed. Memcached rejects a key longer
  than 250 bytes outright, and a response-cache key carries the
  hostname, path, query, and Vary fingerprint, so any reasonably long
  URL produced a key the server refused. Those requests missed on every
  single read. Keys are now hashed before they go on the wire.
• Memcached TTLs are clamped at 30 days. The protocol reads any
  expiry above 30 days as an absolute Unix timestamp rather than an
  offset, so a longer configured TTL was stored as a moment in 1970 and
  the entry was dead the instant it was written. Relative TTLs are now
  capped at the protocol ceiling.
• The file cache no longer discards entries it was asked to keep. A
  stale-while-revalidate read deleted the entry it had just fetched, so
  the grace window it existed to serve was gone after one request.
• Concurrent file-cache writes no longer tear. Two threads writing
  the same key shared one staging file and could interleave their bytes
  into it, and the atomic rename then published the mixture. Each write
  now stages in its own file.

[1.7.0] - 2026-07-22

The admin release. The console is rebuilt around the editorial brand
system, gains live sampled charts, and, most importantly, stops hiding
data the proxy was already collecting: request sessions, custom
properties, and the gateway's own decisions now reach the operator,
and the alerting engine finally has a face. Per-origin scoping runs
across the estate so a multi-tenant gateway reports per tenant.

Added

• Sessions. Requests carrying X-Sb-Session-Id (and optionally
  X-Sb-Parent-Session-Id) are reconstructed into logical
  interactions. A session index ranks recent work by requests, tokens,
  cost, wall-clock duration, and worst status, indenting child
  sessions under their parent; a detail page reads one session's call
  chain oldest first with each call's gateway decisions, identifiers,
  AI route, tokens, cost, and properties. This is a view over the
  in-memory request ring, not durable trace storage.
• Custom properties as first-class dimensions. Bounded
  X-Sb-Property-* headers are captured, redacted per configuration,
  and carried on the request log, where they become filter and column
  choices. Properties named in an origin's properties.rollup_keys
  are promoted to durable spend dimensions, so the Spend page can
  group a window by a business dimension the caller supplied.
• Gateway decisions on every request row. The log now records what
  the gateway actually did: cache result, retry count, whether
  failover engaged and between which providers, the load-balancer
  strategy and target, and the guardrail outcome. The console reads
  them as one causal rail per row, answering whether the resilience
  configuration fired without opening a body.
• Alerts page. The alerting runtime is visible for the first time:
  rule thresholds, current reading, sample floor, and evaluation
  state; sanitized channel targets with delivery health and bounded
  errors; and recent fired, resolved, and test events. A targeted
  channel test exercises delivery without changing configuration.
  sb.yml remains authoritative and the page is read-only.
• Live metrics. The Metrics page samples the Prometheus endpoint
  and charts what happened between samples: request rate, error rate,
  latency percentiles from histogram bucket deltas, and AI token
  throughput, with numeric tiles and trend sparklines.
• Per-origin scoping. The attributed AI counters and the durable
  usage rollups carry the origin the request arrived on, and Metrics,
  Spend, Cache, and Logs can scope to one origin. Panels whose series
  have no origin dimension say so rather than showing unscoped numbers
  under a filter.
• Context-compression reporting. The compression policies report
  compressed requests, tokens and cost saved, per-lever savings,
  outcomes, and average ratio per lever.

Changed

• sbproxy apply now actually applies to the running proxy. It used
  to compile the config into its own short-lived process, swap that
  process's pipeline, print apply: reloaded config from ..., and exit
  without ever contacting the proxy. A running server picked the change
  up only if its file watcher happened to notice the file, so exit 0 was
  not evidence that the config had been accepted, or even seen. A config
  the server would have rejected still exited 0. Apply now pushes the
  config over the admin API and reports what the server did with it, so
  the exit code means something: 4 if the proxy refused the config, 7 if
  no proxy answered, 8 if it loaded but a subsystem kept stale state.
  The admin endpoint defaults to http://127.0.0.1:9090 and is
  overridable with --admin-url or SB_ADMIN_URL.

  This changes the contract. Apply previously needed no running proxy
  and always exited 0; it now needs to reach one. If you call apply in
  CI as a validation step, switch it to the new --validate-only, which
  runs every check and stops without contacting anything. That flag is
  the honest name for what the old behaviour was actually doing.

• The admin console follows the sbproxy.dev editorial system.
  Paper and ink surfaces, a persistent top bar carrying the admin
  host, a live health dot, and the cluster node count, mono
  microcopy, and square corners. Every mutation confirms or fails
  through a toast; validation detail and revision conflicts stay
  inline next to the form that caused them.

• The admin rate-limit default is 240 requests per minute per
  client IP, up from 60, with the global cap still ten times that.
  A busy console no longer trips its own limiter.

Fixed

• Cache hit and miss counts are no longer always zero. The Cache
  page read a metric name the server never emitted.
• The playground reaches locally served models. A chat against a
  served or managed deployment returned 404 because the request
  skipped the runtime's endpoint resolution and fell back to a
  localhost URL pointing at the proxy itself.
• Spend groups by a promoted property. The group-by parameter was
  read without percent-decoding, so the console's own
  property:<key> selection failed as an unknown dimension.
• Spend history reports a disabled rollup store as a hint, not as
  a failed view.
• The overview lists managed models by name with their reserved
  memory, instead of "unknown".
• An engine that dies after reaching readiness reports why. The
  health path now carries the bounded, redacted stderr tail into the
  retained error rather than logging only that the process exited.

[1.6.2] - 2026-07-21

Added

• The local llama.cpp engine pin follows your macOS version. Pinned
  builds now carry their measured minimum macOS, and the host selects the
  newest compatible one: macOS 26 gets the current build, macOS 14 and 15
  get the newest build published against the older toolchain. Previously
  the single pin targeted macOS 26 and died at dynamic-link time on
  anything older. A host older than every pin fails before download with
  the versions named; an explicit version: still wins.

Fixed

• Loading the admin UI no longer spends the admin rate budget. Static
  UI bundle assets are exempt from the per-IP admin rate limiter, so
  opening the dashboard cannot starve API polling behind 429s.
• sbproxy --version reports the real product version instead of a
  stale crate stub.
• The installer reports the binary it just installed, not whatever an
  earlier install left on PATH.

[1.6.1] - 2026-07-21

A point release fixing operational defects found immediately after the
1.6.0 cut.

Added

• Configurable admin rate limit. proxy.admin.rate_limit_per_minute
  (default 60, the previous hardcoded value; valid 1 to 100000). Automation
  and dashboards that poll admin endpoints faster than once per second per
  node can now raise the cap instead of silently reading 429s.

Fixed

• Docker images start again. The published linux binaries are built
  against glibc 2.36 so the container runtime image can execute them.
• Gateway-only clusters no longer report a standing pseudo-outage.
  Nodes without the worker role are not graded on the model plane, so a
  cluster of pure gateways shows healthy nodes in /admin/cluster/status
  and dashboards instead of a permanent degraded state. Worker health
  semantics are unchanged.
• Model engine launch failures are diagnosable. A failed engine start
  logs its bounded, credential-redacted stderr tail instead of holding it
  only in memory, and the release certification artifact carries the boot
  log and durable job records.

[1.6.0] - 2026-07-20

The cluster release. The mesh gains durable replicated state, governed
budgets that mean the same thing on every node, full
self-instrumentation, and a Kubernetes operator that forms it. Local
model serving grows a real deployment control plane and serves across
nodes, tensor-parallel GPU groups, replicas, LoRA adapters, and a
second Python engine. Two load-time behavior changes to note under
Changed: invalid retry_on entries and max_attempts above 16 now
fail the load, and sbproxy validate now fails a config that would
refuse to boot. The serve-related YAML fields remain unpinned, as in
v1.5.0.

Added

• Managed model deployments. Local serving gains a real control
  plane: a canonical model_host.deployments desired state (existing
  serve: entries lower onto it), content-addressed weight artifacts
  with resumable sha256-verified pulls and protected LRU garbage
  collection, durable deployment revisions and operation jobs, and one
  process-wide runtime manager for atomic reload, warm rolling or
  recreate rollouts with capacity preflight and rollback, admission,
  keep-alive, idle eviction, drain, health, and crash-loop retention.
  Operated through authenticated lifecycle APIs and sbproxy models pull / list / show / ps / stop / remove.
• Governed multi-node model serving. A fleet of gateways serves one
  model estate: constrained node enrollment with strict manual-PKI
  identity verification, a model directory carrying the full node
  roster with stable exclusion reasons and explicit unhealthy-node
  callouts, deterministic capability-aware placement with rolling
  handoffs, durable generation fencing, and signed deployment-authority
  state. A dedicated private HTTP/2 model plane (production mTLS,
  signed one-hop dispatch envelopes, bounded replay protection) routes
  governed requests across current-generation local and peer replicas
  with coordinated cold starts, streaming backpressure, client
  cancellation, and failover only before any client output. Model
  discovery stays OpenAI-shaped and topology-free.
• Tensor-parallel groups and N replicas per node. The fit planner
  searches tensor-parallel degrees 1, 2, 4, and 8 over homogeneous GPU
  groups and picks the smallest degree at which a candidate quant fits,
  so a model larger than the largest single card (a 70B at fp16 needs
  about 140 GB) shards across a group instead of being unservable. A
  deployment can also run several replicas of one model on disjoint
  device sets of the same node, so a dense GPU box no longer idles its
  other cards; asking for more replicas than the node can hold fails
  with a reason naming the shortfall.
• The fit planner understands model shape. Catalog entries carry a
  modality (chat, embedding, rerank, speech_to_text,
  text_to_speech, image): a non-decode model stops being charged
  autoregressive KV-cache VRAM, vLLM launches an embedder in embed
  mode, and a locally served embedder answers /v1/embeddings instead
  of a blanket 501. A mixture-of-experts model that does not fit VRAM
  whole keeps attention, shared, and dense tensors on the GPU and
  spills the fewest whole expert layers to CPU RAM (llama.cpp's
  --n-cpu-moe), which is how a 30B-A3B-class model runs on a 12 GiB
  card. The planner also predicts decode throughput per placement,
  calibrated against live A100 measurements.
• SGLang engine driver. engine: sglang serves safetensors models
  on CUDA through SGLang, acquired via uvx or a digest-pinned
  container and dispatched over the same OpenAI shape as vLLM. vLLM
  stays the default; SGLang is a one-line opt-in for prefix-heavy agent
  traffic, where the measured head-to-head favors it. The benchmark
  behind that guidance is published in
  docs/serving-engine-benchmark.md.
• Container engine provisioning is the default when a runtime is
  present. Standing up vLLM from a bare host environment needs its
  whole build toolchain and fails in a cascade on a stock GPU box, so
  when docker or podman is on PATH and the operator has not configured
  provisioning, the Python engines (vLLM, SGLang) now provision from
  curated digest-pinned container images, the exact digests validated
  on real GPU hardware. The host uvx path remains available by
  configuration.
• The embedded in-process engine moves to mistral.rs 0.9
  (PagedAttention default-on for CUDA, CUDA graphs, FlashInfer). The
  dependency stays opt-in and off by default.
• Accurate prompt token counting with a pre-flight context-fit
  gate. Locally served models count prompt tokens against the
  model's own tokenizer (prefetched alongside the weights, parsed once,
  cached) instead of a chars/4 heuristic, and an over-context prompt is
  rejected before dispatch with a clear error instead of failing
  opaquely inside the engine.
• LoRA adapters over one resident base model. A vLLM serve entry
  with lora_adapters launches the base model with each adapter
  registered by name, so a client requests a fine-tune by name over one
  resident base instead of paying for a separate engine per fine-tune.
  vLLM-only for now; other engines reject the fields with a clear
  reason.
• Per-deployment engine tuning and version pins. Canonical managed
  deployments carry the engine tuning knobs (chunked_prefill,
  including a TTFT-target mode that derives the batch size,
  tool_call_parser, swap_space_gib, cpu_offload_gib,
  extra_args), and the vLLM passthroughs now actually reach the
  engine instead of being rejected at prepare. A deployment can pin its
  own engine_version / engine_image / engine_sha256 over the
  node-wide engine policy, so two models on one node can run different
  vLLM versions (canary an upgrade on one model, hold another to its
  certified version); latest versions and unpinned images are
  rejected at config validation, and the served engine version surfaces
  in deployment status.
• Per-completion local-vs-cloud savings. A serve entry can declare
  the hosted model it displaces and that model's per-million-token
  price in a reference: block; every completion the local model
  serves is priced at the reference into a durable ledger, and
  GET /admin/model-host/value reports completions and dollars saved
  per model. Explicit config only: no reference means no savings claim,
  never a guessed cloud price.
• sbproxy update acts on stale artifacts. A plain run now
  fetches, verifies, and atomically swaps a stale engine prebuilt, and
  --self replaces the sbproxy binary from its release channel;
  --check keeps the report-only behavior. A pinned artifact, or one
  managed elsewhere (a path, brew, or apt engine), is reported and
  never mutated; the new update.{channel, auto, check_interval}
  block configures it, and auto only ever reports in the background.
• Weight-cache and artifact management. The admin plane gains a
  verified-artifact inventory (GET /admin/model-host/files),
  fail-closed artifact deletion, on-demand garbage collection, per-node
  cluster artifact totals, and a Storage view in the admin UI. A cache
  miss can reuse a discovered Ollama, LM Studio, or Hugging Face cache
  read-only instead of re-downloading weights. sbproxy models lock
  pins resolved artifacts to a lockfile, models verify-lock reports
  drift, and --locked refuses to serve anything off-lock. sbproxy models prune reclaims content-addressed weight blobs no cached
  artifact references.
• Served-model priority lanes. serve.max_concurrent_requests caps
  in-flight requests into a local engine behind a queue ordered by the
  calling key's priority lane (interactive, standard, batch),
  FIFO within a lane, so a batch flood cannot starve interactive keys;
  an interactive request that would queue spills immediately to the
  next non-served provider when one exists. The lane binds to the key
  record, never a client header.
• Governed key policy enforces end to end. One canonical
  effective-policy contract covers configured and dynamically stored
  keys, and lifecycle, tenant, model, provider, route, principal, PII,
  tool, prompt-injection, rate, budget, and admission policy all act on
  the live request path; admin mint, preview, and revisioned PATCH are
  fail-closed and the Keys UI is driven by the server's schema. Keys
  gain a working per-key tokens-per-minute cap, a priority lane,
  inject_mcp on dynamically stored keys, and PATCHable metadata, and
  immutable key and attribution dimensions propagate through usage,
  access logs, metrics, traces, and bounded audit events.
• Cluster-coherent governed-key budgets. A governed key's request,
  token, and cost limits enforce through a reserve-then-settle flow on
  the live AI path and mean the same thing on every gateway node, in
  two tiers: approximate (the default; each node disseminates settled
  usage over the mesh and admission weighs the whole fleet's spend
  within a bounded staleness window, no external database) and strict
  (atomic reserve and settle against a shared Redis backend, so two
  nodes cannot both admit a request only one has budget for). Strict
  without a Redis backend fails config validation.
• MCP guardrails. Deterministic OpenAPI-derived egress policies
  with redirect-target validation, lethal-trifecta session risk
  tracking and enforcement, opt-in dual-LLM quarantine, run-as-user
  credential minting that carries the caller's own Authorization on the
  federation wire, token compaction, and a supervised local stdio MCP
  transport.
• Traffic governance fills out, and LiteLLM import stops dropping
  keys silently. OTel, S3, and GCS usage sinks join the existing sink
  set; purpose-scoped egress, quota headroom- and reset-aware routing,
  and local fair-share pools land alongside them. config import-litellm now classifies every unknown key as mapped, warned,
  or unsupported instead of silently dropping it, and known sink
  callbacks and max_budget emit real config.
• Durable replicated cluster state. proxy.cluster.replication
  turns the mesh's single-owner in-memory state into a replicated,
  durable substrate: each key maps to a preference list of nodes on the
  existing hash ring, writes and reads choose one, quorum, or all
  consistency with read repair, every replica persists write-through to
  redb so an owner restart loses nothing, deletes replicate as
  tombstones collected only after every replica confirms them and a
  grace period passes, and fleet admin runs over topology-safe bounded
  pagination.
• The mesh reports on itself. Gossip probe round-trip time and
  indirect-probe retries, enrollment outcomes, transport RPC errors by
  phase and durations by operation, owner-routing outcomes, and a live
  peer-count gauge; every mesh metric now sits in the executable
  stability catalogue under the sanctioned mesh_ prefix.
• The Kubernetes operator forms the mesh. With
  spec.clustering.enabled, the operator reconciles a StatefulSet
  (stable per-pod identity, one-peer-at-a-time rolling restarts), a
  headless Service publishing the gossip and transport ports, a
  shared-key Secret, and a rendered proxy.cluster block with
  full-ordinal seed lists and per-pod node identity, built through the
  typed config so invented fields are impossible. Includes the two
  fixes live validation on kind surfaced: the operator now installs its
  TLS crypto provider (it previously panicked on its first handshake
  and reconciled nothing), and DNS-name gossip seeds resolve before the
  probe path (they were silently skipped, leaving every pod a one-node
  mesh).
• Compression session state can live on the mesh. Stateful
  compression's state.backend: mesh now runs on the durable
  replicated substrate: conditional versioned session commits with
  deterministic cross-node conflict resolution, tombstoned deletes that
  survive partition and heal, and the same admin list, inspect, and
  purge over fleet pagination. Redis remains the default and
  recommended backend.
• Measured 3-node cluster benchmark. docs/performance.md gains a
  clustered section from a real 3-node GCP mesh run: forming the mesh
  costs within noise on a single node (43,129 vs 43,958 requests per
  second), three nodes sustain 119,178 requests per second aggregate
  with zero errors, governed spend becomes visible on a peer in 15 to
  20 seconds, and survivors run at 100% success through a mid-run node
  kill, with rejoin about 10 seconds after restart.
• Request-selectable AI context compression. Declare named route-local
  profiles and explicit input budgets, then select them through
  X-Compression, governed virtual keys, or CEL with deterministic precedence
  and safe invalid-selector behavior. Phase 1 adds rag_select,
  compact_serialization, and position_reorder for explicit line-delimited
  retrieval blocks. The levers use deterministic ranking, reversible
  sbproxy_table_v1 encoding, closed fail-open outcomes, and semantic-cache
  bypass before the final window_fit bound. Stateful summaries use Redis as
  the canonical session store while request workers remain stateless;
  authenticated Admin APIs list, inspect metadata for, and purge that state.
  Per-lever results now appear in bounded metrics and one content-free summary
  event per executed pipeline; reducing levers also feed bounded value metrics,
  dashboards, and the model-host value report. Live request-path acceptance and
  five independently authored structural smoke reports cover the production
  stateless pipeline.
• MCP tool rollout plane. Publish several versions of one tool at once
  and roll out breaking changes without breaking callers: a rollout: block
  under the mcp action's tool_versioning declares versions, where each
  routes, and who gets which. Resolution walks a ladder (per-call _meta
  requirement, per-session requirements declared at initialize, operator
  pins on the authenticated principal, search_v1-style catalogue aliases,
  then the default), all as semver ranges. Old versions can route to the
  upstream that still serves them or run JavaScript request/response
  adapters against the new one, carry a sunset date that warns or blocks
  past it, and every versioned call lands on
  sbproxy_mcp_tool_version_calls_total{tool, version, via, deprecated} so
  migration is observable. tools/list advertises the consumer's resolved
  version per tool with the available versions and sunset in _meta;
  results carry the version that served them. The tool_versioning.lockfile
  is now optional so the rollout plane works without the version-bump gate.
  See docs/tool-versioning.md and examples/mcp-tool-rollout/.
• Model deployment management in the built-in admin UI. Operators can
  browse catalog evidence, add or edit the complete desired deployment map,
  resolve revision conflicts explicitly, and run Load, Stop, or Reset. The
  same UI respects file-managed, admin-managed, and signed cluster-authority
  ownership; file-managed and verifier nodes stay read-only.
• Cluster operations and unhealthy-node alerts in the admin UI. The
  Cluster page now shows every node, placement and rollout state, deployment
  authority, and prominent links to unhealthy roster entries. Health remains
  visible when metrics fail, and the last cluster snapshot stays on screen
  with a stale warning after a refresh error.
• Streaming responses now run every built-in output guardrail, with
  verdicts matching the buffered path. A per-stream session matches the
  substring guardrails (injection, toxicity, jailbreak, content safety)
  over a cumulative window of decoded deltas, so a pattern split across
  chunk boundaries still blocks, and word-boundary rules never
  false-block on split words.
• Streamed tool calls are assembled per call and judged by the
  agent-alignment guardrail as each call completes. Block mode holds
  tool-call frames until their call is judged while text keeps flowing;
  flag mode logs and counts without touching the stream.
• Per-entry stream_policy (chunk, close, off) on output
  guardrails, plus new metrics:
  sbproxy_ai_stream_guardrail_violations_total,
  sbproxy_ai_stream_guardrail_skipped_total, and
  sbproxy_ai_stream_guardrail_decode_fallback_total.
• A TPOT histogram (sbproxy_ai_inter_token_latency_seconds)
  completing the TTFT / TPOT / throughput serving triple, OpenMetrics
  exemplars on the AI latency histograms so a spike links to its
  trace, and the OTel GenAI metric instruments
  (gen_ai.client.operation.duration, gen_ai.client.token.usage)
  mirrored over OTLP so GenAI-aware backends chart without relabeling.
• headers: on the telemetry block for authenticated OTLP export to
  hosted backends. Values accept secret references that resolve at
  boot and fail loud, apply to traces, mirrored metrics, and the
  OTLP log sink, and are masked in config printouts. Every signal now
  carries detected resource attributes (host, process, Kubernetes
  downward API, OTEL_RESOURCE_ATTRIBUTES), with explicit
  resource_attrs winning conflicts.
• Durable windowed spend rollups: hour and day usage buckets that
  survive restarts, a windowed /api/usage/spend
  (window/group_by/from/to), and a spend-history section on
  the admin Spend page. On by default with bounded retention; rows
  carry no prompt content and no raw key material.
• Access log AI columns: cost_usd_micros (integer micro-USD) and
  guardrail_category / guardrail_action on every guardrail
  intervention, mirrored onto the request envelope and the admin
  request ring; /api/requests accepts guardrail_action and
  guardrail_category filters.
• Slack and PagerDuty alert delivery channels as formatters over the
  existing webhook transport (PagerDuty trigger/resolve keyed on a
  stable per-rule deduplication key), plus Prometheus alert examples
  for AI budget utilization, provider error burn, and spend velocity.
• MCP execute_tool spans following the OTel GenAI agent
  conventions, parented into the caller's trace so agent request,
  tool dispatch, and LLM calls render as one tree; the AI request
  span emits tool-call span events (ids and names always, arguments
  only under trace_content).
• Admin views: AI performance (TTFT / TPOT / throughput and provider
  health with failovers, cascade tiers, and router decisions), Spend
  (live attributed cost, token, and request breakdowns by model,
  provider, key, team, and project),
  Guardrails (blocks by category and wasted tokens / spend by kind),
  live tail on the Logs view with full-record row expansion and
  operator-configurable trace deep links
  (admin.trace_url_template).
• Executable capability registry. SBproxy's claims about itself are
  now checkable code: every capability claim carries a support level,
  nothing may be called stable unless a test proves a production caller
  consumes it, and config-only is the honest, permitted name for a
  surface that parses and does nothing. Build guards fail on a
  published metric no code writes and on a tenant-relevant metric
  family missing its tenant labels, and the shipped Prometheus alert
  rules are validated with promtool in CI. This machinery surfaced the
  availability SLO that read 100 percent forever and the
  never-incremented metric families fixed in this release.
• Getting started and framework integrations. A dedicated
  docs/getting-started.md, install and quick start grouped together
  in the README, and five framework one-pagers (LangChain, Vercel AI
  SDK, Pydantic AI, Mastra, n8n) whose snippets were all executed
  against a running gateway before landing. The README lede now leads
  with what the gateway does today.

Changed

• sbproxy validate runs the boot path. validate, plan, and
  apply now construct the same compiled pipeline the server and
  reload paths construct, so a config that would refuse to boot fails
  validation instead of validating clean (measured before the change:
  five published examples validated but refused to boot). Custom YAML
  tags are rejected at compile: serde_yaml strips unknown tags, so a
  password: !env ADMIN_PASSWORD silently became the literal string
  ADMIN_PASSWORD; the error now points at ${VAR} interpolation,
  and ${VAR:-default} fallbacks work.
• Status-code upstream retries moved onto a dedicated decision
  hook. The retry decision fires on the pinned Pingora fork's new
  upstream-response hook, once per upstream response and before any
  bytes reach the client, replacing the response-filter workaround;
  connect-time and status retries now share one attempt counter and
  cap. Load-time validation is a behavior change: a retry_on entry
  must be connect_error, timeout, or a status in 100..=599 (junk
  entries used to deserialize and silently never match; they now fail
  the load naming the entry), and max_attempts above 16 is rejected.
  Retries land on
  sbproxy_upstream_status_retries_total{origin, status}.

Fixed

• retry_on: timeout is honored, in both upstream phases. The
  token was accepted and documented but nothing consulted it. A
  connect-phase timeout now retries under either timeout or
  connect_error, and an established-connection upstream read or write
  timeout retries when the policy allows it, sharing the same attempt
  cap, and only when the request is replayable and no response bytes
  have reached the client. The fork's retry loop also gains a backstop
  refusing any retry after response bytes were sent, regardless of what
  marked the error retryable.
• Redis L2 connections keep their TLS, AUTH, and database
  semantics. redis:// and rediss:// URLs preserve ACL and
  percent-encoded credentials, IPv6 hosts, the selected database,
  private CAs, and mutual TLS uniformly across the general L2 store,
  compression state, and admin paths, compiled once per config
  generation into an immutable connection snapshot. The blocking
  plaintext RESP path is replaced with a real client, and connection,
  TLS, authentication, and command failures classify without leaking
  endpoints or credentials.
• Cross-node mesh RPCs no longer stall about 40 ms on Linux. The
  transport wrote a frame's length prefix and body as two separate
  writes and never set TCP_NODELAY on accepted sockets, so Nagle plus
  the delayed-ACK timer held every response leg. Frames now leave as
  one write and the server sets nodelay on accept; a small-frame
  replica fetch drops from about 41 ms back to sub-millisecond.
• The MCP gateway speaks the spec's camelCase on the wire.
  initialize results, tool results, and tool annotations serialized
  as snake_case (protocol_version, is_error, read_only_hint),
  which the official TypeScript SDK's schema rejects outright, so a
  strict client could not connect at all and tolerant clients silently
  dropped tool error flags. Serialization is now camelCase; snake_case
  still parses so results from older nodes survive mixed-version
  rollouts.
• Raw hf: references serve through the live path. The production
  runtime manager only resolved fully pinned catalog artifacts, so a
  raw hf:Org/Repo reference in a serve: block failed reconciliation
  and, in practice, no open-weight model could be served on a GPU from
  the gateway. Raw references now resolve, pull, and serve end to end,
  validated on real multi-GPU NVIDIA hardware across vLLM, SGLang,
  embeddings, and tensor-parallel launches.
• SGLang serving hardening. The launcher passes a runtime-owned
  memory fraction so SGLang no longer OOMs at launch; liveness probes
  hit a non-generating endpoint instead of one that generated tokens
  (and returned 503 under load); one transient health-probe miss no
  longer kills a ready engine; and the probed SGLang version is
  recorded on the provisioned engine.
• Self-host admin edges. Attributed AI token and cost metrics now
  populate /api/usage/spend for locally served providers; direct AI
  responses no longer log status 0 when a real response was written; an
  upstream-TLS native-certificate failure on macOS is an actionable
  startup error instead of a panic; and the admin Keys UI submits the
  backend's full key-policy shape.
• The alerting: block alerts, and declared metrics record. The
  alerting config parsed and silently discarded its settings; it now
  drives a live dispatcher (delivering through the Slack and PagerDuty
  channels above). The response-cache hit/miss, circuit-breaker
  transition, and guardrail-block families were declared and scraped
  but always zero; they are now written by the live request path, and
  the metric drift guard follows aliased writers it was blind to.
• The release provenance push no longer clobbers the SBOM
  attestation. The provenance step replaced the image's attestation
  tag wholesale after the CycloneDX attest step, so
  cosign verify-attestation --type cyclonedx failed on every
  published image; the jobs are reordered so the SBOM attestation
  appends last, and the offline verification recipe in
  SUPPLY-CHAIN.md now works with current cosign.

Removed

• The unattributed AI metric families sbproxy_ai_requests_total,
  sbproxy_ai_tokens_total, sbproxy_ai_cost_dollars_total, and the
  per-virtual-key trio. They were registered but never written on the
  live path, and counter series register lazily, so no released
  binary ever exposed a sample under these names. Consumers read the
  attributed families; details in docs/metrics-stability.md.
• The Go-era secret:<name> colon form. It resolved through a
  logical-name map with an environment fallback and was superseded by
  the provider-URI secret://<backend>/<name> schemes; a stale
  reference now fails config load with a migration pointer instead of
  resolving through a side channel. proxy.secrets.map still parses
  for schema-v1 compatibility and warns at boot that it has no effect.
• Dead mesh scaffolding and write-only key counters. The
  unreferenced leader-election, health-monitor, consistency, and
  membership-protocol modules are gone (live membership is the gossip
  loop), along with a legacy wire variant only tests constructed and
  the per-request mesh key counters that were incremented on every AI
  request but never read anywhere. Governed-key budget enforcement is
  unaffected, and the AI hot path now does no counter work at all.
• Two dead metric families: sbproxy_dedup_cache_size (registered,
  never written, no readers) and the hostname-keyed
  sbproxy_cache_hits_total duplicate; the overview dashboard reads
  the now-live sbproxy_cache_results_total instead.

[1.5.0] - 2026-07-08

Model serving lands: run open models on your own GPU behind the same
gateway that fronts the 66 hosted providers, plus the engine-acquisition
and self-host work queued since v1.4.0. No promises about backward
compatibility for any of the new YAML fields below until a later version
pins them.

Changed

• Duration strings parse consistently everywhere. The ms/s/m/h/d
  units, compound forms like 1h30m, decimals like 1.5h, and a bare
  number (seconds) are now accepted by every duration field, instead of
  each config block supporting a different subset (so a value like 1h
  that parsed in one block and errored in another now works in both). This
  only widens what is accepted; no previously valid value changes meaning.
• Unresolvable upstream hosts always fail closed. The upstream SSRF
  guard no longer blocks the request worker on a per-request DNS resolve
  (it resolves asynchronously now), and as part of that an upstream host
  that fails to resolve is uniformly rejected, closing an edge where an
  origin with a private-CIDR allowlist could previously fail open.

Removed

• Two rate-limit config options that parsed but never enforced anything
  are gone. A virtual key's max_tokens_per_minute (and the credential
  policy's tpm) and an origin's per-origin rate_limits: block both
  compiled and round-tripped but were never read at request time, so an
  operator who set them believed they were capped when they were not.
  They are removed rather than wired. Existing configs that still set
  these keys keep loading (the keys are ignored). The live limits are
  unaffected: the top-level workspace rate_limits: budget, and the AI
  gateway's model_rate_limits / per-surface limits, all still enforce.
• Two build-only feature flags that nothing enabled were removed
  (sbproxy-platform/postgres-store and an unused sbproxy-modules
  rate-limit feature), along with roughly 4,300 lines of verified
  zero-caller internal code. No shipped configuration or public API
  changes; the redb/SQLite storage stack is unaffected.

Added

• vLLM, provisioned with uvx. vLLM is a Python package, not a
  single-binary release, so sbproxy now acquires it by fetching uv
  (Astral's single-binary package manager) and running the engine through
  uv tool run (uvx): a cached, ephemeral environment that uv sets up
  on first use, bringing its own Python if the host lacks one. The default
  wheel is CUDA-enabled, so a safetensors model offloads to an NVIDIA GPU
  on a box that carries only the driver. Opt in with
  engines.vllm.acquire.source: uvx; sbproxy run <model> sets it for
  you. sbproxy doctor reports it as the recommended vLLM path.
• sbproxy update: is any of it out of date. A dry-run freshness
  report: sbproxy update checks the inference engine release feed (the
  pinned llama.cpp prebuilt vs the latest) and the cached models (flagging
  any that track a moving ref like main and could be behind upstream);
  --self also checks the sbproxy binary against its release channel.
  --json for tooling. Reports only, nothing is mutated; a pinned artifact
  is never swapped without an explicit run.
• sbproxy config print: see the effective config, with secrets
  masked. Prints the config after built-in defaults + the file +
  ${ENV} interpolation, so it is obvious what a box will actually do.
  Inline secret values (an api_key, client_secret, token, ...) are
  masked; secret references (vault://, ${ENV}, file:, ...) are
  shown, since they are pointers, not the secret. --json for tooling,
  YAML by default.
• sbproxy models list / show: discover what this host can run.
  sbproxy models (or models list) prints one row per catalog model
  with a real per-GPU fit verdict (reusing the same probe doctor uses),
  the resolved engine, params, and cache status (cached / not-pulled).
  sbproxy models show <id> prints the full entry: HF repo, source,
  revision, sha256 digests, engine, pull policy, and quants. --json on
  both for scripts and the admin UI; --catalog-file points at an
  operator manifest. Resident / serving state needs a running gateway and
  is not shown by this offline view.
• sbproxy run <model>: serve a model in one command, no YAML.
  sbproxy run qwen3-14b (or sbproxy run hf:Org/Repo:Q4_K_M --name coder) synthesizes a minimal serving config, checks the model can run
  on this host (the same detection sbproxy doctor uses, so a model with
  no viable engine fails now with a remediation instead of a later 502),
  and boots the gateway with an OpenAI-compatible endpoint on loopback at
  http://127.0.0.1:<port> (both the IP and localhost route). The
  engine and weights are acquired on the first request. Flags override
  the port, engine, acceleration, and cache directory; --dry-run prints
  the resolution and the synthesized config without serving.
• Model pull honors manifest pins and works for safetensors/vLLM on a
  fresh box. A model's weight pull now uses the manifest revision
  (was hard-coded main) and verifies the per-file sha256 when one is
  pinned, so a digest mismatch fails the pull loudly instead of serving
  bad weights. And a safetensors model served via vLLM now pre-fetches
  its config.json on first use, so it admits on a box that has never
  pulled it (previously it failed with "no model metadata").
• sbproxy acquires the inference engine, not just finds it on PATH.
  A serve: block can now carry a per-engine engines.<engine>.acquire:
  block: for llama.cpp, source: release (the default) fetches a pinned
  ggml-org prebuilt for the host platform and acceleration
  (accel: auto|cuda|vulkan|metal|cpu; on Linux a GPU build means the
  Vulkan asset, since there is no upstream CUDA Linux prebuilt),
  sha256-verified when a digest is pinned, while source: path points at
  an operator-installed binary for an air-gapped box. A host with no
  engine now serves a GGUF model instead of failing at the first request,
  and a bad acquisition (a path source with no path, a latest
  version) is rejected at config load, not at runtime. Engine identity
  stays the allowlisted set (vllm, llama_cpp, embedded); only how
  the binary is obtained is configurable. The gateway also detects a
  container runtime now, so engine: auto can resolve to vLLM's
  container path for safetensors weights.
• The released binary is GPU-aware out of the box. The gpu-nvidia
  (NVML GPU discovery with an nvidia-smi fallback) and model-weights
  (Hugging Face weight download) features moved into the sbproxy
  binary's default feature set, so one downloaded artifact adapts to its
  host: the NVIDIA driver library is loaded at runtime when present,
  never linked, and a GPU-free host still runs the same binary (a
  serve: provider rejects admission cleanly there). Building with
  --features gpu-nvidia,model-weights is no longer needed for local
  model serving. Library consumers of the workspace crates still opt in
  per crate.
• sbproxy doctor is the self-host front door. The subcommand now
  reports the full picture of what the binary can do on this host and
  how to make it serve: OS and arch, CPU and RAM, free disk in the cache
  directory, the GPU (or CPU / unified-memory budget) the serve:
  admission path sees, NVIDIA driver and CUDA / Metal / ROCm, container
  runtimes and daemon liveness, package managers, Python and uv, and
  Hugging Face reach plus whether HF_TOKEN is set. For each engine
  (llama.cpp, vLLM, embedded) it lists what is installed (with version)
  and which acquisition sources are viable here, each with a reason.
  Pass a config file (sbproxy doctor sb.yml) and it adds, per serve:
  model, what engine: auto resolves to and a coarse fit preview, and
  exits non-zero when a configured model has no viable engine.
  --format json emits a stable machine-readable report; collection is
  read-only.
• Local model serving runs on Macs and CPU boxes, not just NVIDIA.
  The fit planner used to see zero devices on anything but an NVIDIA GPU,
  so a serve: block on a Mac or a GPU-less server rejected every model.
  The GPU probe is now layered: NVIDIA discrete GPUs first, then Apple
  Silicon unified memory (reported as the working-set budget), then a CPU
  budget sized to a fraction of system RAM. A small GGUF is admitted
  against unified memory or RAM and served by llama.cpp or the embedded
  engine; FP8 and other datacenter quants are still refused on hardware
  that lacks the kernels. Set SBPROXY_CPU_MEMORY_FRACTION=0 to opt back
  into rejecting admission on a GPU-less host. The weight cache defaults to
  ~/.cache/sbproxy/models for a non-root run (and the service path
  /var/lib/sbproxy/models when running as root), so serving works out of
  the box without configuring cache_dir.
• Serve-preflight warnings at config load. A config that declares
  serve: on a host with no visible GPU, or with a serve entry whose
  engine has no binary and no container runtime, now logs a warning at
  startup and on every hot reload naming the model, the resolved
  engine, and the blocker, instead of degrading silently until the
  first request fails over.

Changed

• A forward rule whose header matcher names an invalid HTTP header now
  fails at config load. The header: matcher on a forward_rules:
  entry precompiles its name at load time; a name that is not a valid
  header (for example one containing spaces) previously loaded and then
  silently never matched, and now reports a clear error at load and on
  reload. Valid configurations are unaffected.

Fixed

• Revoking a key now blocks OIDC/JWT identities mapped to it. With
  key_management.oidc_claim_map configured, a verified token whose mapped
  claim named a revoked, blocked, or expired record was silently downgraded to
  an ungoverned request (no per-key policy) instead of being denied. The
  mapped-claim path now mirrors the bearer path: an inactive record denies with
  403, a claim naming a missing record denies with 401, and a store outage
  fails closed unless failure_mode_allow is set. Tokens that carry no mapped
  claim are unaffected.

• Error responses now emit valid JSON when the message contains a quote or
  backslash. The shared send_error helper and the ledger, policy, and
  storage error paths built the {"error": "..."} body by string
  interpolation, so a message carrying a client-supplied value (for example a
  rejected AI model name) could break the JSON envelope or inject a sibling
  field. Every error body is now serialized, so the message is always escaped.

• JSON threat protection now scans the whole request body. The depth, key,
  and size checks read only the first body chunk, so a JSON payload whose
  oversized structure began past the first chunk could slip past the scan while
  the full body still reached the upstream. The scan now accumulates the
  complete body, bounded by max_total_size (or a hard ceiling when unset),
  before validating it.

• The in-memory idempotency cache and the native SSE reassembly buffer are
  now bounded. The single-instance idempotency store grew without limit under
  unique keys and is now a capacity-bounded LRU. The native streaming framer
  buffered upstream bytes until a frame boundary and now caps the reassembly
  buffer, so an upstream that never closes a frame cannot grow it without limit.

[1.4.0] - 2026-06-27

Fourth minor release on the Rust v1.x line. Hardening and reach for the
AI gateway and the clustering mesh: mutually-authenticated TLS on the
peer transport, external HTTP guardrail providers on the request and the
response, native Langfuse and Datadog usage sinks, and per-server
namespace control for MCP federation. One correctness fix promotes
budget windows from parsed-but-ignored to enforced. No config-breaking
changes; existing sb.yml files compile unchanged, and every new field
is default-off.

Added

• Mesh peer mTLS. The mesh peer transport can run over
  mutually-authenticated TLS: set key_management.cache.mesh.peer_tls with
  cert_file, key_file, and ca_file (plus an optional server_name,
  default sbproxy-mesh). Every inbound connection must present a CA-signed
  client certificate and every outbound connection presents this node's
  certificate, both verified against the CA, so an untrusted peer cannot join
  the cache fabric. Plaintext when unset.

• Per-server namespace mode for MCP federation. A federated upstream can
  set namespace: always to expose every tool as <prefix>.<tool> and every
  resource as <prefix>/<uri>, where the prefix is the server's prefix (or
  a name derived from its origin). The default, on_collision, keeps bare
  names and only qualifies one when it clashes with an earlier server.

• External HTTP guardrail providers. An AI origin's guardrails.external
  list runs external guardrail services alongside the built-in checks.
  Input-mode entries (pre_call / during_call) inspect the request before
  dispatch; output-mode entries (post_call / during_call) inspect the
  non-streaming response before it is cached or sent. Either blocks on a
  not-allowed verdict (logging_only records only), and a transport or parse
  error honors each entry's fail_open flag. Provider presets shape the
  request and response for Presidio (/analyze with a findings array) and a
  generic {"input"} shape that fits Lakera, Aporia, and custom endpoints,
  with an optional API key on a configurable auth header. Streaming-response
  and AWS Bedrock (SigV4) guardrails are not yet wired.

• Native Langfuse and Datadog usage sinks. Alongside the JSONL-file,
  webhook, and ledger sinks, usage_sinks now accepts type: langfuse
  (host plus public/secret key; posts a generation observation to
  /api/public/ingestion) and type: datadog (api_key plus optional
  site / service; posts to the logs-intake API). Both are
  fire-and-forget and never fail the request they record. Object-store
  (S3/GCS) and OTel usage sinks are not yet included.

Fixed

• Budget windows now reset per period. A budget limit with a period
  (daily, monthly, or a duration like 30d) was parsed but never enforced
  as a rolling window, so spend accumulated forever and a daily cap behaved
  like a lifetime cap. Each limit now accrues against its own per-period
  bucket, so a daily cap clears at the next day and a daily and a monthly cap
  on the same scope are tracked independently. Cumulative limits (no period,
  or total / lifetime) are unchanged.

• MCP federation now advertises the disambiguated name on a collision.
  When two upstreams exported the same tool name, the gateway kept the
  prefixed name only as an internal registry key while still advertising the
  bare name, so the second tool was unreachable and tools/list showed a
  duplicate. The disambiguated name (<server>.<tool>, or <server>/<uri>
  for resources) is now the advertised, routable name; resource reads still
  forward the original upstream URI.

[1.3.1] - 2026-06-25

Patch release. Fixes TLS, which was broken on startup in v1.2.0 and v1.3.0.

Fixed

• TLS no longer panics on startup. The OCSP-staple and ACME-renewal
  background tasks were spawned before the proxy runtime existed, so any HTTPS
  listener with a manual cert (tls_cert_file / tls_key_file) or enabled ACME
  crashed the process on boot ("there is no reactor running"). The tasks now
  spawn on a runtime that is always available.
• HTTP/2 is now negotiated over TLS. No TLS listener advertised h2 in ALPN,
  so every HTTPS connection fell back to HTTP/1.1. The manual-cert, ACME, and
  mTLS listeners now enable h2; clients that do not offer it still get HTTP/1.1.

[1.3.0] - 2026-06-25

Third minor release on the Rust v1.x line. Two headlines: dynamic key
management with an open-source mesh for clustering, and a wave of
state-of-the-art AI-gateway capabilities. No config-breaking changes;
existing sb.yml files compile unchanged, and every new field is
default-off.

Added

• Dynamic key management. Inbound virtual keys are a live, governed
  resource: mint, list, rotate, and revoke them at runtime through an admin
  API under /admin/keys, with no reload. Keys are hashed at rest with
  HMAC-SHA256 and a server pepper, and a revoke takes effect on the next
  request. Upstream provider credentials are encrypted at rest with an
  AES-256-GCM envelope or held as a vault reference. Per-key policy travels
  with the key: model and provider allow/deny, rate and token limits, token
  and USD budgets, expiry, required PII redaction, principal selectors, a
  pinned model, injected tools, and an injection-scan bypass. Pluggable
  stores: embedded (redb), Redis, or a secrets manager. OIDC and JWT claims
  can map to a key. New key_management: config block. (#542, #543)
• Open-source mesh clustering. The mesh layer (SWIM gossip, a
  consistent-hash distributed cache) is now Apache-2.0 in this repository.
  Setting cache.tier: mesh keeps the key plane coherent across a replica
  fleet: a key minted on one replica is usable on any, and a revocation on
  one denies on the rest, with no external control plane in the path. Per-key
  spend and rate counters remain node-local; cluster-wide budget enforcement
  uses a shared backend. (#542)
• State-of-the-art AI-gateway differentiation. A verifiable, hash-chained
  and optionally Ed25519-signed usage ledger; a single sandboxed CEL policy
  plane over guardrails, budgets, routing, and principal; a guardrail mesh
  that fuses verdicts on a quorum with a verdict cache; outcome-aware routing
  by realized cost-per-success; predictive budgets that warn, then downgrade,
  then block; and LLM-aware resilience: per-error retry, context-window
  compression, hedged and raced dispatch, and content-policy fallback to a
  more permissive provider. (#538, #539, #540, #541)
• LiteLLM drop-in. A config import-litellm translator, model groups, and
  usage-sink plus budget foundations for moving a LiteLLM proxy over. (#537)
• Model-based routing with a failover metric and a refreshed model-id
  catalog. (#536)
• VHS cassettes for the AI gateway and the example configs. (#534)

Changed

• The mesh wire encoding moved off the unmaintained bincode crate to
  postcard.
• The README and docs now lead with the two-way framing: SBproxy governs the
  AI you call and the AI that calls you.

[1.2.0] - 2026-06-24

Second minor release on the Rust v1.x line. Headline: local ONNX
inference for the embedding semantic cache and the prompt-injection
classifier, a standalone OpenAI-compatible embedding source, a
best-of-class OpenTelemetry story for the AI gateway, and the move to
Apache 2.0. No config-breaking changes; existing sb.yml files compile
unchanged.

Added

• Local ONNX inference for the semantic cache. The embedding
  semantic cache can vectorize prompts on-box, with no per-call API cost
  and no prompt egress. source: sidecar runs the embedder in the
  supervised classifier sidecar; source: inprocess loads an ONNX model
  (all-MiniLM-L6-v2 by default) into the proxy behind an explicit opt-in
  and a max_model_bytes guard. Prompt-injection v2 gains first-class
  ONNX detectors (detector: sidecar, detector: inprocess) next to the
  zero-dependency heuristic default. See
  docs/local-inference.md.
• OpenAI-compatible embedding source (source: openai). Vectorize
  prompts through any standalone OpenAI-compatible /v1/embeddings
  endpoint, decoupled from the origin's chat providers: point it at
  another sbproxy that fronts an embedding model, at OpenRouter, or at a
  hosted provider. Auth defaults to Authorization: Bearer; set
  auth_header / auth_prefix for api-key / x-api-key endpoints, or
  carry the credential in arbitrary extra headers.
• Best-of-class OpenTelemetry for the AI gateway. AI spans now carry
  derived USD cost (and a first-class cost metric), map failures
  (guardrail, provider 429/5xx, content filter) to span status ERROR with
  an error.type, and emit capture-gated, redacted prompt and completion
  content as OpenInference / OTel gen_ai span events. A pinned GenAI
  semantic-convention conformance test guards against attribute drift.
  The reference stack adds Arize Phoenix and Langfuse with provisioned
  dashboards, plus cost-aware (ParentBased + TraceIdRatio) trace
  sampling. docs/observability.md gains a
  verified backend matrix.
• Per-credential, multi-tenant, multi-model AI value tracking in the
  reporting surface.
• GCP Secret Manager vault backend (gcpsm://), joining HashiCorp
  Vault (vault://) and AWS Secrets Manager (awssm://).
• Configurable retry on upstream response statuses.
• Web Bot Auth key IDs now feed the agent identity proof.

Changed

• SBproxy OSS is now licensed Apache 2.0. The previous Business
  Source License field-of-use restriction is dropped; the project is free
  for any use, including production and commercial, with no field-of-use
  limit.
• Vault references moved to per-provider schemes. The scheme now
  selects the backend (vault:// HashiCorp, awssm:// AWS, gcpsm://
  GCP) rather than a vault://<alias> umbrella form. The legacy form
  still resolves during a deprecation window and logs a one-time warning.
• HTTP/3 (QUIC) is temporarily disabled until native support lands in
  the underlying proxy engine. Existing config still parses, but no
  HTTP/3 listener starts.
• The admin playground chat route is gated by default.

Fixed

• Credential selectors are enforced consistently across request paths,
  and the AI preference script context is exposed to request scripts.

[1.1.0] - 2026-06-06

First minor release on the Rust v1.x line. This release carries
breaking changes to the MCP tool-access policy (now closed-by-default
and principal-aware); read the Breaking section and
docs/migration-mcp-rbac.md before upgrading. It also ships 66 native
AI providers behind one OpenAI-compatible API.

Breaking

• MCP default-deny: ToolAccessPolicy flipped from
  open-by-default to closed-by-default. An unknown caller (no
  matching ACL rule) is denied every tool. An empty allowed: []
  list under an ACL rule means "deny all", not "allow all".
  Operators who want the legacy behaviour add default_allow: true
  on the origin's MCP action. The legacy key_permissions: { key: [tools] }
  shape is gone; rewrite to the principal-aware tool_access[]
  selector list. See docs/migration-mcp-rbac.md.

• MCP principal-aware ACL: ToolAccessPolicy now
  carries tool_access[] rules with principals[] selectors
  (virtual_key, sub, team, project, user, role,
  tenant_id) plus an allowed[] tool list. The legacy
  key_permissions: HashMap<String, Vec<String>> map is removed
  along with ToolAccessPolicy::is_tool_allowed(key, tool); the new
  surface is policy.check(&principal, tool) -> ToolAccessDecision
  and policy.filter_tools(&principal, &tools). tools/list now
  filters by RBAC against the inbound principal (the legacy schema
  leaked tool names through tools/list even when the gate would
  deny the matching tools/call). A new tool_quotas[] table
  enforces per-tool sliding-window quotas keyed on
  (tenant_id, principal_id, tool_name). See
  docs/migration-mcp-rbac.md.

Added

• 66 native AI providers behind one OpenAI-compatible API. The
  embedded ai_providers.yml registry ships 66 providers (up from 43),
  adding Hugging Face Inference, GitHub Models, Vercel AI Gateway,
  Nebius, Baseten, Lambda, FriendliAI, Scaleway, Nscale, DigitalOcean
  Gradient, OVHcloud, Inference.net, kluster.ai, OpenPipe, Writer,
  Upstage, Aleph Alpha, MiniMax, Volcengine Ark (Doubao), Tencent
  Hunyuan, Baidu Qianfan (ERNIE), StepFun, and Mixedbread. The catalog
  is plain YAML and operator-extensible at runtime via
  proxy.ai_providers_file; the model field passes through to the
  upstream, so any model a provider serves is reachable without
  per-model config. The "200+ models" reach is native (bring your own
  keys); OpenRouter is one provider among the 66, not a dependency. See
  docs/providers.md#extending-the-provider-catalog.

• Session ledger from live MCP traffic. A new top-level
  session_ledger: block makes SBproxy emit the canonical
  session-ledger-v1 run record (shared with mcptest) from its
  tools/call path: one header per session, then one tool_call
  record per call carrying session_id, a zero-based hop_index, the
  bare tool name and server, redacted params / result, an error
  flag, and the round-trip duration_ms. sink: logging (default)
  emits each record as a session_ledger tracing line; sink: file
  with a path: appends NDJSON. Off unless enabled: true; when off
  the tool-call path pays only a single atomic load. Payloads are
  redacted with the same secret-stripping the access log uses. See
  docs/mcp.md and examples/mcp-federation/sb.yml.

• Structured-log schema v2 (SCHEMA_VERSION = "2"). Three changes
  land together so downstream tooling can read them in one swing:
  optional session_id and user_id top-level fields parallel the
  RequestEvent envelope (cross-surface JOIN no longer relies on
  request_id alone); the field-key redaction marker is normalised
  to [REDACTED:<NAME>] everywhere (was <redacted:name> in v1) so
  the schema-v1 layer matches the existing PII-rule replacement
  shape; the schema bump is additive on the field set (a v1 reader
  parsing a v2 line keeps working because every new field is
  skip_serializing_if = Option::is_none). Marker normalisation is
  a string change; downstream tooling that greps for the old
  <redacted:...> form must update.

• Phase-timing breakdown on the access log + new
  sbproxy_phase_duration_seconds Prometheus histogram. The
  access log carried latency_ms end to end and that was it; an
  operator looking at a slow request could not tell from the log
  whether the time went to the auth provider, the upstream, or a
  response transform. Three new optional fields land on every
  AccessLogEntry: auth_ms (request_start → auth provider
  returned), upstream_ttfb_ms (request_start → first upstream
  response byte), response_filter_ms (first upstream byte → end
  of response_filter). All three are Option<f64> and
  serde-skip when None, so origins that short-circuit (cache
  hit, auth deny) keep compact lines. The same observations also
  feed a new sbproxy_phase_duration_seconds{phase, origin}
  histogram with buckets identical to
  sbproxy_request_duration_seconds for cross-cut dashboards. See
  docs/access-log.md and docs/metrics-stability.md.

• Nine standard HTTP fields on the access log: host, query,
  protocol, scheme, user_agent, referer, upstream_status,
  response_content_type, response_content_encoding. The log
  was missing the canonical fields most HTTP access-log consumers
  expect (Apache, NGINX, Envoy, the cookie-cutter ELK pipeline).
  host is the client-supplied Host header (distinct from
  origin, the matched virtual-host pattern); upstream_status
  is the upstream's response code when the proxy rewrote the
  status the client sees. All nine are Option, serde-skip when
  not applicable. Promoted from the generic header allowlist
  because nearly every analytics consumer wants them. See
  docs/access-log.md.

• Opt-in OpenTelemetry metrics mirror alongside the canonical
  Prometheus surface. New telemetry.export_metrics: true
  (with telemetry.metrics_interval_secs cadence, default 30s)
  installs an OTel MeterProvider that ships observations to the
  same OTLP collector the trace pipeline targets. The first two
  mirrored instruments are sbproxy.phase.duration and
  sbproxy.request.duration; record-paths fall back to OTel's
  global no-op meter when the export is off, so operators pay
  nothing for the mirror unless they opt in. The Prometheus
  surface remains canonical; this is for operators who already
  aggregate via Mimir / Datadog / Honeycomb and want to skip the
  Prometheus scrape.

• OIDC Relying-Party stack shipped end to end.
  /oidc/callback (auth-code + PKCE + sealed session cookie)
  plus the helpers + config wiring for
  /.well-known/openid-configuration discovery, refresh-token
  rotation, RP-initiated logout at /oidc/logout, userinfo →
  X-Auth-* trust headers, an optional server-side session store
  (in-memory + KV-backed redb/file/Redis) for targeted revocation.
  See docs/configuration.md § OIDC auth.

• OpenAI Apps SDK / MCP Apps (SEP-1865) compatibility.
  Gateway-side _meta.mcpApps passthrough for tool definitions,
  params.audit.cause plumbing on tools/call, and a typed
  validator set (apps.template_declared, apps.iframe_sandbox,
  apps.csp_present, apps.cache_metadata) usable by sbproxy,
  the enterprise extension, and any CI gate over the
  sbproxy-plugin surface.

• Web Bot Auth full conformance, publish + sign sides.
  SBproxy now publishes its own JWKS-shaped
  directory at /.well-known/http-message-signatures-directory
  and a Signature Agent Card at
  /.well-known/web-bot-auth/agent-card (opt in via
  web_bot_auth_publish per origin). New
  sbproxy-middleware::signatures::MessageSignatureSigner
  primitive signs outbound requests per RFC 9421, round-trips
  through the existing verifier. See docs/web-bot-auth.md and
  examples/web-bot-auth-publish/.

• Three previously-undocumented OSS policies now have docs +
  runnable examples: object_authz (BOLA + BFLA with
  enumeration detection), content_digest (RFC 9530 request-body
  verification), agent_budget (per-agent semantic rate limit).
  See docs/object-authz.md, docs/content-digest.md,
  docs/agent-budget.md.

• Discoverable FAQ. docs/faq.md covers install, common
  401 causes, OIDC minimal config, log levels, OSS-vs-enterprise
  scope, and pointers into the rest of docs/. Wired into
  docs/README.md under "Getting started".

• Explicit SIGINT/SIGTERM handling with a structured shutdown
  event and a 30s default drain budget. Pingora's
  Server::run_forever already trapped SIGTERM and SIGINT, but
  the proxy emitted no operator-facing log line on receipt, so a
  pod eviction or docker stop looked the same as a crash in the
  log stream. This change subscribes to Pingora's execution-phase
  broadcast and emits shutdown_signal_received,
  shutdown_grace_period, and shutdown_complete tracing events
  with the resolved grace budget. The Kubernetes operator
  (sbproxy-k8s-operator) now installs the same SIGINT/SIGTERM
  handlers via tokio::signal::ctrl_c and
  tokio::signal::unix::signal(SignalKind::terminate()); before
  this change the operator relied on the orchestrator SIGKILL at
  terminationGracePeriodSeconds. The drain budget is the new
  SBPROXY_SHUTDOWN_GRACE_MS env var (or --shutdown-grace-ms
  CLI flag) which defaults to 30000ms, matching Kubernetes'
  default terminationGracePeriodSeconds. The legacy
  SB_GRACE_TIME / --grace-time (seconds) still works and
  takes precedence when explicitly set; an unset legacy var lets
  the new 30s default apply. Operator exits 0 on a clean drain,
  1 when the grace window is exceeded, so the orchestrator can
  alert. Documented in docs/manual.md §3 and
  docs/kubernetes.md §Graceful shutdown.

• Idempotency middleware now engages on AI gateway origins
  (action: ai_proxy). Before this change, the
  RFC 8594 middleware only ran on general HTTP origins
  (action: proxy). AI customers using Idempotency-Key
  headers for Stripe-style retries were double-billed by the
  upstream provider because the proxy did not replay from cache.
  The fix engages the same primitive in handle_ai_proxy after
  the request body is buffered (the AI gateway already buffers
  for the JSON parser, model router, and guardrails) and before
  the upstream call. On a cache hit the gateway writes the
  cached (status, headers, body) triple directly to the client
  with x-sbproxy-idempotency: HIT and never contacts the
  provider. On a body conflict the gateway returns 409
  ledger.idempotency_conflict per the RFC. On a miss the
  gateway forwards, then records the post-translation OpenAI-shape
  bytes the client actually saw so retries replay byte-identical.
  Reuses the same per-request and pool caps shipped on
  CompiledIdempotency: max_request_body_bytes,
  max_response_body_bytes, max_concurrent_buffers. The four
  skip markers (SKIPPED-OVERSIZE-REQUEST, SKIPPED-POOL-FULL,
  SKIPPED-OVERSIZE-RESPONSE, SKIPPED-MULTIPART) stamp on the
  outgoing response so operators see graceful degradation in
  dashboards. Multipart bodies (audio transcription, image edit /
  variation, file upload) skip caching with SKIPPED-MULTIPART
  because the cache primitive stores raw bytes and multipart
  boundaries may be regenerated by clients on retry. Streaming
  (SSE) chat completion responses abandon the cache record on
  oversize because framing-aware capture is out of scope for v1.

• proxy_status and problem_details now cover upstream
  failures. Before this change, proxy_status.enabled: true
  stamped the Proxy-Status header on proxy-generated errors
  (auth deny, policy deny, default 404) but not on upstream
  failures routed through Pingora's fail_to_proxy path (connect
  refused, connect timeout, TLS handshake error, mid-stream
  connection loss). The fix wires both blocks into the
  upstream-failure path so dashboards consuming Proxy-Status see
  consistent coverage across error sources. The status code +
  RFC 9209 error token derive from the Pingora ErrorType via
  a new map_upstream_failure translator: 504 +
  connection_timeout for ConnectTimedout /
  ReadTimedout; 502 + connection_refused for ConnectRefused;
  502 + tls_protocol_error for TLS errors; 502 +
  connection_terminated for mid-stream loss; 502 +
  http_request_error as the catch-all. When
  problem_details.enabled: true the body is now rendered as
  application/problem+json for upstream failures too, with the
  RFC 9209 error token in the detail field so both signals share
  the same vocabulary.

• Idempotency cache check moved to request_filter. Before this
  change, the cache lookup ran in request_body_filter, after
  Pingora had already opened the upstream TCP connection. On a cache
  hit the upstream observed one aborted partial request before the
  proxy served the cached response to the client. The check now runs
  before Pingora's upstream-peer phase: cache hits and body
  conflicts write the response from inside request_filter and
  return Ok(true), so the upstream is never contacted at all. On
  cache miss the proxy buffers the body (bounded by
  max_request_body_bytes from PR #139), then re-injects it via
  request_body_filter at end-of-stream so Pingora's normal upstream
  forwarding picks it up. Existing e2e tests now assert the
  upstream-not-contacted invariant; the previous "may observe one
  aborted partial request" caveat has been removed from
  docs/configuration.md and the example README.

• Idempotency middleware: per-request and pool caps. Three new
  fields on the idempotency: block bound memory usage and let the
  middleware gracefully degrade under pressure rather than buffering
  unbounded bodies. max_request_body_bytes (default 1 MiB) caps
  the per-request buffer; bodies above the cap skip caching with
  x-sbproxy-idempotency: SKIPPED-OVERSIZE-REQUEST stamped on the
  response. max_response_body_bytes (default 1 MiB) caps the
  per-response cache buffer; responses above the cap stream through
  uncached. max_concurrent_buffers (default 256) is a per-origin
  pool over concurrent buffered requests; pool exhaustion skips the
  cache with x-sbproxy-idempotency: SKIPPED-POOL-FULL. Worst-case
  memory is bounded at max_concurrent_buffers * max_request_body_bytes
  per origin.

• RFC 8594 idempotency middleware (idempotency:). Per-origin
  block that engages on POST / PUT / PATCH (configurable via
  methods:) when an Idempotency-Key header is present. The
  middleware sits ahead of policies in the handler chain, hashes the
  request body, and short-circuits the three branches per the RFC:
  cache hits replay the cached (status, headers, body) verbatim
  with x-sbproxy-idempotency: HIT; conflicts (same key, different
  body) return 409 with the ledger.idempotency_conflict JSON body;
  misses forward to the upstream and capture the response for the
  next retry. Workspace-isolated keys prevent cross-tenant
  collisions. Memory backend (default) is per-origin and per-replica;
  backend: redis binds to proxy.l2_store at config-compile time
  for cluster-wide replay. Cached replays do not consume rate-limit
  slots. Documented in docs/configuration.md and demonstrated by
  examples/idempotency/. Known v1 limitation: the cache check
  fires in request_body_filter, after Pingora has already opened
  the upstream connection. On a cache hit the upstream observes one
  aborted partial handshake before the proxy serves the cached
  response to the client; future work moves the check earlier so the
  upstream never sees the replay.

• RFC 9457 problem-details default renderer (problem_details:).
  New per-origin block that opts in to application/problem+json for
  proxy-generated errors (authentication denials, policy denials,
  default 404) that are not matched by an authored error_pages
  entry. The two blocks compose: per-status custom pages still win
  when authored; problem_details catches everything else with a
  structured type / title / status / detail / instance
  body. type_base_uri produces stable per-status type URIs;
  include_detail: false suppresses the internal error string.
  Documented in docs/configuration.md and demonstrated by
  examples/problem-details/.

• Typed error_pages config. The opaque
  error_pages: Option<serde_json::Value> field is now typed as
  Option<Vec<ErrorPageEntry>>. Public types ErrorPageEntry,
  StatusSpec, and ProblemDetailsConfig live in sbproxy-config.
  The authored YAML shape is unchanged: every existing
  error_pages: list keeps parsing, including the status: single-
  int / [status] list shorthand and template: true substitution.
  The OpenAPI emitter now walks typed entries to populate
  per-status responses keys (the previous code inspected the
  field as an object and silently produced no entries; this is a
  bug fix on top of the migration).

• AI gateway Realtime WebSocket dispatch (Phase 7, Option C).
  GET /v1/realtime requests with Upgrade: websocket against an
  ai_proxy origin are now dispatched through the AI gateway
  pipeline:

  • Pre-upgrade gating runs the same surface classification, 501
    capability check (only providers in
    provider_supports_realtime are eligible; today: OpenAI),
    per-surface rate limit, and provider selection as the rest of
    the AI surface set.
  • After the gating passes, Pingora forwards bytes between
    client and provider transparently through the upgraded
    connection. The dispatcher does not terminate the WebSocket;
    per-frame guardrails and frame-exact audio metering are
    reserved for a future enterprise terminate-and-relay path so
    every AI gateway feature added to handle_action continues
    to apply to realtime through one shared code path.
  • sbproxy_ai_realtime_sessions_active (gauge),
    sbproxy_ai_realtime_session_duration_seconds (histogram),
    sbproxy_ai_realtime_audio_seconds_total (counter), and
    sbproxy_ai_realtime_frames_forwarded_total (counter) are
    registered. The OSS dispatch ticks the gauge on session open
    and observes the duration histogram on close. Documented in
    docs/metrics-stability.md.
  • At session close, logging emits a session-end
    AiBillingEvent with AudioSeconds { seconds } valued at
    the wall-clock session duration so realtime usage appears on
    the standard billing-event bus alongside chat/image/audio.
  • RealtimeSessionTracker (lock-free atomic counters) and
    audio_seconds_from_frame(bytes, sample_rate, channels) ship
    in sbproxy-ai::realtime for the eventual terminate-and-relay
    path to consume.
  • docs/ai-gateway.md documents the new dispatch path with a
    YAML example and the per-surface rate-limit knob.

• AI gateway OpenAI surface dispatch (Option A). The ai_proxy
  action now routes every OpenAI-compatible surface through a
  single classifier with per-surface observability and gating:

  • New AiSurface enum + classify_surface(method, path) cover
    chat completions, models, embeddings, assistants and threads
    (full v2 surface), batches, fine-tuning, files, realtime,
    image generation/edits/variations, audio transcription/speech,
    moderations, and reranking. Marked #[non_exhaustive] so
    future variants don't break downstream pattern matches.
  • Method coverage extended past GET/POST: DELETE, PUT, PATCH,
    HEAD, and OPTIONS dispatch through AiClient::forward_with_method
    without engaging the JSON body-parse pipeline.
  • Multipart bodies (image edits/variations, audio transcription,
    file uploads) byte-forward via AiClient::forward_bytes with
    the inbound Content-Type preserved. Previously these surfaces
    returned a 400 "invalid JSON body" from the chat-path body parse.
  • Provider capability matrix in api_routes.rs corrected:
    Anthropic no longer claims audio/reranking/moderations support,
    Gemini no longer claims moderations. A new
    provider_supports_surface matrix gates non-universal surfaces
    with 501 Not Implemented when no configured provider
    supports the surface.
  • Per-surface observability: new
    sbproxy_ai_surface_requests_total{surface, method} counter and
    sbproxy_ai_surface_request_duration_seconds{surface, method}
    histogram. Sibling of the existing per-provider metrics so
    dashboards can pivot between surface and provider views.
    Documented in docs/metrics-stability.md.
  • Per-surface input guardrails: image generation, audio speech,
    reranking, and moderations bodies now have their input field
    (prompt, input, query, input) extracted and run through
    the same guardrail pipeline as chat-style messages.
  • Per-surface rate limits: new per_surface_rate_limits field
    on the AI handler config, keyed by surface label. 429 fires
    before any upstream call when the cap is hit.
  • Surface-aware billing event: new AiBillingEvent carrying
    AiUsage with Tokens, Images { count, resolution },
    AudioSeconds, Characters, RerankUnits, and PerCall
    variants. Every dispatched request emits exactly one event.
    Image generation, audio speech, and reranking emit real cost
    via per-surface pricing tables (lookup_image_price,
    lookup_audio_speech_price, lookup_rerank_price,
    lookup_audio_transcription_price). docs/ai-gateway.md
    documents the new surface, methods, guardrails, and rate-limit
    knobs.

• Policy verdict audit bus + Plugin dispatch.
  Wires the previously-dead Policy::Plugin arm in server.rs to
  call the trait's enforce(), folds the returned PolicyDecision
  into the existing chain reducer, and emits a
  PolicyVerdictEvent for every decision on a bounded
  tokio::sync::mpsc audit bus per
  docs/adr-policy-audit-binding.md. The OSS substrate ships an
  in-memory drain stub; enterprise replaces the consumer with a
  NATS-backed audit-chain subscriber. Multi-policy resolution
  rules from docs/adr-policy-verdict-shape.md are implemented at
  the chain level: any Deny wins, the first Confirm wins over
  AllowWithHeaders, AllowWithHeaders accumulate, otherwise Allow.
  Confirm in OSS routes through the existing AllowWithHeaders
  mechanism with X-Policy-Confirm: <reason> stamped on the
  response; an expires_at already in the past synthesises a 410
  and an SSRF-blocked webhook_url synthesises a 502 at decision
  time. New metrics:
  sbproxy_policy_audit_events_total{verdict, surface, policy_id},
  sbproxy_policy_audit_events_dropped_total{tenant},
  sbproxy_policy_decision_duration_seconds{surface}. New Grafana
  dashboard sbproxy-policy-verdicts covers the surface.
  ([crates/sbproxy-observe/src/events.rs],
  [crates/sbproxy-observe/src/metrics.rs],
  [crates/sbproxy-core/src/policy_bus.rs],
  [crates/sbproxy-core/src/policy_dispatch.rs],
  [crates/sbproxy-core/src/server.rs],
  [crates/sbproxy-plugin/src/traits.rs],
  [dashboards/grafana/sbproxy-policy-verdicts.json])

• Synthetic-transaction /readyz probe. Optional
  background driver that fires an in-process request through the
  compiled handler chain on a fixed cadence and reports the verdict as
  a synthetic_pipeline component on /readyz. Disabled by default;
  opt in via proxy.synthetic_probe.enabled: true and define an origin
  for the configured sentinel hostname (default __synthetic.local)
  pointing at a non-network action (static, mock, echo, noop).
  Failures bump the new
  sbproxy_synthetic_probe_failures_total{reason} counter so they do
  not pollute real-traffic error metrics.
  ([crates/sbproxy-config/src/types.rs],
  [crates/sbproxy-core/src/synthetic.rs],
  [crates/sbproxy-observe/src/synthetic.rs],
  [crates/sbproxy-observe/src/metrics.rs],
  [e2e/tests/synthetic_probe.rs])

• GET /admin/drift config drift endpoint. Returns
  whether the on-disk config file has diverged from what the running
  proxy has loaded, without triggering a reload. Compares a
  content-hash baseline captured at startup (and refreshed on every
  /admin/reload) against a fresh hash of the current file. K8s
  operators and dashboards scrape this so they can flag an edited
  config that has not been hot-reloaded yet. Documented in
  docs/configuration.md § Admin fields.
  ([crates/sbproxy-core/src/admin.rs],
  [crates/sbproxy-core/src/server.rs],
  [docs/configuration.md])

• Deterministic clock-skew testing hooks. ClockSkewMonitor now
  accepts an injected clock source for tests while production continues
  to use the system clock.
  ([crates/sbproxy-observe/src/clock_skew.rs])

• Operator runbook hooks and fast-track ADR template. Added a
  dashboard-oriented operator runbook, linked all Grafana panels to the
  relevant triage sections, and added a fast-track ADR amendment
  template plus OSS threat-model refresh checklist.
  ([docs/operator-runbook.md], [docs/adr-fast-track-amendment.md],
  [docs/threat-model.md], [dashboards/grafana/])

• Live reverse-DNS resolver for agent verification. SystemResolver
  now uses hickory-resolver for PTR and forward-confirmation lookups,
  replacing the previous typed PTR stub.
  ([crates/sbproxy-security/src/agent_verify.rs])

• Multi-window SLO burn-rate replay harness. sbproxy-observe
  now includes a burn-rate evaluator and AlertSnapshot replay helper
  for substrate availability and latency alert taxonomy tests.
  ([crates/sbproxy-observe/src/alerting/burn_rate.rs],
  [e2e/tests/slo_burn_rate.rs])

• Vault-style quote-token seed references. ai_crawl_control.quote_token.secret_ref
  now accepts secret: references resolved through sbproxy-vault
  with the existing environment fallback, in addition to the older
  secret_ref.env and inline seed_hex paths.
  ([crates/sbproxy-modules/src/policy/ai_crawl.rs])

• Operator first-24-hours quickstart. Added a concise
  docs/quickstart-operator.md covering deploy, /readyz, metrics,
  Grafana, logs, and rollback, linked from the README and Kubernetes
  docs.
  ([docs/quickstart-operator.md])

• Hostname cardinality override for metrics. proxy.metrics.cardinality.hostname_cap
  can lower the hostname label budget independently from the default
  per-label cap, enabling deterministic overflow tests and tighter
  multi-tenant Prometheus budgets.
  ([crates/sbproxy-config/src/types.rs],
  [crates/sbproxy-observe/src/cardinality.rs])

• release-fast build profile for CI images. Docker-based CI and
  local kind smoke-test builds can now use CARGO_PROFILE=release-fast
  to skip fat LTO and use more codegen units, cutting link memory/time
  while leaving production release artifacts on the existing release
  profile.
  ([Cargo.toml], [Dockerfile.ci], [Dockerfile.cloudbuild])

• Reproducible build probe workflow. CI now has an informational
  double-build lane that builds the release binary twice on independent
  GitHub-hosted runners, uploads each binary and SHA-256, and publishes
  a comparison report without yet treating non-identical output as a
  failure.
  ([.github/workflows/reproducible-build.yml], [SUPPLY-CHAIN.md])

• Phase 2: CEL features[...] namespace. Per-request
  flags parsed from the x-sb-flags header and ?_sb.<key> query
  prefix are now exposed to CEL expressions. Built-in flags surface
  as bools (features.debug, features.trace,
  features["no-cache"], features.any_set); free-form k=v extras
  surface as strings (features["env"]). Wired into the rate-limit
  CEL evaluator and ExpressionPolicy::evaluate_with_views.
  ([crates/sbproxy-extension/src/cel/context.rs])

• SB_WORKER_THREADS env var. Positive integer overrides the
  auto-detected Pingora worker thread count
  (std::thread::available_parallelism()). Useful for benchmarking
  with a fixed worker count or capping the pool below a cgroup quota.
  ([crates/sbproxy-core/src/server.rs])

• /live, /livez, /ready, /healthz, and rich /health
  admin endpoints.
  /livez returns {"alive":true} on every call and never 503s, so
  K8s liveness probes don't trip on transient readiness failures.
  /live is a bare alias. /ready is an alias for /readyz.
  /healthz stays a fixed liveness body, while /health now returns
  version, build hash, timestamp, uptime, and readiness checks for
  dashboards / SIEM ingestion. Existing /readyz behavior unchanged.
  ([crates/sbproxy-observe/src/health.rs],
  [crates/sbproxy-core/src/admin.rs])

• --request-log-level and SB_REQUEST_LOG_LEVEL. Operators can
  now tune request/access logging independently from application logs.
  The setting appends an access_log=<level> target directive to the
  effective tracing-subscriber filter while preserving the existing
  per-target RUST_LOG escape hatch.
  ([crates/sbproxy/src/main.rs])

• Access-log forced emission and file output. access_log now
  supports slow_request_threshold_ms and always_log_errors so slow
  requests and 5xxs bypass sampling after status/method filters match.
  It also supports output: { type: file, path, max_size_mb, max_backups, compress } for direct JSON-line access-log files with
  size-based rotation and optional gzip compression of rotated files.
  ([crates/sbproxy-config/src/types.rs],
  [crates/sbproxy-core/src/server.rs],
  [crates/sbproxy-observe/src/access_log.rs])

• OCSP stapling for the manual fallback cert. OcspStapler
  (which previously existed but was unwired) now does an immediate
  fetch on startup, refreshes every 12 hours, and pushes the bytes
  into CertResolver::update_fallback_ocsp so subsequent rustls
  handshakes staple the response on the wire. No-op when no manual
  cert is configured or when the cert lacks an AIA extension.
  ([crates/sbproxy-tls/src/ocsp.rs],
  [crates/sbproxy-tls/src/cert_resolver.rs])

• Readiness synthetic probe primitive. sbproxy-observe now ships a
  SyntheticProbe type so startup or test wiring can register an
  in-process readiness probe that exercises a caller-provided path and
  reports through the same /readyz component model as built-in probes.
  ([crates/sbproxy-observe/src/health.rs])

Removed

• sbproxy_ai::IdempotencyCache. The OSS AI gateway never wired
  this cache; it was publicly re-exported but had zero callers in the
  workspace. The new idempotency: block on general HTTP origins
  (above) supersedes it. AI gateway integration is a follow-up tracked
  in docs/missing.md. Plugin authors that imported the removed
  type can switch to
  sbproxy_middleware::idempotency::{IdempotencyCache, InMemoryIdempotencyCache, KvIdempotencyCache} which carries the
  richer surface (workspace isolation, body-hash conflict detection,
  conflict body builder).

Changed

• mTLS now wired on the ACME path. Previously, an operator who
  configured mtls: alongside acme: got plain TLS until they
  noticed clients reaching the upstream without the expected cert
  headers. The ACME branch now mirrors the manual-cert branch:
  builds TlsSettings with the configured ClientCertVerifier and
  falls back to plain TLS only when mTLS setup itself fails.
  ([crates/sbproxy-core/src/server.rs])

• Examples and Kubernetes smoke checks are local-only. The
  Docker-backed examples smoke lane and kind-based Kubernetes operator
  smoke lane no longer run automatically on pull requests. They remain
  available as make examples-smoke and make k8s-operator-smoke for
  explicit local / release validation.
  ([Makefile], [docs/kubernetes.md])

• Reload drain state is now one coherent atomic snapshot. The
  drain flag and active request count are packed into one AtomicU64,
  so is_draining() no longer combines two independent relaxed loads.
  Added loom coverage for the last-request-finish interleaving.
  ([crates/sbproxy-core/src/reload.rs])

• Optional readiness dependencies no longer fail /readyz by
  default. The default admin health registry now registers absent
  ledger and bot-auth-directory probes as not_configured, matching the
  existing future-wave stubs and keeping /readyz green when those
  optional services are not wired in a deployment.
  ([crates/sbproxy-observe/src/health.rs],
  [crates/sbproxy-core/src/admin.rs])

• docs/manual.md rewrites matching what actually ships:

  • §6 Health checks: /livez, /readyz, /healthz, and rich
    /health semantics, replacing the old per-endpoint URL fork
    diagram and stale /health alias wording.
  • §10 Feature flags: CEL accessor table, kill-switch note, and
    a "planned, not yet wired" note for Lua / JS / WASM features
    namespaces and workspace-level pub/sub flags.
  • §3 CPU detection: documents the new SB_WORKER_THREADS knob.
  • §13 env-var table: adds SB_WORKER_THREADS and
    SB_DISABLE_SB_FLAGS; later updates add
    SB_REQUEST_LOG_LEVEL and access-log file/forced-emit examples.

Fixed

• CAP sub binding only fires for a genuinely resolved agent. The
  CAP verifier binds a token's sub to the request's resolved agent id
  (rejecting a mismatch with 403). Because the agent-class resolver is
  installed with the built-in catalog by default and always stamps
  some id (falling through to the human sentinel when no signal
  matches), the binding would have rejected every CAP token whose sub
  was not literally "human", even on origins that never configured
  agent classes. The binding now skips the resolver's fallback / human
  verdict and engages only when the resolver actually identified an
  agent, so an unauthenticated caller falls through to the normal CAP
  validation path. Set cap.require_agent_binding: true to fail closed
  when no agent is resolved.

• Virtual-key model allow/block lists are now enforced. A virtual
  key (or ai_provider credential) with models.allow / models.block
  declared its scope but the AI dispatch path never checked it, so a key
  confined to a subset of the gateway's models could still call any
  model the gateway served. The matched key's allow/block lists are now
  enforced against the effective model (after any route_to_model
  rewrite): a request for a disallowed model is rejected with 403
  before any upstream call, the block-list taking precedence over the
  allow-list. Keys with no models.allow are unaffected. See
  examples/ai-virtual-keys/.

• Licensing-projection wire formats now match the canonical specs [BREAKING]. Two projection emitters were producing
  document shapes that didn't match their cited specifications.
  /licenses.xml previously declared the namespace
  https://rsl.ai/spec/1.0 and emitted a flat
  <rsl><license urn=...>...</license></rsl> document. The canonical
  RSL Collective spec at https://rslstandard.org/rsl uses the
  namespace https://rslstandard.org/rsl and a nested
  <rsl><content url="..."><license>...</license></content></rsl>
  shape; the <content> url attribute is the canonical wildcard
  https://<hostname>/* for the origin-wide license. /.well-known/tdmrep.json
  previously wrapped its policies in a {"version", "generated", "policies": [...]}
  envelope; the W3C TDMRep CG-FINAL spec mandates a bare JSON array
  at the document root with location, tdm-reservation
  (integer 0 or 1), and tdm-policy (URL of the policy document)
  fields per entry. Both emitters now produce the canonical shapes.
  Operators consuming /licenses.xml or /.well-known/tdmrep.json
  programmatically must update their parsers to the new shapes; the
  in-process JSON envelope and the response middleware that stamps
  TDM-Reservation: 1 and the URN-bearing license field are
  unaffected. Conformance is asserted by the active structure-shape
  tests; the earlier schema-validation tests were removed because
  neither standard publishes a machine-readable schema to validate
  against (RSL 1.0 is prose-only; W3C TDMRep ships no JSON Schema).
  ([crates/sbproxy-modules/src/projections/licenses.rs],
  [crates/sbproxy-modules/src/projections/tdmrep.rs],
  [e2e/tests/rsl_licenses_projection_e2e.rs],
  [e2e/tests/tdmrep_projection_e2e.rs])

• Build under prometheus 0.14 type inference. Sites in
  sbproxy-observe::metrics and sbproxy-core::server that passed
  heterogeneous &[&String, &str] arrays to
  prometheus::with_label_values no longer compile on prometheus
  0.14 because Rust unifies the array element type to &String and
  rejects bare &str literals. Coerced all such call sites to
  uniform &[&str] via .as_str() so the workspace builds clean
  again. No behavioural change.
  ([crates/sbproxy-observe/src/metrics.rs],
  [crates/sbproxy-core/src/server.rs])

• WASM extension docs corrected. CLAUDE.md previously labeled the
  WASM surface as "WASM stub" while marketing docs claimed
  production-grade support; the runtime is real
  (wasmtime + WASI preview-1 with sandboxed memory and CPU caps,
  stderr capture, no FS or network). llms.txt also incorrectly
  claimed "WASI networking with host allowlist" but allowed_hosts is
  parsed-but-inert until WASI sockets land. CLAUDE.md and llms.txt now
  match the shipped surface.
  ([CLAUDE.md], [llms.txt],
  [crates/sbproxy-extension/src/wasm/mod.rs])

• E2E proxy startup flake under CPU contention. The e2e
  ProxyHarness keeps its HTTP-level readiness probe, but now gives
  release/debug proxy boots a 10-second window instead of 5 seconds so
  tests like action_graphql do not fail spuriously while cargo is
  competing for CPU.
  ([e2e/src/lib.rs])

• Docs CI Rust snippet failures. Workspace-dependent documentation
  examples that cannot compile as standalone rust-script programs are
  now tagged rust,no_run, keeping docs-ci focused on executable
  snippets instead of illustrative API fragments.
  ([docs/architecture.md], [docs/audit-log.md], [docs/cache-reserve.md])

• Unsafe-code drift guardrails. Crates that do not need unsafe now
  forbid it at the crate root, while sbproxy-vault explicitly allows
  its narrowly-scoped volatile zeroization unsafe with an inline
  justification.
  ([crates/sbproxy-*/src/lib.rs])

• Outbound webhook delivery identity headers. Signed customer
  webhooks now include Sbproxy-Subscription-Id,
  Sbproxy-Delivery-Id, and 1-based Sbproxy-Attempt headers, with a
  fresh delivery ULID on every retry attempt.
  ([crates/sbproxy-observe/src/notify.rs])

• AI client retry resilience. MemoryBatchStore now uses
  parking_lot::Mutex so a panic in one worker cannot poison the
  in-memory batch map for every later operation. Provider retries now
  honor provider.max_retries as same-provider retry attempts with
  bounded jittered exponential backoff before recording provider
  failure and moving to the next eligible provider.
  ([crates/sbproxy-ai/src/batch.rs],
  [crates/sbproxy-ai/src/client.rs])

• Dynamic Web Bot Auth directory dispatch. The main request auth
  path now invokes BotAuthProvider::verify_async when a configured
  hosted directory and Signature-Agent header are present, so dynamic
  directory failures surface distinctly instead of falling through the
  static inline-agent verifier.
  ([crates/sbproxy-core/src/server.rs])

• ACME/Pebble order polling. Certificate issuance now polls the
  authorization to valid after responding to the HTTP-01 challenge
  before polling the order to ready, matching Pebble's stricter state
  progression. Finalization also parses the order returned by the
  finalize response and falls back to polling the original order URL,
  avoiding accidental POST-as-GET polling of the finalize URL when
  Location is absent.
  ([crates/sbproxy-tls/src/acme.rs])

• JWKS unknown-kid key rotation. JWTs that reference an unseen
  kid now trigger one rate-limited JWKS refetch before failing
  closed, with a Prometheus counter for success / failure /
  rate-limited outcomes. This avoids requiring operator intervention
  for routine IdP key rotation.
  ([crates/sbproxy-modules/src/auth/jwks.rs],
  [crates/sbproxy-modules/src/auth/mod.rs],
  [crates/sbproxy-observe/src/metrics.rs])

• Rate-limit LRU pollution bypass. Per-key local token buckets now
  preserve deny state in a bounded cold tier after hot LRU eviction, so
  a spray of attacker keys cannot reset an already-throttled
  legitimate client.
  ([crates/sbproxy-modules/src/policy/mod.rs])

Open follow-ups

Tracked in Linear, not in this changeset:

• the upstream issue full configurable
  synthetic transaction through the live request pipeline. The
  SyntheticProbe readiness primitive has landed; config and pipeline
  execution remain.
• Phase 2.5: Lua / JS / WASM features namespace, plus
  workspace-level flags via messenger pub/sub
• the upstream issue remaining
  rate-limiter proptest coverage. The reload-drain loom portion has
  landed.

[1.0.1] - 2026-05-04

Patch release. No runtime behavior changes.

Fixed

• Container image publish: the release.yml workflow's docker
  prepare step extracted the flat-layout tarballs into /tmp/
  directly, which tripped a sticky-bit Cannot utime error on the
  archive's ./ entry and caused ghcr.io/soapbucket/sbproxy:1.0.0
  to never publish. Each platform tarball now extracts to a per-arch
  staging dir before the binary moves into the docker context.

[1.0.0] - 2026-05-03

First Rust release of SBproxy on this repository.

What changed

• Implementation: SBproxy is now written in Rust on Cloudflare's
  Pingora. The Go implementation that previously occupied this repo
  (v0.1.0 through v0.1.2) has moved to
  soapbucket/sbproxy-go,
  preserved as the v0.1.2-go-final branch and tag, and is now in
  maintenance-only mode.
• Data plane: routing, AI gateway, MCP gateway, guardrails, security
  policies, and scripting (CEL, Lua, JavaScript, WebAssembly) all ship
  open source in this release. See docs/architecture.md
  for the request pipeline shape.
• Enterprise tier: see docs/enterprise.md for
  what enterprise adds on top of the OSS data plane and how to request
  access.

Upgrading from v0.1.x (Go)

The internal config schema (schema-v1) is supported by both the Go
v0.1.x line and this Rust v1.x line, so existing sb.yml files
should compile unchanged. See MIGRATION.md for the
full upgrade path.