feat(router): hard cost-envelope enforcement — warn/downgrade/block (A6) - #116
Merged
Conversation
… paths (A6) Per-tier daily budgets only ever warned; upstream evidence shows a Claude tier running ~25x its daily budget with nothing but budget_exceeded WARN lines. This vendors the upstream private deployment's stdlib-only decision module (budget_enforcement.py: resolve_mode / decide / BudgetDecision / downgrade_header / block_detail) verbatim and wires it into every router path behind BUDGET_ENFORCE_MODE: - warn (default): behavior unchanged — serve + WARN log (ship-dark safe) - downgrade: serve BUDGET_FALLBACK_TIER (default gpt4o-mini) and mark the response with X-Router-Budget-Downgrade + _router.budget_downgraded_from - block: 429 with a machine-readable budget_exceeded body Coverage: - /v1/chat/completions: enforcement wraps select_tier as its final stage (covers Anthropic direct-SDK dispatch and ephemeral passthrough tiers; the pre-existing gpt4o-mini->phi4 redirect is untouched) - /v1/messages (native Anthropic path, bypasses select_tier): explicit check; block returns an Anthropic-shaped 429; downgrade only serves an Anthropic-backed fallback (response must stay Anthropic-shaped), otherwise degrades to warn. The path also now records its spend to the daily ledger (streaming + non-streaming) — it previously recorded nothing, which would have left enforcement blind on this exact path - /v1/embeddings (no tier at all): spend now accrues to a dedicated 'embeddings' ledger bucket capped by EMBEDDING_DAILY_BUDGET_USD (0 disables, mirroring PER_CALLER_DAILY_USD); no same-vector-space downgrade target exists so downgrade degrades to warn; block 429s. Also gains the aaf-0005 per-caller budget gate for parity Fail-open guarantees: invalid mode -> warn; unregistered/self-referential fallback -> warn (the fallback tier is the designated floor and is exempt). No selftest/probe traffic exists in this router, so nothing needs a spend exemption. Dockerfile now ships the new module alongside main.py. Tests: 200 -> 237 (37 new in tests/test_budget_enforcement.py) covering the pure module, warn-is-ship-dark, downgrade + header/meta, block 429, both bypass paths, fallback exemption, and invalid-mode fail-open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…terraform BUDGET_ENFORCE_MODE + BUDGET_FALLBACK_TIER + EMBEDDING_DAILY_BUDGET_USD (and the EMBEDDING_PRICE_PER_MTOK estimator fallback) documented and defaulted ship-dark (warn) everywhere the router's budget env already lives: .env.example, root docker-compose.yml, deploy/mac-site compose anchor, and both Terraform router sidecars (hermes pod + memory-governor pod, the latter carrying the embeddings cap for governor vector retrieval). New module variables budget_enforce_mode (validated), budget_fallback_tier, embedding_daily_budget_usd. Router README gains a 'Budget enforcement' section with the mode table and per-path semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends the v1.5 cost-governance layer from observe-only to enforce-with-downgrade/block for every router path, behind a single env knob. Default is
warn— behavior identical to today (ship-dark safe).Provenance: port of the upstream private deployment's just-merged M5 "acting budget enforcement". The decision module (
services/model-router/budget_enforcement.py) was designed there as a portability contract — stdlib-only, framework-free, zero host imports — and is vendored verbatim (docstring host references sanitized). The host wiring is thin and adapted to this router's shape.Semantics
BUDGET_ENFORCE_MODE)warn(default)budget_enforceWARN log — exactly the pre-A6 behaviordowngradeBUDGET_FALLBACK_TIER(defaultgpt4o-mini); response carriesX-Router-Budget-Downgrade: <from>-><to>+_router.budget_downgraded_fromblockbudget_exceededbody (error,tier,spent_usd,limit_usd,mode)Guarantees:
warn(a config typo must not brick the router).warnrather than stranding requests.Path coverage (where this router differs from upstream)
/v1/chat/completionsselect_tieras its final stage — covers explicit tiers, persona routing, ephemeral passthrough tiers, and the Anthropic direct-SDK dispatch inside_call_model. The pre-existinggpt4o-mini→phi4over-budget redirect is untouched./v1/messages(native Anthropic, bypassesselect_tier)blockreturns an Anthropic-shaped 429 (rate_limit_error) soanthropic_messagestransports parse it like any upstream error. Divergence from upstream: this router has no Anthropic→OpenAI response translation, sodowngradeonly takes effect whenBUDGET_FALLBACK_TIERis itself Anthropic-backed; otherwise it degrades towarn(module's fallback-unavailable rule). This path also now records its spend (streaming + non-streaming, via list-price estimate) — previously it recorded nothing, which would have left enforcement blind on the exact path the feature was built for./v1/embeddings(#112, no tier at all)embeddingsledger bucket (visible indaily_cost_rollup) capped byEMBEDDING_DAILY_BUDGET_USD(default1.00;0disables, mirroringPER_CALLER_DAILY_USD). No same-vector-space downgrade target exists (the model pin is the point), sodowngradedegrades towarnhere;block429s. Also gains the aaf-0005 per-caller budget gate for parity with the chat endpoints.Config
BUDGET_ENFORCE_MODE—warn(default) |downgrade|blockBUDGET_FALLBACK_TIER— defaultgpt4o-miniEMBEDDING_DAILY_BUDGET_USD— default1.00;0disablesEMBEDDING_PRICE_PER_MTOK— list-price estimator fallback (default0.02, matchestext-embedding-3-small)Plumbed through
.env.example, root + mac-site compose, and both Terraform router sidecars (hermes pod + memory-governor pod) via new validated module variables (budget_enforce_mode,budget_fallback_tier,embedding_daily_budget_usd). Dockerfile now shipsbudget_enforcement.pyalongsidemain.py.Test evidence
New coverage: pure module (resolve_mode normalization + invalid-mode fail-open, decide matrix, block_detail/downgrade_header shapes), warn-is-ship-dark (select_tier + endpoint unchanged, no header/meta), downgrade (+header, +
_routermeta, generic across tiers, unavailable-fallback degradation), block (429 before any upstream call), native/v1/messages(warn serves direct, block 429 Anthropic-shaped, non-Anthropic fallback degrades to warn, Anthropic fallback served natively with header, spend now recorded), embeddings (bucket accrual, warn/block/downgrade, zero-cap disable, per-caller gate), and floor-tier exemption.scripts/scan-internal-refs.sh: clean.docker compose config: valid.terraform fmt -check: clean.🤖 Generated with Claude Code