ProxyOpenAIChatCompletion never consumes UsageBypass: bypassToAnthropic + 429→routeFor fallthrough absent

[rohith500]

TL;DR

runTurnLoop already sets UsageBypass=true for both wire formats when the per-installation gate engages, but only ProxyMessages consumes it via bypassToAnthropic (strict subscription pass-through, skip ledger, and on a retryable 429 fall through to routeFor). ProxyOpenAIChatCompletion ignores routeRes.UsageBypass entirely and continues normal dispatch. Happy-path OpenAI turns often look fine by coincidence (same decision + OAuth). The 429 fallthrough does not: Messages recovers via scorer reroute; OpenAI returns a raw 429 to the client.

Confirmed unintentional incomplete port (same class as #771): gate is shared; Anthropic bypass consumer was Messages-only in #512. No open PR implements this — #762 / #773 explicitly defer it.

Expected vs actual

Happy path (gate on, subscription has headroom, Anthropic-covered requested model)

	Expected (Messages contract)	Actual /v1/messages	Actual /v1/chat/completions
Scorer	not called	not called	not called
x-router-decision	usage_bypass	usage_bypass	usage_bypass
Served model	caller-requested	caller-requested	caller-requested
Client	200	200	200
Path	bypassToAnthropic short-circuit	yes	no — normal OpenAI dispatch of the turn-loop decision
Billing	skip ledger (early return)	skip ledger	emitBilling with SubscriptionServed → delta=0 notional row
OTel	router.usage_bypass span	yes	no (normal upstream span path)

Happy path often “works” for the client on OpenAI — coincidence from the shared turn-loop decision, not parity with Messages’ dedicated consumer.

Failure path (UsageBypass engaged, Anthropic returns buffered weekly-limit 429)

	Expected	Actual /v1/messages	Actual /v1/chat/completions
Client	200 via recovery	200	raw 429
After bypass 429	clear UsageBypass, routeFor (scorer)	scorer runs once; decision becomes cluster pick	scorer never runs; decision stays usage_bypass
Dispatch	bypass attempt → routed attempt	429 then OK on routed path	3× same-binding OAuth 429, then surface error
Deploy-key / Weave retry	via post-bypass routed path (and/or #555 subscription retry on that path)	recovers	no bypassToAnthropic fallthrough on main

Summary / mechanism

1. Gate (shared): usageBypassEngaged / usageBypassDecision in internal/proxy/usage_bypass.go; runTurnLoop sets res.UsageBypass + Decision {Provider: anthropic|openai, Model: requested, Reason: "usage_bypass"} (turnloop.go ~357–361, ~571–574).
2. Consumer (Messages only): ProxyMessages at service.go ~2168:

   if routeRes.UsageBypass && routeRes.Decision.Provider == providers.ProviderAnthropic {
       err := s.bypassToAnthropic(...)
       if !errors.Is(err, errBypassRetryable) { return err }
       // else: clear UsageBypass, routeFor, continue normal dispatch
   }

3. OpenAI: after runTurnLoop (service.go ~4188), immediately decision := routeRes.Decision with zero reads of routeRes.UsageBypass / bypassToAnthropic / errBypassRetryable.

bypassToAnthropic (usage_bypass.go ~227–369) skips scorer, planner, pin, semantic cache, and billing; on providers.IsRetryable pre-commit errors returns errBypassRetryable so the caller can reroute.

Codex-shaped UsageBypass (Decision.Provider == OpenAI) never enters bypassToAnthropic even on Messages — it already uses normal dispatch. This issue is specifically Anthropic UsageBypass consumption on the OpenAI wire (and the 429→routeFor contract).

Is this intentional? Checked, not assumed

• proxy: per-installation subscription usage-bypass gate #512 wired bypassToAnthropic + Messages consult; docs (internal/proxy/usage/CLAUDE.md) say “ProxyMessages passes the request straight through”.
• turnLoopResult.UsageBypass godoc names ProxyMessages.
• But the gate already runs for OpenAI: TestUsageBypass_CodexSubscriptionPreservesRequestedModel asserts OpenAI usage_bypass headers + scorer skip. No doc says “OpenAI must not recover from bypass 429.”
• fix(proxy): suppress exhausted Claude subscription on OpenAI path #762 (open): “Revisit when bypassToAnthropic/UsageBypass consumption is wired on this surface (separate, not-yet-filed issue).”
• fix(proxy): port post-dispatch failover to OpenAI chat completions #773 (open): “UsageBypass on the OpenAI path — still absent; separate issue.”
• No open PR implements this wiring. No closed issue declaring OpenAI UsageBypass out of scope.

Verdict: unintentional incomplete port — Messages-first implementation; OpenAI inherited the gate via the shared turn loop but never got the Anthropic consumer / fallthrough.

Live verification (Go fixture, main @ 5637f3c)

Mirrored bypassFixture / TestUsageBypass_BelowThreshold_SkipsScorer / TestProxyMessages_BypassWeeklyLimit_FallsBackToRoutedDispatch against both surfaces (same models: requested claude-sonnet-4-6, scorer pick claude-haiku-4-5, util 0.20, threshold 0.80, Claude OAuth on ctx).

Happy path

Messages: status=200 decision="usage_bypass" model="claude-sonnet-4-6" scorer=0
OpenAI:   status=200 decision="usage_bypass" model="claude-sonnet-4-6" scorer=0

429 fallthrough

Messages: status=200 decision="cluster:v0.2" model="claude-haiku-4-5" scorer=1
OpenAI:   err=upstream returned status 429 (buffered) status=429
          decision="usage_bypass" model="claude-sonnet-4-6" scorer=0 anthCalls=3
          oai[0..2]: oauth=true src="subscription" err=429

Docker-stack curl was not used as primary proof: UsageBypass needs usage_bypass_enabled on the installation plus a controllable Anthropic 429, which the unit fixture exercises cleanly. Illustrative client shapes for the same surface (local stack) are below.

Illustrative curls (client surface)

Requires: seeded rk_… key, installation with usage_bypass_enabled=true, and a Claude subscription token in X-Weave-Anthropic-Subscription. Replace placeholders.

Messages (works today — including 429 recovery when upstream weekly-limits):

curl -sS -D- -o /tmp/msg.json \
  -H "Authorization: Bearer rk_REPLACE" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -H "X-Weave-Anthropic-Subscription: sk-ant-oat01-REPLACE" \
  http://localhost:8080/v1/messages \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 64,
    "messages": [{"role":"user","content":"Refactor the auth middleware and add tests."}]
  }'
# Expect on success: x-router-decision: usage_bypass, x-router-model: claude-sonnet-4-6

OpenAI wire (gate engages; 429 does not recover like Messages):

curl -sS -D- -o /tmp/oai.json \
  -H "Authorization: Bearer rk_REPLACE" \
  -H "Content-Type: application/json" \
  -H "X-Weave-Anthropic-Subscription: sk-ant-oat01-REPLACE" \
  http://localhost:8080/v1/chat/completions \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 64,
    "messages": [{"role":"user","content":"Refactor the auth middleware and add tests."}]
  }'
# Happy path often: 200, x-router-decision: usage_bypass (coincidence).
# On Anthropic weekly-limit 429: client sees 429; Messages would have rerouted.

Billing note (minor aside)

Path	Ledger when Anthropic+subscription serves under UsageBypass
Messages bypassToAnthropic success	No row (early return before emitBilling)
OpenAI success	emitBilling with SubscriptionServed → delta=0 + notional

Not an overcharge; different trail. Port should decide whether OpenAI should match Messages’ skip-ledger or keep the notional row.

Not the bug / not overlapping

Issue / PR	Why distinct
#761 / #762	Preemptive exhaustion suppress before first credential inject — different region
#771 / #773	Post-dispatch baseline + subscription-credit retry — #773 does not wire bypassToAnthropic; at best a Weave-key retry after OAuth 429, not UsageBypass’s clear-flag + routeFor
#516 / #729	Telemetry visibility for usage-bypass turns — metrics, not OpenAI consumer

Fix

Wire Anthropic UsageBypass consumption into ProxyOpenAIChatCompletion after runTurnLoop, mirroring Messages:

1. If routeRes.UsageBypass && Decision.Provider == Anthropic, run an OpenAI-wire equivalent of bypassToAnthropic (or shared helper with format-specific emit/flush — same design call as ProxyOpenAIChatCompletion has no post-dispatch failover: OSS-outage baseline retry and subscription-credit retry both absent #771/fix(proxy): port post-dispatch failover to OpenAI chat completions #773).
2. On errBypassRetryable: subscription-only → refuse; else clear UsageBypass, refresh subsidy, load switch history, routeFor, continue normal OpenAI dispatch.
3. Align billing/telemetry with the chosen Messages parity (skip ledger vs delta=0 notional; router.usage_bypass span).

Acceptance criteria

• OpenAI test mirroring TestUsageBypass_BelowThreshold_SkipsScorer: usage_bypass decision, requested model, scorer=0 (happy path — may already pass via coincidence; keep as regression).
• OpenAI test mirroring TestProxyMessages_BypassWeeklyLimit_FallsBackToRoutedDispatch: buffered Anthropic 429 must not reach the client; scorer runs; routed decision/model served.
• Subscription-only + bypass retryable → refuse (Messages parity), if that guard is ported.
• Billing/telemetry behavior explicitly chosen and tested.
• No change to ProxyMessages bypass behavior; no change when the gate is off.

Duplicate check

Searched open PRs/issues for UsageBypass / bypassToAnthropic / “usage bypass” on OpenAI consumption: none implement this. Closest mentions are deferrals in #762 and #773.