Skip to content

proxy: usage-bypass turns skip fireTelemetry and invisible to dashboard metrics and cost charts #516

Description

@rohith500

Expected — telemetry row written for the bypass turn

SELECT request_id, decision_model, decision_provider, decision_reason
FROM router.model_router_request_telemetry
ORDER BY created_at DESC LIMIT 3;

-- Expected:
--  <uuid> | claude-haiku-4-5              | anthropic | usage_bypass
--  <uuid> | gemini-3.1-flash-lite-preview | google    | classifier_hard_pin
--  <uuid> | gemini-3.1-flash-lite-preview | google    | classifier_hard_pin

Actual — bypass turn produces no row

SELECT request_id, decision_model, decision_provider, decision_reason
FROM router.model_router_request_telemetry
ORDER BY created_at DESC LIMIT 3;

-- Actual:
--  c922abf6 | gemini-3.1-flash-lite-preview | google | classifier_hard_pin
--  6542d281 | gemini-3.1-flash-lite-preview | google | classifier_hard_pin
--  1170d0d1 | gemini-3.1-flash-lite-preview | google | classifier_hard_pin
--
-- The usage_bypass turn is completely absent.

Root cause

ProxyMessages (internal/proxy/service.go:1563):

if routeRes.UsageBypass {
    return s.bypassToAnthropic(...)  // ← returns at line 1564
}

// ...614 lines later — never reached for bypass turns...
s.fireTelemetry(InsertTelemetryParams{...})  // line 2178

bypassToAnthropic (internal/proxy/usage_bypass.go) calls only:

  • otel.Record (line 160) — writes to the OTLP span pipeline
  • otel.Flush (line 173)

It never calls fireTelemetry or s.telemetry.InsertRequestTelemetry. The dashboard reads from s.telemetry.GetTelemetryRows (backed by router_request_telemetry), not the OTLP span pipeline — so bypass turns produce no DB row and never appear in any metrics query.

Importantly, in the normal path fireTelemetry is called regardless of upstream outcome (200, 401, 429, 500) — the UpstreamStatusCode is recorded in the row. On the bypass path, even a successful 200 response produces no telemetry row.

Evidence the omission is unintentional

The bypassToAnthropic comment explicitly lists what it skips:

"deliberately skips the cluster scorer, planner, session pin,
semantic cache, AND billing"

Telemetry is absent from this list. Billing is intentionally skipped because the customer's plan already pays for the tokens. Telemetry is just observability — it has no cost impact and should still be recorded.

No test in usage_bypass_test.go asserts presence or absence of telemetry rows. All 8 tests assert only response headers and dispatch
counts.

Dashboard impact

For installations with bypass enabled, every dashboard widget that reads from router_request_telemetry shows incorrect data:

Widget Bypass off (correct) Bypass on (broken)
Requests count 100 40 (bypass turns missing)
Cost saved Based on 100 turns Based on 40 turns only
Substitution rate Accurate Understated (bypass turns not counted)
Models served gemini + claude-haiku gemini only
Provider split pie Google 60%, Anthropic 40% Google 100%

The bypass gate was built to serve Claude subscription turns on Anthropic, but those are the exact turns that go missing from the dashboard. An operator enabling bypass would see their Anthropic traffic disappear from the model distribution and have no way to verify the gate is working.

Scope

Anthropic-surface only. usageBypassEngaged gates on m.PrimaryProvider() == providers.ProviderAnthropic, so ProxyOpenAIChatCompletion is not affected.

Fix

Call fireTelemetry inside bypassToAnthropic before returning, with DecisionReason: "usage_bypass" and SubscriptionServed: true. Token counts are available from the upstream response (same as the normal path).
The notional cost should be recorded as a shadow trail (matching billing behaviour in #513) with actual debit = 0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions