Recognize gateway-routed models in telemetry and render scope - #151
Merged
Conversation
teamchong
force-pushed
the
gateway-model-telemetry
branch
from
July 26, 2026 13:30
8112075 to
a82bdc8
Compare
Requests arriving through a gateway's OpenAI-compatible route were invisible to the dashboard: /compat/chat/completions and /grok/chat/completions matched no known wire shape, so the proxy skipped the parse block, and the emitted event carried no model and defaulted to anthropic accounting. 70 compat and 306 grok events rendered with Model, Sent as, As text, Sent and Saved blank. Four changes, all in service of that: - Recognize one leading provider segment on OpenAI chat/responses paths, so gateway-prefixed requests are read as the OpenAI shapes they are. Routing is untouched: a provider-prefixed path still forwards to the passthrough upstream. - Resolve profiles by full id then bare final segment, so moonshotai/kimi-k3 and workers-ai/@cf/moonshotai/kimi-k3 resolve as kimi-k3. Vendor segments pick an upstream, not a geometry. - On POST paths pxpipe does not transform, read the top-level model field so the row is labeled. Buffering is gated to JSON under a size cap, since that route also carries uploads; when the gate declines, the model stays blank as before and the body streams through unchanged. - Stop refusing ids that name no measured family. They now resolve to DEFAULT_GPT_PROFILE, gated with OpenAI tile math as an approximation. The last one widens render scope, so state its limits plainly. PXPIPE_MODELS becomes the only gate for these models: listing one asserts it is worth imaging. Savings for them is an approximation, not a measurement, and nothing verifies the model accepts images at all — a text-only model 400s on the first imaged request. Deliberately not adding a vision-capability gate: the scope list is hand-written, so a capability check re-asks a question the operator already answered, and the failure is a loud 400 rather than silent corruption. Docs record the tradeoff.
teamchong
force-pushed
the
gateway-model-telemetry
branch
from
July 26, 2026 14:02
a82bdc8 to
af0004f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
70 compat and 306 grok events rendered with Model, Sent as, As text, Sent and Saved blank.
/compat/chat/completionsmatched no known wire shape, so the parse block was skipped and the event defaulted to anthropic accounting.Non-obvious mechanism:
One leading provider segment is accepted on OpenAI chat/responses paths. Routing is unchanged — a prefixed path still forwards to passthrough.
Profiles resolve by full id, then bare final segment. Vendor picks an upstream, not a geometry.
On POST paths pxpipe does not transform, the top-level
modelfield is read for labeling only. Gated to JSON under a size cap, since that route also carries uploads. Gate declines -> model blank as before, body streams unchanged.Ids naming no measured family resolve to
DEFAULT_GPT_PROFILEinstead of being refused.Trust boundary moved. Matching was hardcoded to
google/; it now strips everything before the last/.Intended:
PXPIPE_MODELSstays the gate on which models get imaged, and the prefix is not part of that decision. Ifmodelis attacker-controlled, this widens billing misattribution surface.For unmeasured families, savings is OpenAI tile math, not a measurement, and nothing checks the model accepts images — a text-only model 400s on the first imaged request. No vision gate by design: the scope list is hand-written, so a capability check re-asks a question the operator already answered, and the failure is a loud 400 rather than silent corruption.