Skip to content

v0.2.6 — aliases on the Responses path

Choose a tag to compare

View all tags
@thezillo thezillo released this 07 Sep 17:25
· 26 commits to main since this release
v0.2.6
42d5f70

The model alias map now runs on /v1/responses — the endpoint Codex actually uses.

The gap

[defaults.model_aliases] existed to stop a bare gpt-6 / gpt-5.6 from reaching the upstream, where a ChatGPT account answers:

400 {"detail":"The 'gpt-6' model is not supported when using Codex with a ChatGPT account."}

But it only ever ran on the /v1/chat/completions translation. Codex speaks wire_api = "responses", so the one client the map exists for never reached it: the bare name went upstream verbatim, 400'd, and the request fell through to a paid fallback provider. Nothing failed loudly — the fallback answered, so a session looked healthy while being billed off-subscription. Worse, /v1/models advertises those bare names, so a client picking one out of the proxy's own list was the one being diverted.

--model gpt-5.6 against this proxy now lands on the subscription pool.

How the rewrite stays a passthrough

Three steps, and the ordering is the design:

  1. Read the model with a one-field struct. serde walks the JSON but builds no tree — every other field is skipped as IgnoredAny. A body with no alias to apply returns here and is forwarded byte-for-byte, never parsed into a Value.
  2. Splice the new name into the raw bytes. Not premature optimisation: bodies reach server.max_body_bytes (16 MiB), and an aliased Codex client pays this on every request of a session whose history keeps growing. Measured on a 16 MiB body — the obvious Value round-trip costs +64 MB and 75 ms, the splice +17 MB and 17 ms. Deployments size this proxy as "baseline + the bodies currently in flight"; the splice keeps that arithmetic true.
  3. Verify by re-reading the model, and fall back to the Value round-trip when it doesn't match. Correctness rests on serde, never on the byte scanner.

That third step is what makes step 2 safe rather than clever. A body can defeat the scan — a nested "model" member appearing first, a JSON-escaped model name, even a member whose name ends in an escaped quote ({"a\"model": …}, whose trailing bytes are literally "model"). Each is now a test; each fails closed into the tree rewrite and forwards a correct body.

An unrecognized id is deliberately not replaced by defaults.model the way resolve_model does on the other path. Here the id is the client's own, and rejecting it is the upstream's call, not the proxy's.

Breaking, if you configured a fallback model_map

model_map keys are matched against the model actually sent upstream. On /v1/responses that used to be whatever the client sent; it is now the aliased name, matching what /v1/chat/completions already did. A provider keyed by a bare name stops matching there — and an unmapped model makes the provider skipped, not degraded:

[fallback.model_map]
"gpt-5.6-sol" = "..."   # not "gpt-5.6"

The old wording in config.toml promised the previous behaviour; it and the README are corrected in this release.

Full changelog: v0.2.5...v0.2.6

Assets 2
Loading