Skip to content

v0.3.0 — quota-aware pool, fallback guardrails

Choose a tag to compare

@thezillo thezillo released this 22 Sep 17:57
· 18 commits to main since this release

The paid fallback chain now runs only when the subscription pool really can't serve a request: quota exhaustion is tracked per account, unknown models and pool-side request errors no longer leak onto paid providers, and a Codex session stays on one account so its prompt cache survives.

Breaking changes

  • Unknown models are rejected by default. A model outside /v1/models and [models] extra gets a 400 model_not_found listing what is available, after alias resolution, before the pool or any paid provider sees it. Set reject_unknown = false to restore the old pass-through.
  • gpt-5.5 is no longer accepted. The ChatGPT-account upstream answers it with 400 "not supported", so its only effect was paid-fallback traffic. The sample config.toml also drops the gpt-4o/gpt-4.1 aliases: they would now be rejected. If your config maps retired names, add them to [models] extra or remove them.
  • Pool 4xx is relayed, not retried on the paid chain. A pool 4xx other than 401/403/429 (e.g. "model is not supported when using Codex with a ChatGPT account") goes back to the client. fallback_on_bad_request = true restores the old behavior.
  • All-accounts-down goes straight to the chain. With a [[fallback]] chain configured, a request arriving while every account is cooling down or quota-exhausted skips the pool, including after a transport failure. Previously the next request retried upstream.

Quota-aware pool (#1, #2)

A 429 whose body says usage_limit_reached is the 5-hour or weekly Codex quota, not a transient throttle. The account is marked quota-exhausted until the reset the 429 reports (resets_at, resets_in_seconds, per-window rate_limits or the x-codex-*-reset-after-seconds headers; the latest wins, clamped to 60 s – 8 days), separately from account_cooldown_secs. The client still receives the real upstream 429 byte for byte.

While any account is quota-exhausted, the proxy re-reads usage_path for it every quota_check_interval_secs and clears the state as soon as either window shows headroom, so an early or manual reset is picked up without waiting. Healthy accounts are never polled; a failed poll keeps the state (fail-open).

[upstream]
usage_path = "/wham/usage"
quota_check_interval_secs = 600   # 0 = polling off

Fallback guardrails (#3)

[models]
reject_unknown = true
extra = []
fallback_on_bad_request = false

# On a plain 429, retry the POOL with a lower model before paying the fallback.
# Defining the section replaces the built-in map; empty disables downgrades.
[models.downgrades]
"gpt-6-astra" = "gpt-5.6-sol"

A usage_limit_reached 429 is account-wide and skips downgrades. If every downgrade step fails, the paid chain runs with the original model.

Session-pinned pool accounts (#3)

Round-robin sent consecutive turns of one Codex session to different accounts, so each turn re-read the whole conversation on a cold prompt cache. A request carrying session-id (or thread-id) now starts its pool scan at a home account picked by a stable FNV-1a hash. Availability ordering is unchanged: a cooling or quota-held home hands the session to the next account until it recovers. Requests without these headers still round-robin.

Observability

New counters, so alerts can tell a quota outage from a dead model id:

  • codexproxy_failovers_total{client,model,reason,fallback} with reason such as quota_exhausted or cooling_down
  • codexproxy_model_downgrades_total{client,from,to,outcome}
  • codexproxy_rejected_requests_total{client,reason}

Image: ghcr.io/thezillo/codex-proxy:v0.3.0.

Full changelog: v0.2.7...v0.3.0