Why codexSessionAffinityTtlMs is 0 by default?

[powellnorma]

Are there any downsides to enabling it?

Keeps one Codex conversation on the same account for this many seconds

What if that account has reached its quota limit? I assume then another account is used instead?

Is it correct that enabling this would improve the (upstream) cache-efficiency (and thereby potentially reducing our token cost)?

[diegosouzapw]

Hey @powellnorma! Short answer: it is 0 by default because sticky sessions trade a bit of load-balancing for cache/continuity, and the team left it opt-in rather than forcing that trade-off on everyone.

How it works: src/sse/services/auth.ts -- the affinity key comes from a header (x-codex-session-id / x-session-id) or fields in the body (conversation_id, session_id, prompt_cache_key) via extractSessionAffinityKey (auth.ts:280-305). The TTL is resolved per request (auth.ts:1489-1498) and the actual account pick happens in selectSessionAffinityConnection (auth.ts:728-777), backed by a SQLite key_value table (src/lib/db/sessionAccountAffinity.ts).

On quota: you are right, it does NOT get stuck on the exhausted account. Cooldown/rate-limit filtering (auth.ts:1318-1401) and quota-exhaustion filtering (isAccountQuotaExhausted, auth.ts:1452-1483) both run before selectSessionAffinityConnection is even called (auth.ts:1501) -- so the affixed account is already excluded from the candidate list if it is unhealthy. Inside selectSessionAffinityConnection, if the previously-affixed account is not in that list anymore, the stale affinity row is deleted (auth.ts:755) and a new one is created against the next available account (LRU order, auth.ts:762-765). The same explicit-cleanup pattern shows up on stream failure in src/sse/handlers/chat.ts:1358-1370 and :1408-1419. So enabling it does not risk getting your session "stuck" -- only that one session's account mapping gets discarded and rebuilt.

Real downside: it concentrates one conversation's traffic onto a single account, which burns that account's quota window faster than round-robin would (less spreading across your pool during the TTL window). There is also a small extra DB read/write per request to look up and refresh the affinity row.

On cache-efficiency: I cannot confirm that part with certainty. The key extraction does accept prompt_cache_key as one of the affinity signals (auth.ts:299), which suggests it was designed with OpenAI's cache-key concept in mind, but I do not have documentation or a benchmark proving that keeping the same account improves upstream cache-hit rate -- that is ultimately controlled by OpenAI's infrastructure, not something OmniRoute can guarantee. If you do enable it and notice a token-cost difference, that data point would actually be useful to report back here.