What issue are you seeing?
Background memory stage-1 requests can fail with invalid_request_error: context_length_exceeded even though build_stage_one_input_message nominally caps the rollout at 7% of the active model0's effective input window.Observed in Codex Desktop on Windows. The failure is in the detached background request_kind=memory request; it does not block the foreground thread, but the failed stage-1 job eventually exhausts its retries and is not automatically retried after a client fix.For a 272,000-token model with a 9% effective window, the configured rollout budget is:text272000 * 0.95 * 0.70 = 180880 tokensHowever, this budget is passed to TruncationPolicy::Tokens(180880). The current implementation converts that token budget to a fixed 4-byte-per-token byte limit, so it can retain roughly 723,520 bytes of serialized rollout text. JSON, tool output, source code, and punctuation-heavy text can tokenize much more densely than 4 bytes/token. The final memory request can therefore exceed the actual context window and is rejected by the server.I observed 23 memory_stage1 jobs fail with context_length_exceeded; the affected jobs exhausted their retries.
What steps can reproduce the bug?
- Enable memory generation.
- Use a model with a 272,000-token context window and 95% effective window.
- Create an eligible rollout with sufficiently token-dense serialized content, for example JSON-heavy tool output, source code, or punctuation-heavy text.
- Allow the background memory stage-1 worker to process the rollout.
- Observe that the detached memory request can fail with
context_length_exceeded despite the nominal 70% rollout budget.
The implementation makes this deterministic in principle: TruncationPolicy::Tokens is an approximation rather than model-tokenizer accounting, and it keeps up to max_tokens * 4 UTF-8 bytes.
What is the expected behavior?
Memory stage-1 should never submit an input beyond the selected model's actual context window.
The client should tokenize the final rendered request with the selected model's tokenizer, accounting for the static prompt/template, structured request overhead, and reserved output tokens before sending it. If exact tokenization is unavailable, it should use a conservative byte cap plus a final request-size preflight, rather than assuming 4 bytes per token.ge-1 should never submit an input beyond the selected model's actual context window.The client should tokenize the final rendered request with the selected model's tokenizer, accounting for the static prompt/template, structured request overhead, and reserved output tokens before sending it.
Additional information
Relevant current code:
codex-rs/memories/write/src/prompts.rs: build_stage_one_input_message computes the 70% rollout budget and passes it to TruncationPolicy::Tokens.codex-rs/utils/string/src/truncate.rs: APPROX_BYTES_PER_TOKEN = 4; approx_bytes_for_tokens(tokens) returns tokens * 4.
A final preflight is also needed because the stage-1 input template and other request content are added after rollout truncation.
Related but distinct: #23129 reports missing global consolidation and only mentions some stage-1 context-window failures as an additional symptom.
What issue are you seeing?
Background memory stage-1 requests can fail with
invalid_request_error: context_length_exceededeven thoughbuild_stage_one_input_messagenominally caps the rollout at 7% of the active model0's effective input window.Observed in Codex Desktop on Windows. The failure is in the detached backgroundrequest_kind=memoryrequest; it does not block the foreground thread, but the failed stage-1 job eventually exhausts its retries and is not automatically retried after a client fix.For a 272,000-token model with a 9% effective window, the configured rollout budget is:text272000 * 0.95 * 0.70 = 180880 tokensHowever, this budget is passed toTruncationPolicy::Tokens(180880). The current implementation converts that token budget to a fixed 4-byte-per-token byte limit, so it can retain roughly 723,520 bytes of serialized rollout text. JSON, tool output, source code, and punctuation-heavy text can tokenize much more densely than 4 bytes/token. The final memory request can therefore exceed the actual context window and is rejected by the server.I observed 23memory_stage1jobs fail withcontext_length_exceeded; the affected jobs exhausted their retries.What steps can reproduce the bug?
context_length_exceededdespite the nominal 70% rollout budget.The implementation makes this deterministic in principle:
TruncationPolicy::Tokensis an approximation rather than model-tokenizer accounting, and it keeps up tomax_tokens * 4UTF-8 bytes.What is the expected behavior?
Memory stage-1 should never submit an input beyond the selected model's actual context window.
The client should tokenize the final rendered request with the selected model's tokenizer, accounting for the static prompt/template, structured request overhead, and reserved output tokens before sending it. If exact tokenization is unavailable, it should use a conservative byte cap plus a final request-size preflight, rather than assuming 4 bytes per token.ge-1 should never submit an input beyond the selected model's actual context window.The client should tokenize the final rendered request with the selected model's tokenizer, accounting for the static prompt/template, structured request overhead, and reserved output tokens before sending it.
Additional information
Relevant current code:
codex-rs/memories/write/src/prompts.rs:build_stage_one_input_messagecomputes the 70% rollout budget and passes it toTruncationPolicy::Tokens.codex-rs/utils/string/src/truncate.rs:APPROX_BYTES_PER_TOKEN = 4;approx_bytes_for_tokens(tokens)returnstokens * 4.A final preflight is also needed because the stage-1 input template and other request content are added after rollout truncation.
Related but distinct: #23129 reports missing global consolidation and only mentions some stage-1 context-window failures as an additional symptom.