Skip to content

Echo reasoning items + prompt_cache_key for codex-oauth#325

Merged
edwin-zvs merged 1 commit into
mainfrom
zarvis-prompt-cache-key
Jun 1, 2026
Merged

Echo reasoning items + prompt_cache_key for codex-oauth#325
edwin-zvs merged 1 commit into
mainfrom
zarvis-prompt-cache-key

Conversation

@edwin-zvs

Copy link
Copy Markdown
Contributor

What

Makes the codex-oauth provider carry the model's reasoning items across turns (gpt-5 Responses API), matching the Codex CLI:

  • request include: ["reasoning.encrypted_content"];
  • capture reasoning items from the stream (response.output_item.done, type reasoning) — id + encrypted_content + summary;
  • store them in conversation history via a new Content::Reasoning(ReasoningItem) (carried on ProviderTurn.reasoning_items);
  • replay them in subsequent requests in their original position.

Also sets a stable per-session prompt_cache_key (= session id).

The openai / anthropic / ollama providers ignore the new Content::Reasoning variant.

Why

Two related gaps found while benchmarking on DeepSWE:

  1. Reasoning continuity. zarvis streamed reasoning summaries to the UI but discarded the reasoning items, so the model lost its chain-of-thought between turns and "re-thought" each round.
  2. Prompt caching. zarvis's prompt-cache hit-rate was ~15–45% (erratic) vs the Codex CLI's ~97% on the same backend.

Honest scope / results

  • Verified end-to-end on a DeepSWE task: the backend accepts the echoed items (no 400) and the task passed (reward 1.0 on that trial).
  • Caching is not fully fixed by this change — hit-rate stayed ~18% with occasional 89–95% spikes. The residual gap appears to be transport: the Codex CLI uses a persistent WebSocket for the Responses API (codex-rs/core/src/client.rs caches a "Responses WebSocket connection"), while zarvis uses stateless HTTP POST + SSE; the prompt cache is likely connection-scoped. That's a separate, larger change.
  • This PR's value is reasoning continuity (and laying the groundwork for caching), not a caching win on its own.

Test plan

  • cargo build + cargo test -p agentd-adapter-zarvis → 138 passed, 0 failed.
  • Ran a DeepSWE trial with this build (Codex OAuth, gpt-5.5): accepted, solved, no errors.

🤖 Generated with Claude Code

Carry the model's reasoning across turns on the gpt-5 Responses API: request
`include: ["reasoning.encrypted_content"]`, capture reasoning items from the
stream (response.output_item.done), store them in history as a new
`Content::Reasoning`, and replay them in later requests with their id +
encrypted_content — matching the Codex CLI. This restores reasoning
continuity between turns. Also set a stable per-session `prompt_cache_key`.

New `Content::Reasoning(ReasoningItem)` + `ProviderTurn.reasoning_items`;
the openai/anthropic/ollama providers ignore the variant.
@edwin-zvs
edwin-zvs merged commit 63359bd into main Jun 1, 2026
1 check passed
@edwin-zvs
edwin-zvs deleted the zarvis-prompt-cache-key branch June 1, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant