Skip to content

Responses adapter: reasoning continuity across mediated tool rounds (encrypted reasoning passthrough) #33

Description

@mostlydev

Follow-up to #31 / PR #32. The outbound Responses adapter is deliberately a compatibility layer, and for chat-shaped runners it is lossless in expressiveness — but not in model quality on multi-round tool use.

The limitation

The adapter sends store: false (preserving the chat/completions property that turns are not retained upstream) and drops reasoning output items when translating replies back to chat shape. Consequence: on every managed-mediation round, the model re-reasons from scratch — it never sees its own prior chain-of-thought. On exactly the models the adapter exists for (reasoning models), this costs quality and tokens on rounds ≥ 2.

Proposed fix

The Responses API supports stateless reasoning continuity without store: true:

  1. request include: ["reasoning.encrypted_content"];
  2. retain the encrypted reasoning items alongside the mediated round (the same hidden-state mechanism managedcontinuity uses for Anthropic thinking blocks);
  3. replay them as input items on the next round.

No plaintext reasoning crosses into agent-visible state; governance surfaces are unaffected because the encrypted items live only in the adapter's hidden replay path.

For the record: what the adapter intentionally does not cover

Scoping notes from review, so the next reader doesn't rediscover them:

  • Responses-native runners (e.g. Codex CLI speaks /v1/responses natively) still 404 at the mux. That — and only that — is the scenario that would justify a governed inbound /v1/responses route, with all governance surfaces re-proven for the second shape.
  • Server-side tools (web_search, code_interpreter, computer use) are inexpressible in chat shape and out of scope for the adapter.
  • True streaming: the adapter buffers and synthesizes SSE; time-to-first-token equals time-to-completion. Acceptable for tool-mediated turns, noticeable for long direct completions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions