fix(models): send AnyLLM Responses reasoning as a mapping - #4138
Merged
seratch merged 1 commit intoAug 3, 2026
Conversation
_to_dump_compatible only materializes lazy iterables, and a pydantic model iterates as key/value pairs, so ModelSettings.reasoning was serialized as a list of pairs. any-llm types ResponsesParams.reasoning as a mapping, so every AnyLLM Responses request configuring reasoning failed validation before reaching the provider. Dump the model instead.
seratch
approved these changes
Aug 3, 2026
seratch
enabled auto-merge (squash)
August 3, 2026 04:28
seratch
disabled auto-merge
August 3, 2026 04:29
seratch
enabled auto-merge (squash)
August 3, 2026 04:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AnyLLMModelon the Responses path sendsModelSettings.reasoningas a list of[key, value]pairs instead of a mapping, so every AnyLLM Responses request that configures reasoning fails with a pydanticValidationErrorbefore it reaches the provider._fetch_responses_responsebuilds the payload with:_to_dump_compatibleexists to materialize lazy iterables (#1683, #3700). It has noBaseModelbranch, and a pydantic model iscollections.abc.Iterable— it iterates as key/value pairs — so it falls through to the generic iterable branch:any-llm types
ResponsesParams.reasoningasdict[str, Any] | None, and_call_any_llm_responsesconstructs that model, so the request dies at validation:This is not a rare path.
_build_responses_transport_kwargsalways sets aUser-Agentheader, sotransport_kwargsis never empty and_call_any_llm_responsesalways takes theResponsesParamsbranch. Reasoning is therefore unusable on the AnyLLM Responses path for bothget_responseandstream_response, which share this payload builder. Present inv0.19.2and unchanged since the adapter landed in #2706.Minimal reproduction
No API key or network call — the failure happens while building the request. Requires the optional
any-llm-sdkextra.Current behavior:
ValidationError; the request never reaches the provider.Corrected behavior:
params.reasoning == {"effort": "low", "summary": "concise"}and the request is sent.Implementation
One call site, one line: dump the model instead of routing it through the iterable materializer.
ModelSettings.reasoningis typed and validated asReasoning | None(adictpassed by a user is coerced toReasoningat construction), somodel_dumpis the source-of-truth conversion.mode="json"matches the existingItemHelpers.copy_tool_call_callerconvention, andexclude_none=Truematches this adapter's own_sanitize_any_llm_responses_value, which stripsNonefrom every payload it hands to any-llm.Why this is minimal: the demonstrated defect is one call site misusing a lazy-iterable materializer for a pydantic model. Broadening
_to_dump_compatibleto special-caseBaseModelwould change tool, message, and replay serialization across the OpenAI Responses, Chat Completions, LiteLLM, run-state, and rollout paths with no demonstrated failure in any of them, so that is an explicit non-goal here.Test plan
Added to
tests/models/test_any_llm_model.py. Both new tests build a genuineany_llm.types.responses.ResponsesParams, so they assert against any-llm's real validator rather than a local stub, and theyimportorskip("any_llm")in line with the neighbouring tests.test_any_llm_responses_path_sends_reasoning_as_a_mapping[False|True]— parametrized over non-streaming and streaming, sinceget_responseandstream_responseshare_fetch_responses_response. Assertsparams.reasoning == {"effort": "low", "summary": "concise"}.test_any_llm_responses_path_omits_reasoning_when_unset— boundary:reasoning=Nonestill sendsNone. This one passes onmain; it guards against the fix regressing the unset case.Pre-fix baseline on
e943deda(fix reverted, tests kept):Post-fix:
Also run 3x with
-W error::RuntimeWarning(39 passed each time) to confirm the streamed case leaves no unclosed async generator; the streaming test explicitlyaclose()s the returned stream.Full stack per
AGENTS.md(.agents/skills/code-change-verification/scripts/run.sh):git diff --check: clean.Compatibility
No public API, signature, or field-order change. Only the AnyLLM Responses request payload is affected, and only for
reasoning. Chat Completions, LiteLLM, and OpenAI Responses adapters are untouched.Reasoning()with every field unset now serializes to{}instead of raising — an empty mapping is valid forResponsesParams.reasoning.Related but distinct
#2823 / #2822 also stemmed from a pydantic model being iterable, but in
_flatten_any_llm_reasoning_value— the response-side reasoning-text extraction on the chat-completions path. This is the request-side payload builder on the Responses path: different function, different direction, different failure mode. That fix is merged and unaffected here.Issue number
No GitHub issue. Self-identified defect with the live repro above, following the precedent in #3700.
Checks
.agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PR