fix(bedrock): disable thinking mode when forcing tool_choice #1490
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes #1485 - Cannot use structured_output with thinking enabled.
Problem
When using BedrockModel with thinking mode enabled via
additional_request_fields, if structured_output fails to generate a tool call on the first attempt, the event loop forcestool_choice(set to{"any": {}}) to retry. However, Bedrock's API doesn't allow thinking mode to be enabled whentool_choiceforces tool use, resulting in:Solution
This PR modifies the
_format_requestmethod inBedrockModelto temporarily remove thethinkingconfiguration fromadditionalModelRequestFieldswhentool_choiceis forcing tool use (i.e., not{"auto": {}}).Key changes:
tool_choiceis forcing a tool (notautoand notNone)additional_request_fieldswithout thethinkingkey when forcingautoorNone)additional_request_fieldswhen removing thinkingTesting
Added comprehensive unit tests in
test_bedrock_thinking_structured_output.pycovering:{"any": {}}){"auto": {}})Behavior
Before: Agent crashes with ValidationException when structured_output + thinking enabled
After: Agent gracefully retries structured_output by temporarily disabling thinking for that specific retry
Related Issues
Fixes #1485
Type of Change
Testing
Checklist