Skip to content

Conversation

@strands-agent
Copy link
Contributor

Description

Fixes #1485 - Cannot use structured_output with thinking enabled.

Problem

When using BedrockModel with thinking mode enabled via additional_request_fields, if structured_output fails to generate a tool call on the first attempt, the event loop forces tool_choice (set to {"any": {}}) to retry. However, Bedrock's API doesn't allow thinking mode to be enabled when tool_choice forces tool use, resulting in:

ValidationException: The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use.

Solution

This PR modifies the _format_request method in BedrockModel to temporarily remove the thinking configuration from additionalModelRequestFields when tool_choice is forcing tool use (i.e., not {"auto": {}}).

Key changes:

  1. Detect when tool_choice is forcing a tool (not auto and not None)
  2. Create a copy of additional_request_fields without the thinking key when forcing
  3. Preserve thinking mode for all normal agent interactions (when tool_choice is auto or None)
  4. Preserve all other additional_request_fields when removing thinking

Testing

Added comprehensive unit tests in test_bedrock_thinking_structured_output.py covering:

  • ✅ Thinking removed when forcing tool use ({"any": {}})
  • ✅ Thinking preserved with auto tool choice ({"auto": {}})
  • ✅ Thinking preserved when no tool choice (None)
  • ✅ Other additional fields preserved when thinking is removed
  • ✅ Models without thinking config work normally

Behavior

Before: Agent crashes with ValidationException when structured_output + thinking enabled
After: Agent gracefully retries structured_output by temporarily disabling thinking for that specific retry

Related Issues

Fixes #1485

Type of Change

  • Bug fix

Testing

  • Added unit tests
  • Logic verified with inline tests

Checklist

  • I have read the CONTRIBUTING document
  • I have added tests that prove my fix is effective
  • My changes generate no new warnings
  • This is a non-breaking change (thinking mode still works for normal interactions)

When structured_output fails to generate a tool call and the event loop
forces tool_choice (using 'any' or specific tool), Bedrock's API raises:
'Thinking may not be enabled when tool_choice forces tool use.'

This fix temporarily removes the 'thinking' configuration from
additionalModelRequestFields when tool_choice is not 'auto', allowing
structured_output to work correctly with thinking-enabled models.

The thinking mode is only disabled for the forced tool call retry,
preserving it for normal agent interactions.

Fixes strands-agents#1485
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Cannot use structured_output with thinking enabled

1 participant