Send ThinkingPart back for OpenAIChatModel models which require reasoning content

[aravindh28]

Description

See @DouweM's latest comments on #2040

Context:

We have a custom API that converts between OpenAI's /chat/completions endpoint format and Bedrock Converse API format. So that our users can use OpenAI-compatible syntax with Bedrock models. Our custom API also supports thinking, tool calling, and streaming.

With thinking + tool calling, the assistant response will look like the following:

{
    "role": "assistant",
    "content": "...",
    "tool_calls": [
        {
            "id": "...",
            "type": "function",
            "function": {
                "name": "...",
                "arguments": "..."
            }
        }
    ],
    "reasoning_content": "...",
    "reasoning_signature": "..."
}

where the reasoning_content and reasoning_signature is required by Bedrock Converse API for the next conversation turn. These fields are not part of OpenAI syntax but we added them to make Bedrock compatible.

However, intializing the model like this -

model = OpenAIModel(
    "us.anthropic.claude-sonnet-4-20250514-v1:0",
    provider=OpenAIProvider(
        base_url= END_POINT_URL,
        api_key= API_KEY,
    ),
)

Gives the following issue -

[ERROR] Exception in agent run: 400: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: messages.3.content.0.type: Expected thinking or redacted_thinking, but found text. When thinking is enabled, a final assistant message must start with a thinking block (preceeding the lastmost set of tool_use and tool_result blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable thinking. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking

@DouweM 's suggestion -

@aravindh28 OpenAIChatModel doesn't currently send thinking parts back:

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py

Lines 577 to 581 in 21c87a5

	elif isinstance(item, ThinkingPart):
	# NOTE: We don't send ThinkingPart to the providers yet. If you are unsatisfied with this,
	# please open an issue. The below code is the code to send thinking to the provider.
	# texts.append(f'<think>\n{item.content}\n</think>')
	pass

We could add an OpenAIModelProfile option to do so (similar to BedrockModelProfile.bedrock_send_back_thinking_parts), but we'd need to make sure we send them back in the same field where we received them -- i.e. reasoning_content for DeepSeek and your API, or in <think> tags for APIs that returned the thoughts embedded in text parts. We can likely track that on ThinkingPart.id.

I'm guessing it would not be an option for your Pydantic AI users to use the Bedrock Converse API directly, while users of other frameworks can use the Chat Completions-compatible API?

This is worth creating a new issue for, by the way.

Originally posted by @DouweM in #2040

References

#2040 (comment)
#2040 (comment)

[pydanticai-bot]

PydanticAI Github Bot Found 1 issues similar to this one:

1. "Anthropic extended thinking: invalid request error when using tools #2040" (95% similar)

[DouweM]

@aravindh28 Thanks for filing the issue. We may want to start sending back all thinking parts: #2594

[aravindh28]

@DouweM I tried to directly use BedrockConverseModel to see if i could circumvent this, but I got an error and I'm assuming it's also tied to #2594? Or is this separate to BedrockConverseModel since it just specifies to mention the thinking.signature field?

bedrock_client = boto3.client('bedrock-runtime', region_name='us-west-2')

settings = BedrockModelSettings(temperature=1,
                         max_tokens=50000,
                         #parallel_tool_calls= True,                  
    bedrock_additional_model_requests_fields={
        "thinking": {
            "type": "enabled",
            "budget_tokens": 4000
        }
    },
    extra_headers={"anthropic_beta": "Interleaved-thinking-2025-05-14"} # Uncomment this for interleaved thinking, need to test if this actually does something (since we had a API workaround)
)


model = BedrockConverseModel("us.anthropic.claude-sonnet-4-20250514-v1:0",
                             provider=BedrockProvider(bedrock_client=bedrock_client),
                             #settings=settings
                             )

agent = Agent(model,
              deps_type=Deps,
              retries= 3,
              instructions= INSTRUCTIONS,
              #system_prompt= ,
              model_settings= settings,
                
)

[ERROR] Exception in agent run: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: messages.1.content.0.thinking.signature: Field required

For the first conversation turn, this seems to work, but in the 2nd turn - it generates this issue

If this is a bug - should I create a new issue?

And just curious - do you know of any timeline of when #2594 will be implemented (possibly solving this too)?

I'm fairly a beginner in both agents/software engineering in general but I'm loving the library so far - planning to build more serious projects!

[DouweM]

@aravindh28 I hope to work on #2594 later this week.

As for that error, do you have some code that reproduces the issue? We have a test for thinking with Anthropic via Bedrock:

pydantic-ai/tests/models/test_bedrock.py

Lines 666 to 686 in 55ad5bd

	async def test_bedrock_anthropic_tool_with_thinking(allow_model_requests: None, bedrock_provider: BedrockProvider):
	"""When using thinking with tool calls in Anthropic, we need to send the thinking part back to the provider.
	This tests the issue raised in https://github.com/pydantic/pydantic-ai/issues/2453.
	"""
	m = BedrockConverseModel('us.anthropic.claude-3-7-sonnet-20250219-v1:0', provider=bedrock_provider)
	settings = BedrockModelSettings(
	bedrock_additional_model_requests_fields={'thinking': {'type': 'enabled', 'budget_tokens': 1024}},
	)
	agent = Agent(m, model_settings=settings)
	@agent.tool_plain
	async def get_user_country() -> str:
	return 'Mexico'
	result = await agent.run('What is the largest city in the user country?')
	assert result.output == snapshot("""\
	Based on your location in Mexico, the largest city is Mexico City (Ciudad de México). It's not only the capital but also the most populous city in Mexico with a metropolitan area population of over 21 million people, making it one of the largest urban agglomerations in the world.
	Mexico City is an important cultural, financial, and political center for the country and has a rich history dating back to the Aztec empire when it was known as Tenochtitlán.\
	""")

And the cassette shows the signature being received on the 1st turn:

pydantic-ai/tests/models/cassettes/test_bedrock/test_bedrock_anthropic_tool_with_thinking.yaml

Line 37 in 55ad5bd

signature: ErcBCkgIBhABGAIiQDYN+P1S3ACL3r24cAMKCrRNiuPPxvmT2uzREPLRyKEUXagRbXn97QCke6L7OEZvlh7NdA/MQNTwMZV8TuB4qPASDLNxYxDx1S3luCIfARoMIwLZXwhsvjjTN72XIjAJrEl5ryAvv6C1+6YMCPC73ffE+kgwB96IcZaOuDFQtyaoWwcFcDPBguM6YNp5e3cqHbDQ3QF5dR4PP5q+3K23pual3pUdT/0e7khyIxXkGAI=

And sent back on the 2nd turn:

pydantic-ai/tests/models/cassettes/test_bedrock/test_bedrock_anthropic_tool_with_thinking.yaml

Lines 61 to 65 in 55ad5bd

	body: '{"messages": [{"role": "user", "content": [{"text": "What is the largest city in the user country?"}]}, {"role":
	"assistant", "content": [{"reasoningContent": {"reasoningText": {"text": "The user is asking for the largest city in
	their country. To answer this, I first need to determine what country the user is from. I can use the `get_user_country`
	function to retrieve this information.\n\nOnce I have the user''s country, I can then provide information about the
	largest city in that country.", "signature": "ErcBCkgIBhABGAIiQDYN+P1S3ACL3r24cAMKCrRNiuPPxvmT2uzREPLRyKEUXagRbXn97QCke6L7OEZvlh7NdA/MQNTwMZV8TuB4qPASDLNxYxDx1S3luCIfARoMIwLZXwhsvjjTN72XIjAJrEl5ryAvv6C1+6YMCPC73ffE+kgwB96IcZaOuDFQtyaoWwcFcDPBguM6YNp5e3cqHbDQ3QF5dR4PP5q+3K23pual3pUdT/0e7khyIxXkGAI="}}},

[aravindh28]

@DouweM

from pydantic_ai import Agent
from pydantic_ai.models.bedrock import BedrockConverseModel, BedrockModelSettings
from pydantic_ai.providers.bedrock import BedrockProvider, BedrockModelProfile
import boto3

#from dotenv import load_dotenv()

bedrock_client = boto3.client('bedrock-runtime', region_name='us-west-2')

settings = BedrockModelSettings(temperature=1,
                         max_tokens=50000,
                         #parallel_tool_calls= True,                  
    bedrock_additional_model_requests_fields={
        "thinking": {
            "type": "enabled",
            "budget_tokens": 4000
        }
    },
    #extra_headers={"anthropic_beta": "Interleaved-thinking-2025-05-14"}
)

profile = BedrockModelProfile(
    bedrock_send_back_thinking_parts=True
)

model = BedrockConverseModel("us.anthropic.claude-sonnet-4-20250514-v1:0",
                             #profile= profile,
                             provider=BedrockProvider(bedrock_client=bedrock_client),
                             #settings=settings
                             )

agent = Agent(model,
              #deps_type=Deps,
              retries= 3,
              #instructions= "assistant",
              #system_prompt= ,
              model_settings= settings,
                
)

result = agent.to_cli_sync()

**Above script produces the following warning/error -

pydantic-ai ➤ hello
/opt/anaconda3/envs/mi-chatbot-env/lib/python3.13/site-packages/pydantic_ai/models/bedrock.py:644: UserWarning: Only text reasoning content is supported yet, but you got {'signature': 
'EosCCkgIBxABGAIqQEHCbEwvtVRBlO/52B+QoVQEUt6f/te2f7IKYhj141XTeZMMeKEpVLNR1yEGzvaEU3eJJcBXQshy6ayMP5oADkkSDDiLSXeeXXiju5ialBoMBhDR7bksvwtHTeCmIjA4njd+V+vPksJEdWyEtK9L156/kbk5kHgv7lkf2X1X6wrzFYsPCJwNutTkErnou0MqcWyWjW7QQjdQjwJ3I9hrQ/zXLyvbmitZAlpkxuRENFgfx+bOvrC7u5qmlEDdzo1p2d
dIaplLcUG537PKmuwSFkImneWV4nDrvVX7mIQWUH7IneowF3lpACOtmLsf0I3v46kLoV0oAa+9kOaGGIzjUcFhGAE='}. Please report this to the maintainers.
  warnings.warn(
Hello! How are you doing today? Is there anything I can help you with?                                                                                                                                                                                                             
pydantic-ai ➤ sup

ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: messages.1.content.0.thinking.signature: Field required
pydantic-ai ➤

The first turn shows a warning and the second turn throws the ValidationException

[DouweM]

@aravindh28 Thanks, I've fixed this in #2785.

[DouweM]

@aravindh28 In your custom Chat Completions-compatible API that sits in front of Bedrock, would it be an option to support the format suggested by OpenAI at https://cookbook.openai.com/articles/gpt-oss/handle-raw-cot, which references OpenRouter's https://openrouter.ai/docs/use-cases/reasoning-tokens#responses-api-shape with suggests a reasoning_details format? We're more likely to support that than your bespoke reasoning_signature.

[aravindh28]

@DouweM Sorry for the delayed response, Let me ask my team and get back to you. Our API is still in beta & we are building as we go, so we should be flexible.