Support models that return output tool args as `{"response': "<JSON string>"}` #2836

shaheerzaman · 2025-09-08T18:11:19Z

This PR fixes #2829

Summary

Problem: Bedrock sometimes returns output tool args with the payload under response as a JSON string (e.g., '"[...]"'), causing Pydantic to error (“Input should be a valid list/object”).
Root Cause: output_type schemas that are not plain objects (e.g., list[T], tuples, unions) are wrapped under an outer_typed_dict_key (response). When the nested value is a stringified JSON, the validator sees a str instead of the expected list/object.
Fix: Pre-normalize output tool args by JSON-decoding the nested response value if it’s a string before running Pydantic validation.

Changes

pydantic_ai_slim/pydantic_ai/_tool_manager.py
In _call_tool, when tool.tool_def.outer_typed_dict_key is set:
If call.args is a JSON string: parse top-level, then json.loads the nested response if it’s a string, and validate via validate_python.
If call.args is a dict: attempt json.loads on response when it’s a string, then validate.
Graceful fallback to existing validation paths if decoding fails.

Impact

Stabilizes output_type=list[T] (and tuples/unions) with Bedrock by preventing spurious validation errors.
Backwards compatible and scoped to output tools only; function tools unaffected.
If decoding fails or payload is malformed, existing validation errors still surface clearly.

Notes for Reviewers

Applies only when outer_typed_dict_key is present (e.g., non-object output schemas).
No API changes or doc updates required.

keep upto date

DouweM · 2025-09-08T19:49:37Z

@shaheerzaman Thanks! I think we can implement this more cleanly using https://docs.pydantic.dev/2.0/usage/types/json/.

Specifically, I think we can change this section:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_output.py

Lines 631 to 642 in 96d895d

    
               self.outer_typed_dict_key = 'response' 
        
               response_data_typed_dict = TypedDict(  # noqa: UP013 
        
                   'response_data_typed_dict', 
        
                   {'response': cast(type[OutputDataT], output)},  # pyright: ignore[reportInvalidTypeForm] 
        
               ) 
        
               type_adapter = TypeAdapter(response_data_typed_dict) 
        
           # Really a PluggableSchemaValidator, but it's API-compatible 
        
           self.validator = cast(SchemaValidator, type_adapter.validator) 
        
           json_schema = _utils.check_object_json_schema( 
        
               type_adapter.json_schema(schema_generator=GenerateToolJsonSchema) 
        
           )

We can use the existing type adapter for the JSON schema generation, which should be strict, but for the self.validator, use a more lenient schema that's effectively response: OutputDataT | Json[OutputDataT], so that it will also correctly validate a JSON string.

Can you give that a try please? Let me know if you'd like any additional pointers.

shaheerzaman · 2025-09-09T08:49:59Z

@shaheerzaman Thanks! I think we can implement this more cleanly using https://docs.pydantic.dev/2.0/usage/types/json/.

Specifically, I think we can change this section:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_output.py

Lines 631 to 642 in 96d895d

self.outer_typed_dict_key = 'response'

response_data_typed_dict = TypedDict( # noqa: UP013

'response_data_typed_dict',

{'response': cast(type[OutputDataT], output)}, # pyright: ignore[reportInvalidTypeForm]

)

type_adapter = TypeAdapter(response_data_typed_dict)

# Really a PluggableSchemaValidator, but it's API-compatible

self.validator = cast(SchemaValidator, type_adapter.validator)

json_schema = _utils.check_object_json_schema(

type_adapter.json_schema(schema_generator=GenerateToolJsonSchema)

)

We can use the existing type adapter for the JSON schema generation, which should be strict, but for the self.validator, use a more lenient schema that's effectively response: OutputDataT | Json[OutputDataT], so that it will also correctly validate a JSON string.

Can you give that a try please? Let me know if you'd like any additional pointers.

@DouweM I have made the changes as you suggested.

DouweM · 2025-09-09T15:04:24Z

pydantic_ai_slim/pydantic_ai/_tool_manager.py


        return tool_result
+
+    def _validate_tool_args(


@shaheerzaman I don't think we need these changes anymore now that we use Json[]

DouweM · 2025-09-09T15:06:30Z

pydantic_ai_slim/pydantic_ai/_output.py

            json_schema['description'] = self._function_schema.description
        else:
            type_adapter: TypeAdapter[Any]
+            schema_validator: SchemaValidator


Can we use the PluggableSchemaValidator type here, so that we only need to do the cast once on the self.validator = line?

pydantic_ai_slim/pydantic_ai/_output.py

DouweM · 2025-09-09T15:07:15Z

pydantic_ai_slim/pydantic_ai/_output.py

+                    'response_validation_typed_dict',
+                    {'response': cast(type[OutputDataT], output) | Json[cast(type[OutputDataT], output)]},  # pyright: ignore[reportInvalidTypeForm]
+                )
+                validation_type_adapter: TypeAdapter[Any] = TypeAdapter(response_validation_typed_dict)


Why do we need the type hint here?

DouweM · 2025-09-09T15:07:32Z

pydantic_ai_slim/pydantic_ai/_output.py

+                    {'response': cast(type[OutputDataT], output) | Json[cast(type[OutputDataT], output)]},  # pyright: ignore[reportInvalidTypeForm]
+                )
+                validation_type_adapter: TypeAdapter[Any] = TypeAdapter(response_validation_typed_dict)
+                schema_validator = cast(SchemaValidator, validation_type_adapter.validator)


Since we don't use validation_type_adapter elsewhere, we can inline it here

AGENTS.md

DouweM · 2025-09-09T15:08:21Z

pydantic_ai_slim/pydantic_ai/_output.py

                type_adapter = TypeAdapter(response_data_typed_dict)

+                # More lenient validator: allow either the native type or a JSON string containing it
+                # i.e., response: OutputDataT | Json[OutputDataT]


Let's mention the specific model that does this wrong

We should also add a test for the case where the model returns nested JSON and verifies that it's parsed correctly

mentioned bedrock model in the comment and added a test file

… SchemaValidatorProt-based validator path

AGENTS.md

DouweM · 2025-09-09T18:23:00Z

tests/test_output_json_union.py

I was hoping to see a single test like this one:

pydantic-ai/tests/test_agent.py

Lines 66 to 75 in 0047a68

def test_result_tuple():

def return_tuple(_: list[ModelMessage], info: AgentInfo) -> ModelResponse:

assert info.output_tools is not None

args_json = '{"response": ["foo", "bar"]}'

return ModelResponse(parts=[ToolCallPart(info.output_tools[0].name, args_json)])

agent = Agent(FunctionModel(return_tuple), output_type=tuple[str, str])

result = agent.run_sync('Hello')

assert result.output == ('foo', 'bar')

Can you create one like it, for the output type you ran into issues with, with the nested-JSON API response that has been failing but will now work?

DouweM · 2025-09-09T18:23:34Z

pydantic_ai_slim/pydantic_ai/_output.py

+                # More lenient validator: allow either the native type or a JSON string containing it
+                # i.e., response: OutputDataT | Json[OutputDataT] for some models that respond well to
+                # instructions. in this case BedrockConverseModel - 'us.meta.llama3-2-11b-instruct-v1:0'model
+                response_data_typed_dict = TypedDict(  # noqa: UP013 # pyright: ignore[reportGeneralTypeIssues]


Can you please show me the error we were seeing without this ignore?

TypedDict must be assigned to a variable named "response_validation_typed_dict"PylancereportGeneralTypeIssues
Convert response_data_typed_dict from TypedDict functional to class syntaxRuffUP013
(variable) response_data_typed_dict: type[response_validation_typed_dict]

@shaheerzaman Ah interesting. Then maybe we can use the original variable name for the new type adapter, that will actually be used for validation, and have a new variable name for the one only used for JSON.

pydantic_ai_slim/pydantic_ai/_output.py

DouweM · 2025-09-10T17:20:04Z

@shaheerzaman Thanks Shaheer!

shaheerzaman added 4 commits September 8, 2025 23:26

adds decode stringified JSON under output tool’s response

d4b5659

fixed linting errors

361c504

fixed ruff formatting

ed933f7

Merge branch 'main' into fix/bedrock-error

144d457

keep upto date

DouweM mentioned this pull request Sep 8, 2025

Tool Output with Bedrock struggles with lists of objects #2829

Closed

2 tasks

DouweM self-assigned this Sep 8, 2025

DouweM added the awaiting author revision label Sep 8, 2025

shaheerzaman added 3 commits September 9, 2025 12:19

more test coverage

bafb729

fixed pre-commit errors

06c9984

Keep strict JSON schema generation using the original TypedDict

a46870b

DouweM requested changes Sep 9, 2025

View reviewed changes

shaheerzaman added 2 commits September 9, 2025 23:46

added focused tests that exercise the new Json union handling and the…

ad211a5

… SchemaValidatorProt-based validator path

removed AGENTS.md from the PR

f953a2d

DouweM requested changes Sep 9, 2025

View reviewed changes

shaheerzaman and others added 5 commits September 9, 2025 23:58

added AGENTS.md back

50691cf

added a new test to check nested strigified json output

354df50

fixed typechecking errors

8a61359

Update pydantic_ai_slim/pydantic_ai/_output.py

5a42b43

Cleanup

01ac951

DouweM changed the title ~~Decode stringified JSON for output tool args (Bedrock lists/objects)~~ Support models that return output tool args as {"response': "<JSON string>"} Sep 10, 2025

DouweM enabled auto-merge (squash) September 10, 2025 17:17

DouweM merged commit 95b80fa into pydantic:main Sep 10, 2025
29 checks passed

	def test_result_tuple():
	def return_tuple(_: list[ModelMessage], info: AgentInfo) -> ModelResponse:
	assert info.output_tools is not None
	args_json = '{"response": ["foo", "bar"]}'
	return ModelResponse(parts=[ToolCallPart(info.output_tools[0].name, args_json)])

	agent = Agent(FunctionModel(return_tuple), output_type=tuple[str, str])

	result = agent.run_sync('Hello')
	assert result.output == ('foo', 'bar')

Support models that return output tool args as {"response': "<JSON string>"} #2836

Support models that return output tool args as {"response': "<JSON string>"} #2836

Uh oh!

Conversation

shaheerzaman commented Sep 8, 2025 • edited by DouweM Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DouweM commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shaheerzaman commented Sep 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

DouweM commented Sep 10, 2025

Uh oh!

Uh oh!

Support models that return output tool args as `{"response': "<JSON string>"}` #2836

Support models that return output tool args as `{"response': "<JSON string>"}` #2836

shaheerzaman commented Sep 8, 2025 •

edited by DouweM

Loading

DouweM commented Sep 8, 2025 •

edited

Loading