Events coming from event_stream_handler are missing some tool output

[nicolas-chaulet]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

I see there are a few convo about streams and tool output but I could not find that particular issue.

I am relying on the event_stream_handler and when a tool generates an image only the ToolReturnPart(tool_name='generate_image', content='See file f7d259',...) makes its way to the stream handler when ideally the user part that contains the actual file would come through as well.

I think those lines are the issue:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py

Lines 907 to 911 in 5e596f1

	tool_parts_by_index[index] = tool_part
	if tool_user_part:
	user_parts_by_index[index] = tool_user_part
	return _messages.FunctionToolResultEvent(tool_part)

Example Code

import asyncio
import json

from pydantic_ai import Agent
from pydantic_ai import ImageUrl
from pydantic_ai.messages import ModelMessage
from pydantic_core import to_jsonable_python


def create_image() -> ImageUrl:
    return ImageUrl(url="https://yavuzceliker.github.io/sample-images/image-1021.jpg")


agent = Agent(model="openai:gpt-4.1", tools=[create_image])


if __name__ == "__main__":
    async def event_stream_handler(_context, event_stream) -> None:
        async for event in event_stream:
            print(f"Event: {event}")  # noqa: T201

    async def main() -> None:
        messages: list[ModelMessage] = []
        while True:
            input_text = input("You: ")
            agent_run = await agent.run(
                input_text, message_history=messages, event_stream_handler=event_stream_handler
            )
            print(f"Assistant final answer: {agent_run.output}")  # noqa: T201
            new_messages = agent_run.new_messages()
            print(f"New messages: {json.dumps(to_jsonable_python(new_messages), indent=2)}")  # noqa: T201

            messages.extend(new_messages)

    asyncio.run(main())

"""
Relevent event streams:

Event: FunctionToolCallEvent(part=ToolCallPart(tool_name='create_image', args='{}', tool_call_id='call_LNMnysikk3UI0XU3Oe4Qg4iR'))
Event: FunctionToolResultEvent(result=ToolReturnPart(tool_name='create_image', content='See file f5a895', tool_call_id='call_LNMnysikk3UI0XU3Oe4Qg4iR', timestamp=datetime.datetime(2025, 10, 2, 16, 42, 26, 395446, tzinfo=datetime.timezone.utc)))
"""

Python, Pydantic AI & LLM client version

pydantic-ai=1.0.13

[DouweM]

@nicolas-chaulet Good catch. Would it work for you if I added an extra field on the FunctionToolResultEvent to hold the str | Sequence[UserContent] returned alongside the tool result, either because it returned a file, or because it returned a ToolReturn object with content? The metadata from ToolReturn is already on ToolReturnPart.metadata.

Another option (perhaps better) would be to add a field on ToolReturnPart to hold these extra parts, instead of putting them into the ModelRequest.parts directly, but that'd be backward-incompatible.

[nicolas-chaulet]

Yes that would be ideal. I actually found a workaround with metadata but I don't think it is really meant for that, ideally I should be able to rely on pydantic's internal mapping from a tool that returns a file to this ToolReturn object that removes the file from the actual return value. I am guessing you are doing this mapping because some providers don't accept files in tool returns?

def create_image() -> ToolReturn:
    asset = ImageUrl(url="https://yavuzceliker.github.io/sample-images/image-1021.jpg")
    return ToolReturn(
        return_value=f"See file {asset.identifier}",
        content=[
            f"This is file {asset.identifier}",
            ImageUrl(url="https://yavuzceliker.github.io/sample-images/image-1021.jpg"),
        ],
        metadata={
            "asset": ImageUrl(url="https://yavuzceliker.github.io/sample-images/image-1021.jpg")
        },
    )

[DouweM]

I am guessing you are doing this mapping because some providers don't accept files in tool returns?

@nicolas-chaulet No provider accepts files in tool returns actually, just text (which could be JSON), while files always need to be sent in special API parts.