JSON serialization for tool outputs in RealtimeSession

[StromFLIX]

Current Behavior

Currently in the RealtimeSession event handler, tool outputs are coerced into strings before being passed along:

https://github.com/openai/openai-agents-python/blob/main/src/agents/realtime/session.py#L413

result = await func_tool.on_invoke_tool(tool_context, event.arguments)
 
await self._model.send_event(
    RealtimeModelSendToolOutput(
        tool_call=event, output=str(result), start_response=True
    )
)

This approach works for simple outputs, but it causes problems when tools return structured data such as dictionaries or Pydantic models. For example, consuming the event on the client side results in non–machine-readable strings like:

data=TestModel(name='Sample Test')"

Solution

Introduce a serialization strategy that preserves structured outputs when possible:

result = await func_tool.on_invoke_tool(tool_context, event.arguments)
 
if isinstance(result, dict):
    output = json.dumps(result)
elif hasattr(result, "model_dump_json"):
    output = result.model_dump_json()
elif hasattr(result, "dict"):
    output = json.dumps(result.dict())
else:
    output = str(result)
 
await self._model.send_event(
    RealtimeModelSendToolOutput(
        tool_call=event, output=output, start_response=True
    )
)

If you think this makes sense, i can open up a PR for this.

[seratch]

Thanks for sharing the idea. Having the above custom code doesn't sound scalable to me. Also, this shouldn't be specific realtime agents. Thus, we enhance it in some way, it should be consistently done across this project.

[StromFLIX]

@seratch if you have any suggestions on how to do this, i would like to look into it 👍we are working on this right now and this is blocking our evaluations. We can always work around this, but it would help to have it in a common machine readable format.