Skip to content

Chat Completions converter can send empty tool output for non-text results #3310

@Aphroq

Description

@Aphroq

Please read this first

  • Have you read the docs? Yes. This concerns internal Chat Completions conversion of tool outputs.
  • Have you searched for related issues? Yes.

Describe the bug

Converter.items_to_messages() can convert a function_call_output whose output contains only non-text content into a Chat Completions tool message with content: [] when preserve_tool_output_all_content=False.

That default is appropriate for direct OpenAI Chat Completions because non-text tool result content is not supported there, but silently sending an empty tool result hides the actual tool output from the model and from the caller.

Impact:

  • A tool can return meaningful image/audio/file output, but the next model turn receives an empty tool message.
  • Debugging is misleading because the tool did produce output; the converter dropped all representable content.
  • Compatible providers that opt into preserve_tool_output_all_content=True are already handled, so the remaining issue is the default non-preserving path.

Relevant edge cases considered:

  • String tool outputs remain unchanged.
  • Text-only list outputs remain unchanged.
  • Mixed text and non-text list outputs still keep the text part by default.
  • Non-text-only list outputs should fail explicitly instead of becoming content: [].
  • preserve_tool_output_all_content=True should continue preserving images/audio/files for compatible providers.

Debug information

  • Agents SDK version: upstream main at bc3607ba
  • Python version: Python 3.12.1

Repro steps

Run this minimal script from a repository checkout.

from agents.models.chatcmpl_converter import Converter


messages = Converter.items_to_messages(
    [
        {
            "type": "function_call_output",
            "call_id": "call_image",
            "output": [
                {
                    "type": "input_image",
                    "image_url": "https://example.com/image.png",
                }
            ],
        }
    ]
)

print(messages)

Actual result on upstream main at bc3607ba:

[{'role': 'tool', 'tool_call_id': 'call_image', 'content': []}]

Expected behavior

When non-text tool output cannot be represented for the default Chat Completions path, the converter should raise an explicit error instead of sending an empty tool message. Compatible providers should still be able to opt into preserving all content with preserve_tool_output_all_content=True.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions