Allow tool functions to return metadata (not just a string) #2098
Description
Background
Currently, tool functions are assumed to return a single string or a ToolOutputText.
However, in some situations, it is helpful to pass structured information to the application layer, for example, metadata (database IDs, error codes, etc.). Current solutions usually require writing the metadata inside the string in a parsable format (e.g., JSON and XML), which might not be ideal if we don't want to share some of the information with the model.
Proposal
Extend the function call output message dictionary to accept a metadata field:
@function_tool
def some_function():
...
return "output", {"some_metadata": true}
Which results in the following message.
{"type": "function_call_output", "output": "Some output content", "call_id": "call_....", "metadata": {}}
Potential Problems
I'm aware that creating this new abstraction around the OpenAI message types might be unwanted,
And in that case, I would like to understand if something like this is a possibility or if anyone has a better solution even.