Python: Return images in tool functions?

[SuperKenVery]

Is it possible to return images in tool functions? The docs didn't mention this.

Also, the FunctionResultContent class annotated result as Any, which is not very helpful...

[markwallace-microsoft]

@SuperKenVery can you provide some more detail about what you are trying to do? Do you want to return a link to an image or the actual bytes?

[SuperKenVery]

Shouldn't both of these be supported? Just like user messages.

[markwallace-microsoft]

If can share what use case you are trying to solve this will help us to plan what needs to be done. Can you share details?

[SuperKenVery]

Yes of course! I'm making the LLM process a lot of articles, putting those reporting the same thing together. I would like to provide a tool that allows the model to read one of the articles, which may contain images.

[markwallace-microsoft]

You can send the image URL to the LLM using the UriContent type, this sample shows how to do this: https://github.com/microsoft/agent-framework/blob/main/dotnet/samples/GettingStarted/Agents/Agent_Step11_UsingImages/Program.cs

[SuperKenVery]

This doesn't seem to provide image from a tool. It's just a user message.

[markwallace-microsoft]

OK, let me experiment a bit I have a scenario I'd like to try.