.Net: Add File Id Reference Handling to Experimental Agent Package so that User Messages can be related to file Ids in OpenAI Storage #5110
Conversation
The `AddUserMessageAsync` and `InvokeAsync` methods in both `ChatThread.cs` and `IAgentThread.cs` have been modified to accept an array of file IDs. This update allows users to reference up to 10 files in their message. Additionally, an unused `AddUserMessageAsync` method without the file ID parameter has been removed from `ChatThread.cs` for cleaner code.
A new optional parameter, fileIds, has been added to the InvokeAsync method in IAgentExtensions and the corresponding private AskAsync method in Agent.cs. This array can handle up to 10 file ids to reference for the message. Changes were applied where these methods are called, incorporating the fileIds parameter.
github-actions
bot
changed the title
@microsoft-github-policy-service agree [company="Wet Ink Corporation"] |
|
@microsoft-github-policy-service agree [company="Wet Ink Corporation"] |
|
@microsoft-github-policy-service agree |
|
@Ryan-Britton - This looks good. Can you also look at this change I published last week: #5113? (I think these are related) (Updated: Actually, I got approval and merged to main. Updated your change and reviewing shortly.) |
|
they're definitely related - looking at your changes, I see that you're using the file_id paramter for file references on chat messages in #5113 - but in your use case it seems to be looking specifically at images (where I was using the file_ids parameter to refer to any file type in OpenAI storage as a context for the chat). My thought during the quick review was that the naming of some of your changes to align it "image types" specifically, may obfuscate the fact that file_ids can point at non-image types as well? |
|
Made a couple small tweaks, including adding a mode to Can you take a look? |
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Noticed that agent examples failing due to resourcing and configuration delta. As part of this discovered agent as plugin regression from community contribution. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> - Include agent templates as embedded resources - Switch from `TestConfiguration.AzureOpenAI.DeploymentName` to `TestConfiguration.AzureOpenAI.ChatDeploymentName` in `Example72_AgentCollaboration` - Remove `fileIds` from `Agent.AskAsync` (not utilized and resulting in failed run creation) - #5110 - Eliminated user-message duplication - Added comments to `ChatMessageType`  ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
Motivation and Context
added the ability to reference file ids when creating a user message in the experimental agents package
Description
added a nullable array of file ids as a parameter on the AddUserMessageAsync and InvokeAsync methods so that it did not break existing tests/examples. Added the file_ids property to the POST payload to the OpenAI Assistants API.
Contribution Checklist