-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net Hugging Face TGI Chat Completion Message API Support #5785
.Net Hugging Face TGI Chat Completion Message API Support #5785
Conversation
…nto features/hugging-face-tgi-chat-completion
…nto features/hugging-face-tgi-chat-completion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nit comments :)
...t/src/Connectors/Connectors.HuggingFace.UnitTests/Services/HuggingFaceChatCompletionTests.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceOpenAIClient.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceOpenAIClient.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceOpenAIClient.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Models/TextGenerationMetadata.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Models/TextGenerationStreamMetadata.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceOpenAIClient.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceClient.cs
Outdated
Show resolved
Hide resolved
Consider creating also generic SseData class |
dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/TextEmbeddingResponse.cs
Outdated
Show resolved
Hide resolved
What's the plan for automatic function calling? Is that going to be in this PR or a follow-up? |
@RogerBarreto for reference, here is guidance from HF for tool calling support as part of the Messages API https://huggingface.co/docs/text-generation-inference/conceptual/guidance#tools-and-functions- |
@luisquintanilla, this PR already adds function calling support... but manual. It's the automated part I was asking about, i.e. where SK itself reads and responds to tool invocation requests rather than passing those back to the caller. |
@stephentoub @luisquintanilla The tool calling behavior will be added as soon we have the abstractions in place. Currently the only piece added to this PR is the |
…nto features/hugging-face-tgi-chat-completion
dotnet/src/Connectors/Connectors.HuggingFace.UnitTests/Connectors.HuggingFace.UnitTests.csproj
Outdated
Show resolved
Hide resolved
...t/src/Connectors/Connectors.HuggingFace.UnitTests/Services/HuggingFaceChatCompletionTests.cs
Outdated
Show resolved
Hide resolved
...nectors/Connectors.HuggingFace.UnitTests/Services/HuggingFaceStreamingChatCompletionTests.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceClient.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/TextGenerationRequest.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceKernelBuilderExtensions.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceServiceCollectionExtensions.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Services/HuggingFaceChatCompletionService.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Services/HuggingFaceChatCompletionService.cs
Outdated
Show resolved
Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs
Show resolved
Hide resolved
…nto features/hugging-face-tgi-chat-completion
…5785) ### Motivation and Context Closes microsoft#5403 1. Adding support to Chat Completion for TGI (Text Generation Inference) Deployment. 3. Adding Missing UnitTests for Streaming and Non Streaming scenarios (Text/Chat Completion) 4. Update Metadata + Usage Details for hugging face clients. ### Contribution Checklist - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
Motivation and Context
Closes #5403
Contribution Checklist