Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net Enable Usage of Custom Compatible Chat Message API Endpoints with OpenAI Connector + Examples #4753

Merged
merged 37 commits into from
Apr 16, 2024

Conversation

RogerBarreto
Copy link
Member

@RogerBarreto RogerBarreto commented Jan 26, 2024

Motivation and Context

Resolves #5353

  • Allow usage of custom Message API (OpenAI ChatCompletion Standard) compliant endpoints with the OpenAI Connector.
  • Refactoring of OpenAI Models and Classes Structure
  • Adding Examples on using the current changes against LMStudio, Ollama and LocalAI Message APIs.

@RogerBarreto RogerBarreto requested a review from a team as a code owner January 26, 2024 10:11
@shawncal shawncal added the .NET Issue or Pull requests regarding .NET code label Jan 26, 2024
@RogerBarreto RogerBarreto self-assigned this Jan 26, 2024
@Krzysztof318
Copy link
Contributor

Krzysztof318 commented Jan 26, 2024

I have never used LM-studio, but what this has common with the openai implementation?

@stephentoub
Copy link
Member

I assume this is temporary and having someone implement their own handlers and extension methods and whatnot isn't actually our answer for this, right?

@RogerBarreto
Copy link
Member Author

As stated in the Connectors ADR, while the biggest benefit of LM Studio is mimicking OpenAI's Endpoints and Contracts no specific Connector for this platform is needed, unless we have a specific requirement that the OpenAI connector isn't enough for consuming LM Studio Http APIs, we will suggest using the OpenAI's Connector.

We will add an extra configuration option for the OpenAI connectors that won't require to manually define an Handler for it and investigate if the usage of the Endpoint configuration would alone suffice for this.

@Krzysztof318 Krzysztof318 mentioned this pull request Feb 9, 2024
4 tasks
@matthewbolanos
Copy link
Member

@RogerBarreto, we should have a better 1st class story for connecting to popular local deployment channels like LLM Studio and Ollama, but in the meantime, I think your sample would be valuable for people who need to customize the OpenAI API endpoint.

Could you update this PR to make it more "generic" (i.e., title the Kernel Syntax Example "custom OpenAI endpoint") so we can get this merged in?

@markwallace-microsoft markwallace-microsoft added the kernel Issues or pull requests impacting the core kernel label Apr 9, 2024
@RogerBarreto RogerBarreto changed the title .Net LMStudio SLM Small Language Models (Local Models) examples .Net Enable Usage of Custom Compatible Chat Message API Endpoints with OpenAI Connector + Examples Apr 10, 2024
@RogerBarreto RogerBarreto added this pull request to the merge queue Apr 16, 2024
Merged via the queue into microsoft:main with commit beef63c Apr 16, 2024
15 checks passed
@RogerBarreto RogerBarreto deleted the features/lmstudio-examples branch April 16, 2024 11:57
LudoCorporateShark pushed a commit to LudoCorporateShark/semantic-kernel that referenced this pull request Aug 25, 2024
…h OpenAI Connector + Examples (microsoft#4753)

### Motivation and Context

- Allow usage of custom Message API (OpenAI ChatCompletion Standard)
compliant endpoints with the OpenAI Connector.
- Refactoring of OpenAI Models and Classes Structure
- Adding Examples on using the current changes against `LMStudio`,
`Ollama` and `LocalAI` Message APIs.

---------

Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
Archived in project
8 participants