Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ChatCompletion streaming over Completions #96

Closed
aaronpowell opened this issue Oct 17, 2023 · 2 comments
Closed

Support ChatCompletion streaming over Completions #96

aaronpowell opened this issue Oct 17, 2023 · 2 comments

Comments

@aaronpowell
Copy link
Contributor

aaronpowell commented Oct 17, 2023

The OpenAI/AzureOpenAI TextGeneration classes use the legacy Completions API (deprecation from OpenAI, announcement and Azure model compatibility).

This poses a challenge when working with Azure OpenAI Service in particular as you are required to deploy a gpt-35-turbo model version of 0301, which is deprecated, but due to quota limits on standard accounts you are unlikely to be able to deploy that plus the embeddings model and a 0613 model for gpt-35-turbo for the application to use.

Having Semantic Memory move off the Completions API to ChatCompletions would unblock the usage in AOAI applications and ensure that applications aren't caught in the upcoming deprecations.

@aaronpowell
Copy link
Contributor Author

Ah, I just realised that I need to be using AzureOpenAIConfig.APITypes.ChatCompletion not AzureOpenAIConfig.APITypes.TextCompletion to go through the right codepath to use the new ChatCompletions endpoint, so this is probably not valid

@crickman crickman added the triage label Nov 7, 2023
@dluc
Copy link
Collaborator

dluc commented Jun 5, 2024

correct, in the settings you can set the APIType to TextCompletion or ChatCompletion, depending on the model used

@dluc dluc removed the triage label Jun 5, 2024
@microsoft microsoft locked and limited conversation to collaborators Jun 5, 2024
@dluc dluc converted this issue into discussion #642 Jun 5, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants