Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming support, implementation of IAsyncEnumberalbe #378

Closed
ColinZeb opened this issue Apr 8, 2023 · 2 comments · Fixed by #829
Closed

Streaming support, implementation of IAsyncEnumberalbe #378

ColinZeb opened this issue Apr 8, 2023 · 2 comments · Fixed by #829
Assignees

Comments

@ColinZeb
Copy link

ColinZeb commented Apr 8, 2023

Because I see that Example App Chat Copilot's response is too slow, I would like to ask if Streaming will be implemented in the future.

Additional description:
Nuget package Azure.AI.OpenAI supports Streaming
Example:

Response<StreamingChatCompletions> response = await client.GetChatCompletionsStreamingAsync(
               deploymentOrModelName: model,
               chatCompletionsOptions);
using StreamingChatCompletions streamingChatCompletions = response.Value;

await foreach (StreamingChatChoice choice in streamingChatCompletions.GetChoicesStreaming())
{
   await foreach (ChatMessage message in choice.GetMessageStreaming())
   {
       Console.Write(message.Content);
   }
   Console.WriteLine("-------------");
}
@shawncal shawncal self-assigned this Apr 11, 2023
@microsoftShannon
Copy link
Contributor

We are working on this.

@dedalo
Copy link
Contributor

dedalo commented May 6, 2023

did not see this and created other option. link here and follow to close once the issue is solved.
#836

I'm using copilot with signalr to connect status change and streaming and the UX changes completely
SK_signalr_stream

@dedalo dedalo mentioned this issue May 6, 2023
shawncal added a commit that referenced this issue May 8, 2023
### Motivation and Context
Get streaming results from chat completion models using the
IAsyncEnumerable pattern.

Closes #378 
Resolves #378 

Added samples how to use those as well as how to implement your own
custom chat completion model using streaming capability.

Co-authored-by: dedalo <dedalo@gmail.com>
Co-authored-by: Shawn Callegari <36091529+shawncal@users.noreply.github.com>
dehoward pushed a commit to lemillermicrosoft/semantic-kernel that referenced this issue Jun 1, 2023
### Motivation and Context
Get streaming results from chat completion models using the
IAsyncEnumerable pattern.

Closes microsoft#378 
Resolves microsoft#378 

Added samples how to use those as well as how to implement your own
custom chat completion model using streaming capability.

Co-authored-by: dedalo <dedalo@gmail.com>
Co-authored-by: Shawn Callegari <36091529+shawncal@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants