[BUG] GetChatCompletionsStreamAsyncSample code not working #40710
Labels
Client
This issue points to a problem in the data-plane of the library.
customer-reported
Issues that are reported by GitHub users external to the Azure organization.
needs-team-triage
Workflow: This issue needs the team to triage.
OpenAI
question
The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Describe the bug
I am not able to get to a working asynchronous chat stream example using GPT-4 through Azure SDK for Java.
I started from the example code provided here
https://github.com/Azure/azure-sdk-for-java/blob/azure-ai-openai_1.0.0-beta.9/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamAsyncSample.java
but it does not work out of the box.
When I executed that code, providing the necessary parameters, I got an error.
Exception or Stack Trace
There was an error getting chat completions.java.lang.NullPointerException: The mapper [it.celi.dialog.GetChat$$Lambda$143/0x00000008002a0000] returned a null value.
To Reproduce
Just execute the example code provided here:
https://github.com/Azure/azure-sdk-for-java/blob/azure-ai-openai_1.0.0-beta.9/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamAsyncSample.java
Code Snippet
I tried to fix the issue by modifying these lines:
if (CoreUtils.isNullOrEmpty(chatCompletions.getChoices())) { return null; }
this way
if (CoreUtils.isNullOrEmpty(chatCompletions.getChoices())) { return ""; }
This change allowed me to execute the code without NPE errors, but the final behavior is different from what I expected it to be.
Expected behavior
I expected to get the response token by token in an asynchronous way, but I instead received only
Role = assistant
Then I increased the timeout from 10 to 60 seconds
TimeUnit.SECONDS.sleep(60);
This way I got:
Role = assistant
immediately...
the full response after more or less 15 seconds.So the streaming is not working in an asynchronous way.
Screenshots
Setup (please complete the following information):
Additional context
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
The text was updated successfully, but these errors were encountered: