[BUG] [Azure OpenAI] block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-2 #40898
Comments
github-actions
bot
added
customer-reported
github-actions
bot
removed
the
needs-triage
|
Hi @joshfree, thanks for picking this up so quickly. I have done a bit more investigation, this time using the Azure OpenAI SDK directly and removing the Spring AI library. @Bean
public OpenAIAsyncClient chatClient() {
return new OpenAIClientBuilder()
.credential(new AzureKeyCredential("my-az-open-ai-key"))
.endpoint("https://my-endpoint.openai.azure.com")
.buildAsyncClient();
}And this code in the controller @PostMapping(path = "stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ChatCompletions> stream(@RequestBody String input) {
List<ChatRequestMessage> chatMessages = new ArrayList<>();
chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."));
chatMessages.add(new ChatRequestUserMessage(input));
return chatClient.getChatCompletionsStream(
"gpt-4",
new ChatCompletionsOptions(chatMessages));
} |
|
Back into Spring AI and stepping through with the debugger, it calls Response<BinaryData> getChatCompletionsSync(@HostParam("endpoint") String var1, @QueryParam("api-version") String var2, @PathParam("deploymentId") String var3, @HeaderParam("accept") String var4, @BodyParam("application/json") BinaryData var5, RequestOptions var6, Context var7);This looks like a potential cause of the blocking issue. I would have expected it to call |
|
Duplicated this issue in Spring AI |
Describe the bug
In a Spring Boot WebFlux project, calling the stream method of Spring AI which then calls the Azure OpenAI SDK, it throws the exception
block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-2Exception or Stack Trace
To Reproduce
Create a new Spring Boot 3.3.1 project through initializer, add Spring AI and Azure Open AI dependencies and configure them in application.properties for autoconfiguration.
Code Snippet
Expected behavior
Azure OpenAI chat streams a response
Setup (please complete the following information):
Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
The text was updated successfully, but these errors were encountered: