Skip to content

Fix streaming APIs in Agents and Projects libraries#48832

Merged
srnagar merged 6 commits intoAzure:mainfrom
srnagar:stream-responses
Apr 16, 2026
Merged

Fix streaming APIs in Agents and Projects libraries#48832
srnagar merged 6 commits intoAzure:mainfrom
srnagar:stream-responses

Conversation

@srnagar
Copy link
Copy Markdown
Member

@srnagar srnagar commented Apr 16, 2026

This pull request introduces a new utility class, FluxInputStream, to bridge between Reactor's Flux<ByteBuffer> and Java's InputStream, and refactors HTTP response handling to use this class. This enables non-blocking, streaming response bodies while maintaining compatibility with existing interfaces. Additionally, the request context and threading behavior for HTTP calls are improved. Comprehensive tests for FluxInputStream are also added.

Core HTTP streaming improvements:

  • Added the new FluxInputStream class, which allows reading from a Flux<ByteBuffer> as an InputStream, handling synchronization, backpressure, and error propagation.
  • Updated AzureHttpResponseAdapter.body() in both azure-ai-agents and azure-ai-projects to use FluxInputStream instead of the previous .getBodyAsBinaryData().toStream(), preparing for future API changes and enabling true streaming. [1] [2]

Testing:

  • Added FluxInputStreamTests, a comprehensive test suite covering normal operation, empty buffers, and error propagation for the new stream class.

Threading and context improvements:

  • Ensured that HTTP response processing and continuations are moved off Netty/OkHttp I/O threads by using .publishOn(Schedulers.boundedElastic()), preventing blocking of event loop threads. [1] [2]
  • Changed the request context in HttpClientHelper to no longer eagerly read responses, improving streaming and resource usage.

Code cleanup:

  • Removed an unused logger field from AzureHttpResponseAdapter.

These changes collectively improve the efficiency, reliability, and test coverage of HTTP streaming in the SDK.

NOTE: FluxInputStream will be moved to azure-core and HttpResponse will be updated to provide API to get the InputStream directly. Agents and Projects will be updated to remove duplication when new version of azure-core is available.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to enable true streaming HTTP response bodies (Flux<ByteBuffer> → InputStream) in the azure-ai-agents and azure-ai-projects libraries while improving request execution threading behavior to avoid completing futures on I/O event loop threads.

Changes:

  • Added a FluxInputStream implementation (in both Agents and Projects) plus corresponding tests.
  • Updated OpenAI HTTP response adapters to return a streaming InputStream backed by the underlying Azure response body Flux.
  • Updated OpenAI HTTP client helper to avoid eager response reading and to complete requests on Schedulers.boundedElastic().

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
sdk/core/azure-core/src/main/java/com/azure/core/http/HttpResponse.java Adds a new getBodyStream() convenience API intended to expose streaming bodies as InputStream.
sdk/ai/azure-ai-projects/src/main/java/com/azure/ai/projects/implementation/http/FluxInputStream.java New Flux→InputStream bridge used by Projects response adapter.
sdk/ai/azure-ai-projects/src/test/java/com/azure/ai/projects/implementation/http/FluxInputStreamTests.java Tests for Projects FluxInputStream.
sdk/ai/azure-ai-projects/src/main/java/com/azure/ai/projects/implementation/http/AzureHttpResponseAdapter.java Switches Projects adapter body() to streaming via Flux-backed InputStream.
sdk/ai/azure-ai-projects/src/main/java/com/azure/ai/projects/implementation/http/HttpClientHelper.java Ensures completion/continuations don’t run on I/O threads; stops eager response reading.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/http/FluxInputStream.java New Flux→InputStream bridge used by Agents response adapter.
sdk/ai/azure-ai-agents/src/test/java/com/azure/ai/agents/implementation/http/FluxInputStreamTests.java Tests for Agents FluxInputStream.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/http/AzureHttpResponseAdapter.java Switches Agents adapter body() to streaming via Flux-backed InputStream.
sdk/ai/azure-ai-agents/src/main/java/com/azure/ai/agents/implementation/http/HttpClientHelper.java Ensures completion/continuations don’t run on I/O threads; stops eager response reading.

Comment thread sdk/core/azure-core/src/main/java/com/azure/core/http/HttpResponse.java Outdated
@srnagar srnagar enabled auto-merge (squash) April 16, 2026 21:09
@srnagar srnagar merged commit 7900738 into Azure:main Apr 16, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants