Skip to content

perf: RemoteTaskActionClient streaming deserialization. - #20005

Merged
gianm merged 1 commit into
apache:masterfrom
gianm:perf-rtac-stream
Aug 14, 2026
Merged

perf: RemoteTaskActionClient streaming deserialization.#20005
gianm merged 1 commit into
apache:masterfrom
gianm:perf-rtac-stream

Conversation

@gianm

@gianm gianm commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

This patch switches RemoteTaskActionClient to use InputStreamResponseHandler rather than BytesFullResponseHandler, which enables deserialization to run without buffering the entire response. This is useful for actions with potentially large responses, such as RetrieveUsedSegmentsAction.

This patch switches RemoteTaskActionClient to use InputStreamResponseHandler
rather than BytesFullResponseHandler, which enables deserialization
to run without buffering the entire response. This is useful for actions
with potentially large responses, such as RetrieveUsedSegmentsAction.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity Findings
P0 0
P1 0
P2 1
P3 0
Total 1
Severity Findings
P0 0
P1 0
P2 1
P3 0
Total 1

Reviewed 2 of 2 changed files.


This is an automated review by Codex GPT-5.6-Luna(max)

response.get("result"),
taskAction.getReturnTypeReference()
);
new InputStreamResponseHandler()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Streaming failures bypass ServiceClient retries

InputStreamResponseHandler completes the service future before the response body has finished. If the body fails mid-stream, deserialization receives an IOException after the future has already completed, so the configured RPC retry policy is bypassed; the previous BytesFullResponseHandler let ServiceClient retry these failures. Complete the response only after the stream is fully consumed, or preserve the retryable failure path.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is OK, for two reasons.

  1. Most failures are going to happen before the response body starts being sent. Generally once that starts, most failures we see would be timeouts or OOMs due to too much data being sent, which isn't generally going to be fixed by a retry anyway. There is precedent for not retrying these failures, especially in CoordinatorClientImpl and OverlordClientImpl.

  2. The alternative is the prior state (not releasing bytes to the JSON parser until they have been fully buffered). In theory it would be possible to incrementally parse the bytes inside an HttpResponseHandler that returns an unfinished response, but I don't see a clean way to do this with Jackson. The JsonParser methods that drive parsing all expect to block while they fetch all their input, which would necessitate another thread, which I'd like to avoid. NonBlockingJsonParser is a thing that does exist in Jackson, but it doesn't seem to work with jackson-databind.

@gianm
gianm merged commit 856f766 into apache:master Aug 14, 2026
44 of 45 checks passed
@gianm
gianm deleted the perf-rtac-stream branch August 14, 2026 17:15
@github-actions github-actions Bot added this to the 39.0.0 milestone Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants