Skip to content

Fix StackOverflowError in netty sync server for large InputStream responses#5159

Merged
adamw merged 3 commits intomasterfrom
fix/5150-inputstream-stackoverflow
Apr 3, 2026
Merged

Fix StackOverflowError in netty sync server for large InputStream responses#5159
adamw merged 3 commits intomasterfrom
fix/5150-inputstream-stackoverflow

Conversation

@adamw
Copy link
Copy Markdown
Member

@adamw adamw commented Apr 3, 2026

Summary

  • Fixes [BUG] inputStreamBody can stack overflow with large streams #5150
  • Add InputStreamSyncPublisher for the direct-style/Identity netty server that uses a while loop instead of recursive monadic calls, preventing StackOverflowError when streaming large response bodies
  • The existing async InputStreamPublisher (used by Future/cats-effect/ZIO backends) is unchanged
  • NettySyncToResponseBody no longer depends on RunAsync or MonadError

Test plan

  • Unit test (InputStreamSyncPublisherTest): verifies no stack overflow with 1MB stream, 1KB chunks, on a thread with 256KB stack
  • Integration test in NettySyncServerTest: verifies 10MB InputStream response is fully received through the netty sync server

🤖 Generated with Claude Code

…ponses (#5150)

Add InputStreamSyncPublisher for the direct-style/Identity server that
uses a while loop instead of recursive monadic calls, preventing stack
overflow when streaming large response bodies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 3, 2026 13:05
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes StackOverflowError when streaming large InputStream response bodies via the Netty sync / Identity server by introducing a non-recursive, synchronous publisher and wiring it into the sync response-body conversion.

Changes:

  • Added InputStreamSyncPublisher which streams InputStreamRange using iterative loops instead of recursive monadic chaining.
  • Updated Netty sync response-body wiring to use InputStreamSyncPublisher (and removed unused RunAsync/MonadError dependency from NettySyncToResponseBody).
  • Added unit + integration tests covering large InputStream responses and regression for #5150.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
server/netty-server/sync/src/test/scala/sttp/tapir/server/netty/sync/NettySyncServerTest.scala Adds integration regression test for large InputStream responses on the sync server.
server/netty-server/sync/src/test/scala/sttp/tapir/server/netty/sync/internal/reactivestreams/InputStreamSyncPublisherTest.scala Adds unit test to ensure no stack overflow with large streams on a small-stack thread.
server/netty-server/sync/src/main/scala/sttp/tapir/server/netty/sync/NettySyncServerInterpreter.scala Wires updated NettySyncToResponseBody constructor.
server/netty-server/sync/src/main/scala/sttp/tapir/server/netty/sync/internal/reactivestreams/InputStreamSyncPublisher.scala New synchronous reactive-streams publisher implementation using a while-loop approach.
server/netty-server/sync/src/main/scala/sttp/tapir/server/netty/sync/internal/NettySyncToResponseBody.scala Switches InputStream wrapping from InputStreamPublisher[Id] to InputStreamSyncPublisher.
server/netty-server/src/main/scala/sttp/tapir/server/netty/internal/reactivestreams/InputStreamPublisher.scala Clarifies non-Id usage and improves stream-close error handling (suppressed exceptions).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +32 to +37
override def request(n: Long): Unit = {
if (n <= 0) subscriber.onError(new IllegalArgumentException("§3.9: n must be greater than 0"))
else {
demand.addAndGet(n)
readNextChunkIfNeeded()
}
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

request(n) signals onError for non-positive n, but the subscription remains active (demand can still be increased and stream may stay open). Reactive Streams §3.9 expects the subscription to be considered cancelled after this error; consider setting isCompleted and closing the stream (e.g., via cancel()) immediately after calling onError to prevent further reads/leaks.

Copilot uses AI. Check for mistakes.
Comment on lines +33 to +37
if (n <= 0) subscriber.onError(new IllegalArgumentException("§3.9: n must be greater than 0"))
else {
demand.addAndGet(n)
readNextChunkIfNeeded()
}
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

demand.addAndGet(n) can overflow if request is called multiple times with large values (e.g., Long.MaxValue), potentially turning demand negative and stalling the stream. Consider using saturating addition (cap at Long.MaxValue) and treating Long.MaxValue as unbounded demand (avoid decrementing in that mode).

Copilot uses AI. Check for mistakes.
}

private def readNextChunkIfNeeded(): Unit = {
// Everything here is be synchronous and blocking - which is against the reactive streams spec
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in comment: "Everything here is be synchronous and blocking" → should be "is" (or rephrase).

Suggested change
// Everything here is be synchronous and blocking - which is against the reactive streams spec
// Everything here is synchronous and blocking, which is against the reactive streams spec

Copilot uses AI. Check for mistakes.

publisher.subscribe(new Subscriber[HttpContent] {
override def onSubscribe(s: Subscription): Unit = {
// Request all chunks at once - this triggers the recursive loop
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says requesting all chunks "triggers the recursive loop", but this test targets the new non-recursive implementation. Consider updating the comment to clarify that this request pattern previously triggered recursion/StackOverflowError, and now validates the while-loop behavior.

Suggested change
// Request all chunks at once - this triggers the recursive loop
// Request all chunks at once - this previously triggered recursive processing
// and StackOverflowError, and now validates the while-loop-based implementation.

Copilot uses AI. Check for mistakes.
stackSize
)
thread.start()
thread.join(30000)
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thread.join(30000), if the thread is still alive, the test will fail due to result being null but the worker thread will keep running and may interfere with the rest of the suite. Consider explicitly failing when thread.isAlive after join, and interrupting/cleaning up the thread.

Suggested change
thread.join(30000)
thread.join(30000)
if (thread.isAlive) {
thread.interrupt()
thread.join(1000)
fail("Worker thread did not complete within 30000 ms")
}

Copilot uses AI. Check for mistakes.
adamw and others added 2 commits April 3, 2026 15:16
…meout handling

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…StreamSyncPublisher

- Cancel subscription after onError for invalid request(n <= 0) per §3.9
- Use saturating addition for demand to prevent Long overflow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@adamw adamw merged commit bded4ac into master Apr 3, 2026
22 checks passed
@adamw adamw deleted the fix/5150-inputstream-stackoverflow branch April 3, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] inputStreamBody can stack overflow with large streams

2 participants