feat(stream-testkit): enable virtual threads for JDK 21+ nightly builds#2880
Closed
He-Pin wants to merge 3 commits intoapache:mainfrom
Closed
feat(stream-testkit): enable virtual threads for JDK 21+ nightly builds#2880He-Pin wants to merge 3 commits intoapache:mainfrom
He-Pin wants to merge 3 commits intoapache:mainfrom
Conversation
Motivation: Sink.asPublisher(fanout = true) still depended on the legacy actor-backed FanoutProcessorImpl runtime, which kept issue apache#2860 blocked on old processor infrastructure and implementation-bound tests. Modification: Route FanoutPublisherSink through a new FanoutPublisherBridgeStage, delete the legacy FanoutProcessor implementation, replace the old actor-bound spec with FanoutPublisherBehaviorSpec, and add the matching 2.0.x MiMa excludes for the removed binary-visible classes. Result: The fanout publisher path now runs on a dedicated GraphStage bridge with the existing terminal-signal contract preserved, broader behavior coverage added, and compile/MiMa/TCK validation passing. References: apache#2860 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation: GitHub's scalafmt diff-ref job flagged FanoutPublisherBridgeStage.scala on PR apache#2874. Modification: Applied scalafmt formatting to FanoutPublisherBridgeStage.scala without changing behavior. Result: The fanout bridge file now matches the repository's CI formatting expectations. References: apache#2860 apache#2874 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation: JDK-8300995 causes ForkJoinPool compensation-thread starvation, leading to sporadic test timeouts when actors block on reply futures. Virtual threads (JDK 21+) bypass this limitation by unmounting when blocking, providing equivalent or better performance. See issue apache#2870. Modification: 1. stream-testkit/reference.conf: Add PEKKO_VIRTUALIZE_DISPATCHER env var support with safe fallback to `off` for stable local testing 2. .github/workflows/nightly-builds.yml: Set env var conditionally for JDK 21+, with detailed comments explaining TIMEFACTOR logic 3. CONTRIBUTING.md: Document virtual thread testing with cleanup guidance and safe one-liner for developers 4. FanoutPublisherBridgeStage.scala: Improve callback management with atomic reference and clarifying comments Result: - Virtual threads automatically enabled on JDK 21+ in nightly builds - Local tests safely default to virtual threads OFF for stability - Nightly test reliability improved; timeouts addressed - Binary compatibility maintained; MiMa checks pass References: Upstream PR apache#2872: Virtual threads support Issue apache#2870: Compensation-thread starvation investigation Issue apache#2573: JDK 25 ForkJoinPool scheduling regression PR apache#2871: ForkJoinPool minimum-runnable tuning (JDK < 21 fallback) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Enable virtual thread support in stream test dispatcher for JDK 21+ nightly builds to address ForkJoinPool compensation-thread starvation (JDK-8300995).
Motivation
JDK-8300995 causes ForkJoinPool compensation-thread starvation, leading to sporadic test timeouts when actors block on reply futures. Virtual threads (JDK 21+) bypass this limitation by unmounting when blocking, providing equivalent or better performance.
Related Issues:
Related PRs:
Changes
PEKKO_VIRTUALIZE_DISPATCHERenv var support with safe fallback toofffor stable local testingResults
Verification
which is now Apache licensed