fix: auto-tune ForkJoinPool minimum-runnable on JDK 25+#2890
Conversation
Motivation: JDK-8300995 / JDK-8321335 changed compensation-thread creation in ForkJoinPool asyncMode (FIFO) to be much more conservative. Pekko fork-join dispatchers using the prior default `minimum-runnable = 1` are then prone to starvation under blocking workloads on JDK 21+, which has shown up as flaky nightly runs (#2870) and is the root cause behind the workflow override added in #2889. Modification: * Introduce `ForkJoinExecutorConfigurator.resolveMinimumRunnable`, an internal helper that computes the effective `minimum-runnable` value from the configured value, the dispatcher parallelism, and the running JDK major version. A negative configured value (the new default `-1`) triggers the JDK-aware policy: on JDK 21+ the value becomes `min(8, max(1, parallelism / 2))`; on JDK < 21 it stays at `1`. Non-negative values are honoured verbatim, so explicit `0` still disables compensation entirely and explicit positive values (including `1`) keep their existing meaning. * Change `pekko.actor.default-dispatcher.fork-join-executor.minimum-runnable` in `reference.conf` to the sentinel `-1` and update the doc block to describe the new auto-selection rule. * Add `ForkJoinExecutorConfiguratorSpec` with three groups of assertions: (1) pure-function matrix on `resolveMinimumRunnable`; (2) directional checks asserting the auto policy strictly raises the value on JDK 21+ and never exceeds the documented cap of 8; (3) wiring integration that builds a `ForkJoinExecutorServiceFactory` from a real dispatcher config and verifies the resolved value reaches the factory (guarding against regressions of the resolver wiring). Result: Production users on JDK 21+ now benefit from the same starvation mitigation that #2889 bolted onto the nightly CI workflow. Source and binary compatibility are preserved (constructor defaults stay at `1`, no signature changes, no MiMa filter required). Users wanting to opt out can set `minimum-runnable = 1` (or any explicit value) to restore the previous behaviour.
|
@pjfanning I think this should be the only way to make JDK25 pass |
|
@He-Pin can you update the dispatcher docs? I'm also wary of changing the default Java 21 behaviour. We've got strong evidence from the nightly tests that Java 25 could really do with the change you are making here but Java 21 seems fine. |
|
Maybe it is best as the jdk issue affects Java 21 but just doesn't seem to have quite the same impact there |
* License header: replace abbreviated header on the new ForkJoinExecutorConfiguratorSpec with the canonical Apache 2.0 header used by other clean-room test files in the project (per pjfanning's review comment). * Narrow auto-policy scope from JDK 21+ to JDK 25+: nightly evidence shows the asyncMode (FIFO) compensation-thread regression (JDK-8300995 / JDK-8321335) surfaces most clearly on the JDK 25 line, while JDK 21 has been running fine on the legacy default of 1 for years. Keep the default unchanged on JDK 21 to avoid a silent behaviour change for users who are not affected. * Document the new auto-tuning behaviour in both docs/src/main/paradox/dispatchers.md (classic) and docs/src/main/paradox/typed/dispatchers.md, including the opt-out instructions. * Update reference.conf doc comment, configurator scaladoc, and the spec assertions / pending guards to reflect the JDK 25+ scope.
|
@pjfanning thanks for the review — addressed all three comments in 904eef2:
Local |
pjfanning
changed the title
2 participants
Summary
ForkJoinExecutorConfigurator.resolveMinimumRunnablethat auto-scales theminimum-runnablesetting on JDK 25+ (min(8, max(1, parallelism / 2))) while keeping the legacy value of1on older JDKs. Explicit user values (including0) are still honoured.reference.conffrom1to the sentinel-1(auto) and update the doc block.ForkJoinExecutorConfiguratorSpeccovering: pure-function matrix; directional checks (auto value strictly higher on JDK 25+, never exceeds documented cap of 8); wiring integration that proves the resolved value reaches theForkJoinExecutorServiceFactory.Context
JDK-8300995 / JDK-8321335 changed
ForkJoinPoolasyncMode (FIFO) compensation-thread creation to be much more conservative. Pekko fork-join dispatchers using the prior defaultminimum-runnable = 1are then prone to starvation under blocking workloads on JDK 21+, which is the root cause behind the workflow override added in #2889.That workflow override only protects nightly CI. This PR pushes the same protection into the library so production users on JDK 25+ get the mitigation automatically.
Behaviour matrix
minimum-runnable-1(default)1(legacy)-1(default)min(8, max(1, parallelism / 2))00(compensation disabled)NNEscape hatch for users on JDK 21+ who want to keep the prior behaviour: set
pekko.actor.default-dispatcher.fork-join-executor.minimum-runnable = 1(or any explicit value).Compatibility
PekkoForkJoinPoolandForkJoinExecutorServiceFactorykeep theirminimumRunnable: Int = 1defaults).Follow-up
Once this PR is merged and a nightly run confirms green, a small follow-up PR will drop the
EXTRA_JVM_OPTSblock that #2889 added to.github/workflows/nightly-builds.yml(the workflow override becomes redundant).Test plan
sbt scalafmtAllsbt actor/compile actor-tests/Test/compilesbt 'actor-tests/testOnly org.apache.pekko.dispatch.ForkJoinExecutorConfiguratorSpec'— all 10 tests pass (1 pending on JDK 21+ for the legacy branch)sbt 'actor-tests/testOnly org.apache.pekko.dispatch.ForkJoinPoolStarvationSpec'— still pending as before, no regressionmainafter merge confirms JDK 17 / 21 / 25 matrix is green