Flaky Test: SchemalessTestSimpleTest.testFullOnTimeseries/testFullOnSearch (JDK 25 SIGSEGV)

[FrankChen021]

Summary

SchemalessTestSimpleTest.testFullOnTimeseries and SchemalessTestSimpleTest.testFullOnSearch are flaky in the JDK 25 S* CI job. When they fail, they return empty results and the Surefire fork can subsequently terminate with a native SIGSEGV.

This was observed while validating PR #19981. The failure is in org.apache.druid.segment.SchemalessTestSimpleTest, not in a migrated test.

CI evidence

• Original failed test-jdk25-[S*] job, workflow run 31571194115, attempt 1
• Same job rerun, workflow run 31571194115, attempt 2

The original job reported two failures in this class and then a JVM crash. The rerun reproduced the same behavior: the Surefire retry mechanism ran the affected methods five times, and each method passed on runs 1, 2, 3, and 5 but failed on run 4.

Exact test failures

From the rerun log:

[ERROR] Tests run: 3, Failures: 2, Errors: 0, Skipped: 0 <<< FAILURE! -- in org.apache.druid.segment.SchemalessTestSimpleTest

[ERROR] org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnTimeseries
org.opentest4j.AssertionFailedError: : key[rows] ==> expected: <11> but was: <0>
    at org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnTimeseries(SchemalessTestSimpleTest.java:161)

[ERROR] org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnSearch
org.opentest4j.AssertionFailedError: expected: <Result{timestamp=2011-01-12T00:00:00.000Z, value=SearchResultValue{value=[Hit{dimension='placementish', value='a'}, Hit{dimension='quality', value='automotive'}, Hit{dimension='placement', value='mezzanine'}, Hit{dimension='market', value='total_market'}]}}> but was: <Result{timestamp=2011-01-12T00:00:00.000Z, value=SearchResultValue{value=[]}}>
    at org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnSearch(SchemalessTestSimpleTest.java:266)

The retry summary is:

[WARNING] Flakes:
org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnSearch
  Run 1: PASS
  Run 2: PASS
  Run 3: PASS
  Run 4: expected the four hits above, but got value=[]
  Run 5: PASS

org.apache.druid.segment.SchemalessTestSimpleTest.testFullOnTimeseries
  Run 1: PASS
  Run 2: PASS
  Run 3: PASS
  Run 4: expected: <11> but was: <0>
  Run 5: PASS

[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 0, Flakes: 2

This pass/fail/pass pattern confirms that both methods are flaky in this environment rather than deterministically broken.

JVM crash evidence

The same rerun log contains:

A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007fcd8c227851, pid=3073, tid=3075

JRE version: OpenJDK Runtime Environment Zulu25.36+15-CA (25.0.4+7)
Java VM: OpenJDK 64-Bit Server VM Zulu25.36+15-CA (25.0.4+7-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)

An error report file with more information is saved as:
  /home/runner/work/druid/druid/processing/hs_err_pid3073.log

The forked VM terminated without properly saying goodbye.
VM crash or System.exit called?
Process Exit Code: 134
Crashed tests:
org.apache.druid.segment.SchemalessTestSimpleTest

The fork command also included -Xmx2048m, -XX:MaxDirectMemorySize=2500m, --add-modules=jdk.incubator.vector, and JaCoCo; the CI log identifies the runtime as Zulu/OpenJDK 25.0.4 on linux-amd64.

Expected behavior

Both tests should be stable and the JDK 25 job should complete without a native JVM crash.

Investigation status

The evidence establishes:

1. Both test methods intermittently produce empty query/segment results.
2. The same failure reproduced in two CI attempts.
3. The failure is specific to the JDK 25 CI environment observed here and is followed by a native JVM crash.
4. The hosted log reports that hs_err_pid3073.log was written, but the file is not exposed in the job output, so the native crash stack is not available for subsystem-level attribution.

A direct-buffer, vector, or other JDK/runtime interaction is a hypothesis based on the JVM configuration, not a confirmed root cause.

Suggested follow-up

• Preserve and upload the hs_err_pid*.log file as a CI artifact when a JVM crash occurs.
• Compare this test on the JDK versions used by the matrix and on another JDK 25 distribution/build.
• Isolate the test with the vector module, JaCoCo, and direct-memory settings varied independently.
• Consider quarantining or retrying these tests in the JDK 25 job until the runtime issue is understood.

[FrankChen021]

changes of the test case

[FrankChen021]

A method level ParameterizedTest also results in the SIGSEGV problem

[FrankChen021]

CI JDK 25 configuration and resolved runtime

The unit-test entry workflow delegates to the reusable CI workflow:

• unit-and-integration-tests-unified.yml lines 54-56 calls .github/workflows/ci.yml.
• ci.yml lines 22-35 sets the unit-test matrix to jdk: ["25"] and passes that value to the worker.
• worker.yml lines 21-24 also defaults the reusable input to 25.
• worker.yml lines 69-73 invokes actions/setup-java@v5 with:
  • distribution: zulu
  • java-version: ${{ inputs.jdk }}

Therefore CI selects the JDK 25 feature line rather than pinning an exact patch/build. setup-java resolves the latest available matching Zulu build on each run.

In PR #19981 workflow attempt 11, S* job, setup reported:

Resolved latest version as 25.0.4+7
Downloading Java 25.0.4+7 (Zulu)
Distribution: zulu
Version: 25.0.4+7
Path: /opt/hostedtoolcache/Java_Zulu_jdk/25.0.4-7/x64

The crash header independently confirmed:

JRE: Zulu25.36+15-CA (25.0.4+7)
VM: OpenJDK 64-Bit Server VM, linux-amd64
Problematic frame: java.nio.DirectByteBuffer.getInt(I)I

Azul's July 2026 release notes map community build Zulu 25.36.15 to OpenJDK 25.0.4+7, so this failure occurred on the current July 2026 Zulu JDK 25 PSU build.