Description
Description
TestJoinUtil.testRandomOrdinalsJoin fails deterministically on branch_10x with a floating-point scoring mismatch. The root cause is that GlobalOrdinalsWithScoreCollector still accumulates scores in float, while the test helper (updated in #16378) now accumulates in double. I've logged this as a test failure because I don't think it impacts actual results.
Analysis of Failure
Branch: branch_10x at HEAD (67515ed)
java.lang.AssertionError: expected:<1.6125519> but was:<1.612552>
at TestJoinUtil.assertTopDocs(TestJoinUtil.java:1661)
at TestJoinUtil.testRandomOrdinalsJoin(TestJoinUtil.java:503)
Test params: codec=Asserting(Lucene104), sim=Asserting(RandomSimilarity(queryNorm=false): {from=DFR GB1, to=DFR GL3(800.0), id=IB SPL-L2, value=IB SPL-DZ(0.3)})
#16378 correctly identified that float addition is non-associative and fixed TermsWithScoreCollector and JoinUtil's numeric point join path to accumulate in double. However, the global ordinals join path via GlobalOrdinalsWithScoreCollector was not updated. It still accumulates total and avg scores in float[].
The test helper (JoinScore) now computes expected scores in double precision, but the actual query result comes from the float-precision ordinals collector. When a RandomSimilarity with DFR scorers is selected and multiple scores are summed, the different precisions produce slightly different results.
Observations
- Reproduces deterministically on branch_10x with the given seed
- Does not reproduce on main after 1000 iterations (maybe because of different codec behavior affecting doc ordering/score distribution?)
- The code looks like the same on both branches so likely hard to reproduce
Gradle command to reproduce
gradlew :lucene:join:test --tests "org.apache.lucene.search.join.TestJoinUtil.testRandomOrdinalsJoin"
-Ptests.seed=C3638DF47596A0F6
-Ptests.jvms=12
-Ptests.useSecurityManager=false
-Ptests.file.encoding=US-ASCII
-Ptests.vectorsize=default
-Ptests.forceintegervectors=false
Description
Description
TestJoinUtil.testRandomOrdinalsJoin fails deterministically on branch_10x with a floating-point scoring mismatch. The root cause is that GlobalOrdinalsWithScoreCollector still accumulates scores in float, while the test helper (updated in #16378) now accumulates in double. I've logged this as a test failure because I don't think it impacts actual results.
Analysis of Failure
Branch: branch_10x at HEAD (67515ed)
Test params:
codec=Asserting(Lucene104), sim=Asserting(RandomSimilarity(queryNorm=false): {from=DFR GB1, to=DFR GL3(800.0), id=IB SPL-L2, value=IB SPL-DZ(0.3)})#16378 correctly identified that float addition is non-associative and fixed TermsWithScoreCollector and JoinUtil's numeric point join path to accumulate in double. However, the global ordinals join path via GlobalOrdinalsWithScoreCollector was not updated. It still accumulates total and avg scores in float[].
The test helper (JoinScore) now computes expected scores in double precision, but the actual query result comes from the float-precision ordinals collector. When a RandomSimilarity with DFR scorers is selected and multiple scores are summed, the different precisions produce slightly different results.
Observations
Gradle command to reproduce
gradlew :lucene:join:test --tests "org.apache.lucene.search.join.TestJoinUtil.testRandomOrdinalsJoin"
-Ptests.seed=C3638DF47596A0F6
-Ptests.jvms=12
-Ptests.useSecurityManager=false
-Ptests.file.encoding=US-ASCII
-Ptests.vectorsize=default
-Ptests.forceintegervectors=false