lucene_snapshot: Fix muted RangeAggregatorTests.testRuntimeFieldTopLevelQueryNot

[ChrisHegarty]

This issue has been filed to track a muted test failure with lucene_snapshot (tracking Lucene 10.3)

REPRODUCE WITH: ./gradlew ":server:test" --tests "org.elasticsearch.search.aggregations.bucket.range.RangeAggregatorTests.testRuntimeFieldTopLevelQueryNotOptimized" -Dtests.seed=4E5C67A3A65D9577 -Dtests.locale=en-HK -Dtests.timezone=Etc/UCT -Druntime.java=24
RangeAggregatorTests > testRuntimeFieldTopLevelQueryNotOptimized FAILED
    java.lang.AssertionError:
    Expected: <class org.elasticsearch.search.aggregations.bucket.range.RangeAggregator$NoOverlap>
         but: was <class org.elasticsearch.search.aggregations.bucket.range.RangeAggregator$FromFilters>
        at __randomizedtesting.SeedInfo.seed([4E5C67A3A65D9577:A90C54C5933D5E3C]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at org.elasticsearch.test.ESTestCase.assertThat(ESTestCase.java:2653)
        at org.elasticsearch.search.aggregations.bucket.range.RangeAggregatorTests.lambda$testRuntimeFieldTopLevelQueryNotOptimized$60(RangeAggregatorTests.java:624)
        at org.elasticsearch.search.aggregations.AggregatorTestCase.debugTestCase(AggregatorTestCase.java:994)
        at org.elasticsearch.search.aggregations.AggregatorTestCase.lambda$debugTestCase$19(AggregatorTestCase.java:926)
        at org.elasticsearch.search.aggregations.AggregatorTestCase.withIndex(AggregatorTestCase.java:887)
        at org.elasticsearch.search.aggregations.AggregatorTestCase.debugTestCase(AggregatorTestCase.java:926)
        at org.elasticsearch.search.aggregations.bucket.range.RangeAggregatorTests.testRuntimeFieldTopLevelQueryNotOptimized(RangeAggregatorTests.java:609)

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[ChrisHegarty]

Muted by f77b0ee

[ChrisHegarty]

This is likely related to a Lucene 10.3 change in behaviour where some range queries have been optimized. E.g. may rewrite to a MatchAllDocs if the range can be proven to match all docs at rewrite time. Similar with match no docs.

[ChrisHegarty]

This is likely related to a Lucene 10.3 change in behaviour where some range queries have been optimized. E.g. may rewrite to a MatchAllDocs if the range can be proven to match all docs at rewrite time. Similar with match no docs.

I can confirm that this is indeed the case. The test expects docs to be split into 3 buckets, with ranges addRange(0, 1).addRange(1, 2).addRange(2, 3). And indexes docs all with IntPoint and SortedNumericDocValuesField with a 0 value - first bucket. Queries against this, of type IndexOrDocValuesQuery, now rewrite to MatchAllDocsQuery. This results in a different type of RangeAggregator.

Enriching the exception in the test we can see:
java.lang.AssertionError: Expected RangeAggregator.NoOverlap.class, but got:class org.elasticsearch.search.aggregations.bucket.range.RangeAggregator$FromFilters

[ChrisHegarty]

//cc @nik9000 - do you think that this is just a test issue? I would expect that the rewrite to now MatchAllDocsQuery, should improve things rather than hurt us, no?

[nik9000]

FromFilters is generally faster than NoOverlap. So long as we have a good example of something that needs to run as NoOverlap, it's good. Great? We're getting more cases that can be optimized now.

[nik9000]

This is assigned to me. Should I try and fix it? Some branch I should open a PR targeting?

[ChrisHegarty]

Thanks Nik. We're getting close to merging the Lucene 10.3 upgrade, and we generally don't want to carry test mutes as part of that. If you can, then fixing this in the lucene_snapshot branch would be great.