Skip to content

Fix flaky test TestCollapseQParserPlugin.testMultiSort#3609

Merged
dsmiley merged 2 commits intoapache:mainfrom
dsmiley:fixFlakyCollapsingTestIndexOrder
Sep 9, 2025
Merged

Fix flaky test TestCollapseQParserPlugin.testMultiSort#3609
dsmiley merged 2 commits intoapache:mainfrom
dsmiley:fixFlakyCollapsingTestIndexOrder

Conversation

@dsmiley
Copy link
Contributor

@dsmiley dsmiley commented Sep 3, 2025

Relying on index order is unreliable.

http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.search.TestCollapseQParserPlugin.testMultiSort

TestCollapseQParserPlugin > testMultiSort FAILED
    java.lang.AssertionError: REQUEST FAILED: xpath=//result/doc[1]/str[@name='id'][.='6']
    	xml response was: <?xml version="1.0" encoding="UTF-8"?>
    <response>
    <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="2" start="0" numFoundExact="true"><doc><str name="id">7</str><int name="id_i">7</int><str name="group_s">group2</str><int name="test_i">5</int><long name="test_l">1000</long><str name="term_s">XXXX</str><long name="_version_">1842211920030990336</long></doc><doc><str name="id">2</str><int name="id_i">2</int><str name="group_s">group1</str><int name="test_i">5</int><long name="test_l">1000</long><long name="_version_">1842211919995338752</long></doc></result>
    </response>

    	request was:q=*:*&fq={!collapse+field%3Dgroup_s+sort%3D'test_l+desc'}&sort=id_i+desc
        at __randomizedtesting.SeedInfo.seed([72818F0507C91947:D7ECFBF8490D9EDE]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:920)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:884)
        at org.apache.solr.search.TestCollapseQParserPlugin.testMultiSort(TestCollapseQParserPlugin.java:115)

Relying on index order is unreliable.
@hossman
Copy link
Member

hossman commented Sep 8, 2025

Why comment them out? just delete them.

IIRC the point of these assertions was to verify that the group head selection logic using sort would be consistent with what doc (in each group) would come first if you did NOT use collapse and instead filtered on the group using that sort -- and once upon a time that ment that (sort) ties were broken by index order, so the test was trying to prove the same tie breaker was used here (ie: first doc wins, not last doc)

if index order as a sorting tiebreaker is no longer a reliable guarantee at the lucene level, then just rip out these lines of testing

@dsmiley
Copy link
Contributor Author

dsmiley commented Sep 8, 2025

I'll delete this part then. Normally I do delete old code instead of comment it. In this case I didn't since it was pretty short and communicates an intention even if it's not practical to execute this commented code (due to unreliability).

I'll do that and merge tonight.

@dsmiley dsmiley merged commit a08e8cf into apache:main Sep 9, 2025
1 of 3 checks passed
@dsmiley dsmiley deleted the fixFlakyCollapsingTestIndexOrder branch September 9, 2025 01:58
dsmiley added a commit that referenced this pull request Sep 9, 2025
Relying on index order is unreliable.

(cherry picked from commit a08e8cf)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants