[AUTOCUT] Gradle Check Flaky Test Report for MinimumClusterManagerNodesIT

[opensearch-ci-bot]

Flaky Test Report for MinimumClusterManagerNodesIT

Noticed the MinimumClusterManagerNodesIT has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
6049587	14040	40080	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock org.opensearch.cluster.MinimumClusterManagerNodesIT.classMethod
9675c4f	14465	41398	org.opensearch.cluster.MinimumClusterManagerNodesIT.classMethod org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
d56d8c8	14489	41572	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
0fc94ca	13799	39263	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
56d0b76	14401	41153	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
c89a17c	13888	40047	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
1386a9b	13930	39885	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
2e13e9c	14107	40782	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
3a38a6c	14365	41154	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
5919409	13945	39654	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
0c2ff03	14230	41005	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
201c673	14458	41410	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
4038a3c	14074	40854	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
64383dd	14561	41702	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
7650e64	14345	41056	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
802f2e6	14424	41253	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
8e32ed7	14394	41336	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
903784b	14414	41239	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
913013b	13948	39666	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
afeddc2	14037	40910	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
b8c7819	12782	41391	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
bb013da	13717	39669	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
bde48a7	14133	40532	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
bf43678	13809	39614	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
c49eca4	13721	40576	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
e1a632f	14340	40973	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
e67ced7	13784	39156	org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the MinimumClusterManagerNodesIT class are:

• 14119
• 13655
• 13821
• 13772
• 14135
• 13703
• 13957
• 14019
• 14115
• 14223
• 14266
• 14479
• 12439
• 13636
• 13813
• 14166
• 14200
• 14399
• 14409
• 14515
• 13374
• 13498
• 13590
• 13708
• 13785
• 13801
• 13807
• 13865
• 13924
• 14025
• 14064
• 14167
• 14391
• 14432
• 14565
• 14574
• 11573
• 12016
• 13131
• 13172
• 13315
• 13474
• 13525
• 13632
• 13637
• 13678
• 13681
• 13684
• 13722
• 13759
• 13810
• 13885
• 13887
• 13934
• 13944
• 13969
• 13997
• 14026
• 14069
• 14095
• 14111
• 14140
• 14151
• 14158
• 14238
• 14261
• 14369
• 14397
• 14400
• 14402
• 14411
• 14415
• 14437
• 14440
• 14487
• 14512
• 14540

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

Adding the Storage:Remote label to this one because I believe it has been traced back to a commit related to that feature. From the original issue:

I believe I have traced this back to the commit that introduced the flakiness: 9119b6d (#9105)

The following command will reliably reproduce the failure for me:

./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock" -Dtests.iters=100

If I select the commit immediately preceding 9119b6d then it does not reproduce.

This is a bit concerning because the commit in question is related to the remote store feature but MinimumClusterManagerNodesIT does not do anything related to remote store, so it is possible there is a significant regression here.