[BUG] org.opensearch.indices.replication.SegmentReplicationAllocationIT.testAllocationAndRebalanceWithDisruption is flaky

[backslasht]

Describe the bug

Details in https://build.ci.opensearch.org/job/gradle-check/36350/testReport/org.opensearch.indices.replication/SegmentReplicationAllocationIT/testAllocationAndRebalanceWithDisruption/

Related component

Storage:Remote

To Reproduce

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.indices.replication.SegmentReplicationAllocationIT.testAllocationAndRebalanceWithDisruption" -Dtests.seed=99CB296230F30435 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-CO -Dtests.timezone=America/Argentina/Cordoba -Druntime.java=21

Expected behavior

The test should not fail.

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@backslasht Thanks for filing

Thanks @mch2 for volunteering to look into this issue.

[andrross]

@mch2 This test case was newly added in #12656. Should we revert that PR?

[mch2]

@andrross I am on board with that or at least muting this test until this is resolved, I've run this locally and the seed is reproducible, haven't yet figured out why though @Arpit-Bandejiya mind taking a look?

[ashking94]

@Arpit-Bandejiya do take a look at this if you can.

[Arpit-Bandejiya]

Taking an look into the issue

[dblock]

#13513 (comment)

[mch2]

@Arpit-Bandejiya wondering if you've had a chance to look at this yet. Seems to be popping up more frequently. I think we should go ahead and mute this test for now?