[CI] WriteLoadConstraintDeciderIT testHighNodeWriteLoadPreventsNewShardAllocation failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #27742 / part1
• elasticsearch-pull-request #89032 / part-1
• elasticsearch-pull-request #89028 / part-1
• elasticsearch-intake #27741 / part1
• elasticsearch-pull-request #89024 / part-1
• elasticsearch-pull-request #89023 / part-1

Reproduction Line:

./gradlew ":server:internalClusterTest" --tests "org.elasticsearch.cluster.routing.allocation.decider.WriteLoadConstraintDeciderIT.testHighNodeWriteLoadPreventsNewShardAllocation" -Dtests.seed=15D5DF8195D36D11 -Dtests.locale=hi -Dtests.timezone=Asia/Srednekolymsk -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: safeGet: listener was completed exceptionally

Issue Reasons:

• [main] 2 consecutive failures in step part1
• [main] 2 consecutive failures in pipeline elasticsearch-intake
• [main] 6 failures in test testHighNodeWriteLoadPreventsNewShardAllocation (66.7% fail rate in 9 executions)
• [main] 2 failures in step part1 (100.0% fail rate in 2 executions)
• [main] 4 failures in step part-1 (57.1% fail rate in 7 executions)
• [main] 2 failures in pipeline elasticsearch-intake (100.0% fail rate in 2 executions)
• [main] 4 failures in pipeline elasticsearch-pull-request (57.1% fail rate in 7 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 3 consecutive failures in test testHighNodeWriteLoadPreventsNewShardAllocation
• [main] 2 consecutive failures in step part-1
• [main] 2 consecutive failures in pipeline elasticsearch-pull-request
• [main] 3 failures in test testHighNodeWriteLoadPreventsNewShardAllocation (60.0% fail rate in 5 executions)
• [main] 2 failures in step part-1 (50.0% fail rate in 4 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (50.0% fail rate in 4 executions)

Build Scans:

• elasticsearch-intake #27741 / part1
• elasticsearch-pull-request #89024 / part-1
• elasticsearch-pull-request #89023 / part-1

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)