[CI] UpgradeClusterClientYamlTestSuiteIT test {p0=mixed_cluster/80_transform_jobs_crud/Test GET, start, and stop old cluster batch transforms} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #4529 / 8.1.3_bwc
• elasticsearch-pull-request #37407 / 8.1.3_bwc

Reproduction Line:

./gradlew ":x-pack:qa:rolling-upgrade:v8.1.3#twoThirdsUpgradedTest" -Dtests.class="org.elasticsearch.upgrades.UpgradeClusterClientYamlTestSuiteIT" -Dtests.method="test {p0=mixed_cluster/40_ml_datafeed_crud/Test old cluster datafeed without aggs}" -Dtests.seed=718F0DCEC6FC0495 -Dtests.bwc=true -Dtests.locale=kn -Dtests.timezone=Antarctica/Syowa -Druntime.java=22

Applicable branches:
8.x

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Failure in test setup: Failed to initialize at least 3 watcher nodes

Issue Reasons:

• [8.x] 2 failures in test test {p0=mixed_cluster/80_transform_jobs_crud/Test GET, start, and stop old cluster batch transforms} (0.2% fail rate in 825 executions)
• [8.x] 2 failures in step 8.1.3_bwc (33.3% fail rate in 6 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch 8.x

Mute Reasons:

• [8.x] 2 failures in test test {p0=mixed_cluster/80_transform_jobs_crud/Test GET, start, and stop old cluster batch transforms} (0.2% fail rate in 825 executions)
• [8.x] 2 failures in step 8.1.3_bwc (33.3% fail rate in 6 executions)

Build Scans:

• elasticsearch-periodic #4529 / 8.1.3_bwc
• elasticsearch-pull-request #37407 / 8.1.3_bwc

[elasticsearchmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)