[ML] Split ES integration tests into parallel steps

[edsavage]

Summary

• Split each per-architecture Elasticsearch integration test Buildkite step into two independent parallel steps: Multi-Node Tests (javaRestTest) and YAML REST Tests (yamlRestTest).
• Based on build 2309 timings, this reduces the critical path by ~11 min on x86_64 (40 min → 29 min) and ~14 min on aarch64 (53 min → 39 min).
• A new ES_TEST_SUITE environment variable in dev-tools/run_es_tests.sh controls which suite to run. When unset, both run sequentially (backward compatible for local use).

Timing breakdown (from build 2309)

Architecture	javaRestTest	yamlRestTest	Old total	New wall-clock
x86_64	27 min	12 min	40 min	~29 min
aarch64	37 min	15 min	53 min	~39 min

Additional benefits

• Immediate failure attribution — visible in Buildkite which suite failed
• Selective retry of just the failed suite
• Independent timeout budgets per suite

Test plan

• PR build shows 4 integration test steps (2 per arch) instead of 2
• All 4 steps run in parallel (both depend only on their build step, not each other)
• Each step runs only its designated suite (check logs for single ./gradlew invocation)
• Running dev-tools/run_es_tests.sh locally without ES_TEST_SUITE still runs both suites

Made with Cursor

[prodsecmachine]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[valeriy42]

LGTM