[CI] S3BlobContainerRetriesTests testReadRetriesAfterMeaningfulProgress failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #4514 / openjdk23_checkpart1_java-matrix
• elasticsearch-periodic #4474 / openjdk23_checkpart1_java-matrix
• elasticsearch-periodic #4454 / openjdk23_checkpart1_java-matrix

Reproduction Line:

./gradlew ":modules:repository-s3:test" --tests "org.elasticsearch.repositories.s3.S3BlobContainerRetriesTests.testReadDoesNotRetryForRepositoryAnalysis" -Dtests.seed=E48173E91C46D576 -Dtests.locale=ta-MY -Dtests.timezone=Cuba -Druntime.java=23

Applicable branches:
8.x

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: failureCount=1

Issue Reasons:

• [8.x] 3 failures in test testReadRetriesAfterMeaningfulProgress (0.6% fail rate in 470 executions)
• [8.x] 3 failures in step openjdk23_checkpart1_java-matrix (23.1% fail rate in 13 executions)
• [8.x] 3 failures in pipeline elasticsearch-periodic (23.1% fail rate in 13 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[ywangd]

The failure is JDK 23 specific. I think it matches this JDK change: https://bugs.openjdk.org/browse/JDK-8331847

The HTTP server no longer immediately sends response headers if chunked mode is selected or if the response has a body.

A few tests failed. But the main problem is with testReadDoesNotRetryForRepositoryAnalysis which fails with an assertion its the HttpHandler. This killed the http server and subsquent tests failed because of that. The test needs the header to be sent so that it opens the stream successfully and then fails at reading time. But if no header is sent, the opening fails with an NoHttpResponseException which leads to retry by s3client and the test does not allow retry.