Java Rest Client swallows exceptions

[BBK-PiJ-2012-88]

Describe the feature:

Elasticsearch version (bin/elasticsearch --version): 5.* and 6.*

Plugins installed: []

JVM version (java -version):

OS version (uname -a if on a Unix-like system):

Description of the problem including expected versus actual behavior:

Actual Behaviour

Make a request to an elastic cluster using the java rest client that fails with a transient exception. If the initial request to the cluster takes longer to fail than the max retry timeout millis the request fails with an IOException and the cause of the initial failure is not added as a suppressed exception.

Expected Behaviour:

Make a request to an elastic cluster using the java rest client that fails with a transient exception. If the initial request to the cluster takes longer to fail than the max retry timeout millis the initial reason for the failure should be added to the IOException as a suppressed exception (this is the behaviour if the client attempts 1+ retry attempts).

Steps to reproduce:

This issue is hard to reproduce in a deterministic fashion. However, we encounter the issue fairly frequently with max retry timeout millis set to 1ms.

Provide logs (if relevant):

[javanna]

I believe this has been fixed with #25576 . Note that we also removed support for maxRetryTimeout in 7.0 with #38085 . WIth earlier versions, I'd recommend against using such a low maxRetryTimeout given the issues that can cause.

[elasticmachine]

Pinging @elastic/es-core-features

[BBK-PiJ-2012-88]

Hi @javanna, thanks for getting back to me so quickly. That does look like it fixes the issue. We have to support elastic 5 and 6 for our clients. Would it be okay if I raise a pull request to back port the fix for 5?

I agree about the timeout in principle. We set it so low because the retry logic in the client isn't stack safe and blew our stack in prod. I guess I should probably raise an issue on that front...

[javanna]

Please do raise an issue!

Are you using the low-level REST client or the high-level one?

[BBK-PiJ-2012-88]

Okay will do.

We're using 'org.elasticsearch.client.RestClient' via the Elastic4s library.

[javanna]

Ok that sounds like you are using the low-level REST client, which means that you can happily use the 6.6 client version (which contains the linked fix) against a 5.6 Elasticsearch cluster.

[BBK-PiJ-2012-88]

I'm not sure this will be possible with Elastic4s (standard Scala elastic client library). I'm pretty sure trying to bring in elastic 6 dependencies in conjunction with any Elastic4s 5.* versions will cause a lot of class path problems. #39920

[javanna]

Ok sounds like Elastic4s depends on the whole Elasticsearch despite it uses low-level REST client to communicate with it. In that case what I suggested will indeed cause problems. The 5.6 branch is not under active development though, hence I doubt that we will back-port that small fix to it at this point.

[BBK-PiJ-2012-88]

Okay, fair enough

[javanna]

Closing, see above