Unable to set query parameters (like `filter_path`) in search requests

[oliverlockwood]

Description

In the RHLC, which this client is meant to replace, you can make a call like:

restHighLevelClient.search(searchRequest, requestOptions);

It is possible to customise requestOptions such that a query param is added to REST requests going to Elasticsearch, e.g.

RequestOptions requestOptions = RequestOptions.DEFAULT.toBuilder()
        .addParameter("filter_path", "-hits.hits._source.annotations")
        .build();

This feature was added by elastic/elasticsearch#67802 and its respective back ports.

In the new Java client, you make a call like:

esClient.search(searchRequest, MyDocument.class)

I have not been able to find any way to customise the SearchRequest to set a parameter like filter_path. Please would you advise me how to do this; and if it's not currently possible, then please treat this as a feature request! It would be unfortunate to be blocked from upgrading to the new Java client, and thereby blocked from upgrading our applications beyond Java 8, simply because the new Java client is less capable than the existing RHLC.

[swallez]

You can achieve the same with the transport options:

TransportOptions filterOptions = esClient._transport().options().with(b -> b
    .setParameter("filter_path", "-hits.hits._source.annotations")
);

esClient.withTransportOptions(filterOptions).search(searchRequest, MyDocument.class);

The notation may feel a bit clunky by having to look up _transport() because of #206 that will improve it.

[swallez]

and thereby blocked from upgrading our applications beyond Java 8

@oliverlockwood I'm curious: can you elaborate on this? As far as I know HLRC should work just fine with any Java8+. Or are you encountering the split-package issue?

[oliverlockwood]

@swallez thank you very much for providing this information on TransportOptions - I'll give it a go.

Yes, we had been encountering the split-package issue. I can't find the comment just now, but I'm sure that last time I looked it up the response from Elastic was that to avoid the split-package issue, it was necessary to switch to using the new Java client.

From the link that you provided in your last message, do I presume that the split-package issue has actually also been fixed for the legacy RHLC, via a package rename on the server-side classes?

[swallez]

do I presume that the split-package issue has actually also been fixed for the legacy RHLC, via a package rename on the server-side classes?

Unfortunately no. elastic/elasticsearch#81984 indeed fixed the split package issue, but only in the main (8.2) and 8.1 branches as part of a larger effort to remove all split packages in the Elasticsearch code base. And HLRC has not been released in the 8.x series. It's still kept for some integration tests that use it, but will ultimately be replaced by the new Java API client and its code removed.

[swallez]

to avoid the split-package issue, it was necessary to switch to using the new Java client

Hmm... there are actually some other options consisting in adding the split package dependencies to the classpath or to the unamed module (see for example here and here). Now I certainly don't want to prevent you from using the new Java client 😉

[oliverlockwood]

Sounds like we don't have much choice but to upgrade in due course! Hopefully you will then fix the blocking issues which make the new client unusable in some cases, e.g.:

• Put Lifecycle method in ILM Client causes 400 Bad Request #160
• CompletionSuggester not working because prefix is placed at wrong position in JSON #150
• Bulk API should support documents encoded in JSON as well as encoded in SMILE #124

Otherwise we're somewhat stuck.

[swallez]

we don't have much choice but to upgrade in due course

For the HLRC split package issue, have you tried the solutions I suggested? We can also add to this list using the shade plugin to build a fat jar.