Getting ElasticsearchWarning on search request

[MikhailMS]

Elasticsearch version (bin/elasticsearch --version):
7.16.2
elasticsearch-py version (elasticsearch.__versionstr__):
7.16.2

Description of the problem including expected versus actual behavior:
For some reason after Elasticsearch Async Client starts and I do first search request to Elasticsearch, I see that client also attempts a call to / path, ie https://es_node:9200/. I don't really understand why it needs to do so and how to disable such behavior.

In our case, Elasticsearch server is secured and user used for search requests only has permissions to access data in the specific index and such calls to path like / would fail. For that specific call I can see an error

{MainThread} WARNING (log_request_fail) in [elasticsearch] => GET https://es_node:9200/` [status:403 request:0.355s]
{MainThread} DEBUG (log_request_fail) in [elasticsearch] => > None
DEBUG (log_request_fail) in [elasticsearch] => < {"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [search_user]"}],"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [search_user]"},"status":403}
 ElasticsearchWarning: The client is unable to verify that the server is Elasticsearch due security privileges on the server side

Steps to reproduce:

1. Have a running Elasticsearch server/cluster
2. Create user that only has index access privilege (in our case user only has read privilege)
3. Create an async client and attempt to submit a search request to Elasticsearch with user created in step 2
4. Now you should observe the same behavior/warning

Need to note, that the search request would complete nonetheless and any subsequent call is fine, but still may confuse when one looks through logs and notes such entry

[sethmlarson]

Hello! Thanks for opening this, it looks like you're running into a warning stemming from product verification. We implement product verification in 7.x via a pre-flight request to / to support historical versions (pre-7.14) that didn't have the X-Elastic-Product HTTP header on responses. Going forward in 8.0 and beyond the X-Elastic-Product header will be sent in all responses and so we don't require the pre-flight request to / so this issue is already "fixed" in the next major release of the client.

There are a couple ways to silence the warning for 7.x though, you can either:

1. Add the monitor privilege to the authenticated user. From the documentation this privilege only gives access to cluster state and health information, it's also a read-only privilege.
2. Capture the ElasticsearchWarning on the first request and ignore it:

import warnings

client = Elasticsearch(...)

with warnings.catch_warnings(record=True) as w:
    client.info()

client.search(...)
# (Further usage of the client)

Another solution is upgrading to Elasticsearch and Elasticsearch-py 8.0+ when they're available, but at the time of writing they're not released yet. Let me know if this answers your questions?

[adamsiwiec1]

Getting this same error

Z:\elasticsearch\elasticsearch_flask\app.py:56: ElasticsearchWarning: The client is unable to verify that the server is Elasticsearch due security privileges on the server side
  print('connected to elastic') if es.ping() else print('not connected to elastic')

[danizen]

The warning is a good thing to have, because Python includes a standard mechanism to turn the display of these warnings off. In my case, I like to see them on the command-line, to remind me to nag DevOps about switching from HTTP BasicAuth to X-Pack.

The following is how to disable the display of these warnings using the standard mechanisms provided by the warnings library package:

import warnings
from elasticsearch.exceptions import ElasticsearchWarning

    # do something with warnings off
    warnings.simplefilter('ignore', ElasticsearchWarning)
    # ...

[MikhailMS]

@sethmlarson
Thank you for suggestions! Unfortunately neither of those 2 work for us

1. We keep privileges list for each user at the bare minimum, so adding monitor is not an option (even if that's quite small privilege)
2. This one works in isolation, but cannot be used in our application

So I'd be waiting patiently for the version 8 where this call would be removed :) and since it is not really a bug or something, I believe the issue could be closed (unless there is more to it here)

[sethmlarson]

Sounds good, thanks for replying back @MikhailMS. So you know we've been releasing v8.0.0 pre-releases recently. If you're interested you can give them a try before the actual release.

Going to close this for now since there's nothing more to do.