[BUG] Elasticsearch module execution via proxies fails with requests version 2.26.0

[sootythebear]

Description
When using the elasticsearch module, where the communication is configured to use "proxies" within the elasticsearch profile, the module fails if the Python environment has requests version 2.26.0 installed. The module, same profile, does not fail if the requests version is 2.25.1 is used.

Setup

Please be as specific as possible and give set-up details.

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD

The Salt Minion is running within a Python virtualenv.

Steps to Reproduce the behavior

In this particular example, the module is being called within a state via:

{% set record_exist = salt['elasticsearch.document_exists'](index="xxxx", doc_type="_doc", id=xxxx) %}

*** Elasticsearch Profile ***

elasticsearch:
  host: 'xxxxxx-xxxx.xxxxx.xxx:443'
  use_ssl: True
  verify_certs: True
  username: xxxxxxxxx
  password: xxxxxxxx
  ca_certs: /xxxx/xxxx/xxxx/xxxx/xxxx/xxxx.crt
  proxies:
    - https: http://xxxxx.xxxxx.xxxxx:8080
    - http: http://xxxxx.xxxxx.xxxxx:8080

*** Salt Minion env output ***

xxxxxxxxxxx:
    LC_MEASUREMENT=C
    LC_PAPER=C
    LC_MONETARY=C
    LANG=en_US.UTF-8
    INVOCATION_ID=xxxxxxxxxxx
    NOTIFY_SOCKET=/run/systemd/notify
    LC_NAME=C
    LC_COLLATE=C
    PWD=/root
    LC_CTYPE=C
    JOURNAL_STREAM=9:42228582
    https_proxy=http://xxxxx.xxxxx.xxxxx:8080
    http_proxy=http://xxxxx.xxxxx.xxxxx:8080
    LC_ADDRESS=C
    LC_NUMERIC=C
    LC_MESSAGES=C
    SHLVL=1
    LANGUAGE=C
    LC_TELEPHONE=C
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/xxxx/xxxx/bin
    LC_IDENTIFICATION=C
    LC_TIME=C
    _=/usr/bin/env

*** Salt Minion debug output ***

2021-08-30 13:05:25,031 [salt.utils.templates:274 ][ERROR   ][2736831] Rendering exception occurred
Traceback (most recent call last):
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/connection/http_requests.py", line 166, in perform_request
    response = self.session.send(prepared_request, **send_kwargs)
  File "/xxxx/xxxx/lib/python3.6/site-packages/requests/sessions.py", line 636, in send
    kwargs.setdefault('proxies', self.rebuild_proxies(request, self.proxies))
  File "/xxxx/xxxx/lib/python3.6/site-packages/requests/sessions.py", line 290, in rebuild_proxies
    no_proxy = proxies.get('no_proxy')
AttributeError: 'list' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/modules/elasticsearch.py", line 159, in _get_instance
    es.info()
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 295, in info
    "GET", "/", params=params, headers=headers
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/transport.py", line 413, in perform_request
    raise e
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/transport.py", line 388, in perform_request
    timeout=timeout,
  File "/xxxx/xxxx/lib/python3.6/site-packages/elasticsearch/connection/http_requests.py", line 184, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError('list' object has no attribute 'get') caused by: AttributeError('list' object has no attribute 'get')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/utils/templates.py", line 501, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/xxxx/xxxx/lib/python3.6/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/xxxx/xxxx/lib/python3.6/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 19, in top-level template code
  File "/xxxx/xxxx/lib/python3.6/site-packages/jinja2/sandbox.py", line 393, in call
    return __context.call(__obj, *args, **kwargs)
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/modules/elasticsearch.py", line 568, in document_exists
    es = _get_instance(hosts, profile)
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/modules/elasticsearch.py", line 163, in _get_instance
    hosts, err
salt.exceptions.CommandExecutionError: Could not connect to Elasticsearch host/ cluster ['xxxxxx-xxxx.xxxxx.xxx:443'] due to ConnectionError('list' object has no attribute 'get') caused by: AttributeError('list' object has no attribute 'get')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/utils/templates.py", line 261, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/xxxx/xxxx/lib/python3.6/site-packages/salt/utils/templates.py", line 529, in render_jinja_tmpl
    tmplstr,
salt.exceptions.SaltRenderError: Problem running salt function in Jinja template: Could not connect to Elasticsearch host/ cluster ['xxxxxx-xxxx.xxxxx.xxx:443'] due to ConnectionError('list' object has no attribute 'get') caused by: AttributeError('list' object has no attribute 'get'); line 19

Expected behavior
The Elasticsearch module is able to source the required proxy information, either from the Salt Minion elasticsearch configuration or via the shell environment.

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report

salt --versions-report

Salt Master and Minion are the same version.

Enclosed pip list provided after versions report ( Note: pip list is from working env, Update requests to 2.26.0 and it fails as above).

 ./salt --versions-report
Salt Version:
          Salt: 3002.6

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.0.1
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.10.1
        pygit2: Not Installed
        Python: 3.6.8 (default, Mar 18 2021, 08:58:41)
  python-gnupg: Not Installed
        PyYAML: 5.4.1
         PyZMQ: 22.1.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: rhel 8.4 Ootpa
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-305.3.1.el8_4.x86_64
        system: Linux
       version: Red Hat Enterprise Linux 8.4 Ootpa

Package            Version
------------------ ---------
certifi            2021.5.30
chardet            4.0.0
charset-normalizer 2.0.1
croniter           1.0.15
distro             1.5.0
elasticsearch      7.13.3
idna               2.10
Jinja2             3.0.1
MarkupSafe         2.0.1
msgpack            1.0.2
pip                21.1.3
psutil             5.8.0
pycryptodomex      3.10.1
pyinotify          0.9.6
python-dateutil    2.8.2
PyYAML             5.4.1
pyzmq              22.1.0
requests           2.25.1
salt               3002.6
setuptools         57.1.0
six                1.16.0
urllib3            1.26.6
wheel              0.36.2

Additional context
The error mentions the inability to source "no_proxy". I did attempt setting the "no_proxy" ENV variable within the service environment and as "no:" within the Elasticsearch profile, but this did not effect the outcome.

[sootythebear]

Reviewing the requests code, a rebuild_proxies() function was added at 2.26.0.

Issues have been raised around this function, and the following PR is presently open:
psf/requests#5893

[anilsil]

The PR mentioned is merged, please retest with the latest salt version and file a new bug if needed.