urllib3 > 1.10 breaks connection

[katrielt]

When using latest urllib3 (1.11 as of now), http connection breaks

AttributeError: 'module' object has no attribute 'HTTPMessage'
WARNING:elasticsearch:GET http:/server:443/es/index/_search [status:N/A request:13.243s]
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 74, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/Library/Python/2.7/site-packages/urllib3/connectionpool.py", line 557, in urlopen
body=body, headers=headers)
File "/Library/Python/2.7/site-packages/urllib3/connectionpool.py", line 388, in _make_request
assert_header_parsing(httplib_response.msg)
File "/Library/Python/2.7/site-packages/urllib3/util/response.py", line 49, in assert_header_parsing
if not isinstance(headers, httplib.HTTPMessage):
AttributeError: 'module' object has no attribute 'HTTPMessage'

Reverting back to urllib3==1.10.4 fixes the problem

setup.py specifies:
install_requires = [
'urllib3>=1.8, <2.0',
]

Perhaps it should be changed to
install_requires = [
'urllib3>=1.8, <1.11',
]

until this is fixed.

[xcambar]

Ha ha excellent!
I've spent my morning trying to fix this (not fluent in python yet) and then I see your issue+fix created one hour ago!

Thank you very much @katrielt, I can confirm both the issue and fix.

[pquentin]

We got bitten by this one too. Only affects ES 1.7, it seems.

[daviddyball]

@pquentin we're running elasticsearch==1.6.0 and elasticsearch-dsl==0.0.5-dev0 and we get the same issues with urllib3 >= 1.11.

[honzakral]

Thanks for the report. I am, however, having trouble replicating this issue.

I tried urllib3 1.11 with ES 1.5 and 1.7, python 2.7 and 3.4 and to no avail - tests pass and I can normally connect and talk to ES.

Could you please provide some code to replicate this issue?

Once we replicate this we also should open an issue with urllib3 since this is clearly a backwards incompatible change on their part which shouldn't happen.

[rrva]

What about python 2.6?

[honzakral]

2.6 works for me too with 1.11

[katrielt]

When working in a clean virtualenv, I too was unable to re-produce.
This seems related to the presence of /Library/Python/2.7/site-packages/http/client.py on OSX., and the addition of the following line to urllib3==1.11 at /Library/Python/2.7/site-packages/urllib3/util/response.py:

try:
    import http.client as httplib
except ImportError:
    import httplib

This was added here - urllib3/urllib3@ffd9539
When changing that import to simply import httplib, it works with 1.11

http.client on my system reads:

from __future__ import absolute_import

from httplib import *

Seems like this is a urllib3 / OSX related issue... ?

[sigmavirus24]

Do people have httpie installed?

[sigmavirus24]

Nevermind, I checked the project and they use the namespace httpie not http. @katrielt what version of Python are you using?

[katrielt]

$ python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin

[sigmavirus24]

This seems related to the presence of /Library/Python/2.7/site-packages/http/client.py on OSX., and the addition of the following line to urllib3==1.11 at /Library/Python/2.7/site-packages/urllib3/util/response.py:

I'm on OSX and I don't see this path at all. I have /Library/Python/2.7/site-packages but I don't have http/ or http/client.py. That seems very ... odd.

[sigmavirus24]

@xcambar @pquentin @daviddyball What versions of Python are you using and on what Operating Systems? This is necessary information to help determine how to fix this since this will likely affect users of urllib3 as well as elasticsearch-py.

[katrielt]

http.client was introduced with Python 3, see the "Note" at the top of https://docs.python.org/2/library/httplib.html

I'm not sure how that module got into my Python 2.7 install. My guess the change to urllib3 was done for Python3 support.

[Koed00]

Same troubles here:

• Python 2.7.9
• Heroku Cedar-14 (Ubuntu 14.04 LTS)
• Bonsai ElasticSearch 1.5.2, Lucene 4.10.4