Skip to content
This repository has been archived by the owner on Oct 23, 2023. It is now read-only.

Conflict with httplib2 #703

Open
ben opened this issue Dec 15, 2015 · 3 comments
Open

Conflict with httplib2 #703

ben opened this issue Dec 15, 2015 · 3 comments

Comments

@ben
Copy link

ben commented Dec 15, 2015

Found this because of a strange exception when integrating with Celery and making HTTP requests, but there's a simpler reproduction scenario. Here's my example program:

import httplib2
http = httplib2.Http()

http.request('https://google.com', 'GET')
print(1)
from raven import Client
print(2)
http.request('https://google.com', 'GET')
print(3)

And here's the output:

$ python3 ~/Desktop/test.py
1
2
Traceback (most recent call last):
  File "/Users/ben/Desktop/test.py", line 8, in <module>
    http.request('https://httpbin.org/post', 'GET')
  File "/usr/local/lib/python3.4/site-packages/httplib2/__init__.py", line 1314, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python3.4/site-packages/httplib2/__init__.py", line 1064, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python3.4/site-packages/httplib2/__init__.py", line 1047, in _conn_request
    response = Response(response)
  File "/usr/local/lib/python3.4/site-packages/httplib2/__init__.py", line 1382, in __init__
    for key, value in info.items():
AttributeError: 'HTTPResponse' object has no attribute 'items'

The very act of importing raven.Client causes httplib2 to stop working. In my case, it surfaced as a mysterious error deep inside the Twilio client, which I'll paste below in case someone else googles for it.

This is using Python 3.4.3, raven==5.9.0, and httplib2==0.9.2, all of which appear to be the latest.

That Twilio library exception:

  File "/usr/local/lib/python3.5/site-packages/twilio/rest/resources/messages.py", line 122, in create
    return self.create_instance(kwargs)
  File "/usr/local/lib/python3.5/site-packages/twilio/rest/resources/base.py", line 365, in create_instance
    data=transform_params(body))
  File "/usr/local/lib/python3.5/site-packages/twilio/rest/resources/base.py", line 200, in request
    resp = make_twilio_request(method, uri, auth=self.auth, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/twilio/rest/resources/base.py", line 152, in make_twilio_request
    resp = make_request(method, uri, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/twilio/rest/resources/base.py", line 117, in make_request
    resp, content = http.request(url, method, headers=headers, body=data)
  File "/usr/local/lib/python3.5/site-packages/httplib2/__init__.py", line 1196, in request
    self.disable_ssl_certificate_validation)
  File "/usr/local/lib/python3.5/site-packages/httplib2/__init__.py", line 850, in __init__
    check_hostname=True)
  File "/usr/local/lib/python3.5/http/client.py", line 1209, in __init__
    super(HTTPSConnection, self).__init__(host, port, timeout,
TypeError: super(type, obj): obj must be an instance or subtype of type
@raylu
Copy link

raylu commented Dec 15, 2015

It turns out this is only an issue with eventlet installed. Importing raven causes all of the transports to get imported, including

raven-python/raven/transport/eventlet.py

Line 14 in 356a7c4

try:

which eventually imports https://github.com/eventlet/eventlet/blob/49773bb12b84e10ccb0a199c301f97fd6528ab68/eventlet/green/httplib.py#L16
which causes the check here to fail https://github.com/jcgregorio/httplib2/blob/cf631a73e2f3f43897b65206127ced82382d35f5/python3/httplib2/__init__.py#L1366
because httplib2 imported http.client before it was replaced by an identical version by eventlet. The simple fix is to import raven before importing httplib2/twilio.

@jstasiak
Copy link
Contributor

Eventlet bug report: eventlet/eventlet#316

jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green HTTP without undesired side effects
d5e753a 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

[1] getsentry/raven-python#703
[2] #316

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green HTTP without undesired side effects
52f04d0 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green HTTP without undesired side effects
12a14cb 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green http without undesired side effects
93cc00a 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
@jstasiak jstasiak mentioned this issue Jun 15, 2016
Closed
jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green http without undesired side effects
0675c47 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jun 15, 2016
@jstasiak
Provide green http without undesired side effects
dc69cf6 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jul 8, 2016
@jstasiak
Provide green http without undesired side effects
2912191 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the originl http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

A new dependency, enum-compat, is added to ensure that the enum module
is present on Python 3.3 (the http package code comes the latest Python
development branch and uses enum).

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
jstasiak added a commit to eventlet/eventlet that referenced this issue Jul 8, 2016
@jstasiak
Provide green http without undesired side effects
c932790 
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye <xdegaye@users.sourceforge.net>
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the original http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

A new dependency, enum-compat, is added to ensure that the enum module
is present on Python 3.3 (the http package code comes the latest Python
development branch and uses enum).

[1] getsentry/raven-python#703
[2] #316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
@jstasiak
Copy link
Contributor

FYI this is fixed in Eventlet master branch and should be released fairly soon.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jstasiak @ben @raylu