Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Requests library incompatibility #1367

Closed
bryanweingarten opened this Issue · 12 comments

5 participants

@bryanweingarten

I'm using requests 1.1.0 which is a required dependency from another library I'm using. In the file boto/cloudsearch/documents.py:197 requests is being called like this:

    r = requests.post(url, data=sdf, config=request_config,
        headers={'Content-Type': 'application/json'})

When I was using requests 1.0, the way boto called it was ok. But with requests 1.1, the config parameter being passed by boto is incorrect. Here is the exception. This is happening using boto 2.8.0.

File "/usr/local/lib/python2.7/dist-packages/boto/cloudsearch/document.py", line 198, in commit
headers={'Content-Type': 'application/json'})
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 87, in post
return request('post', url, data=data, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, *
kwargs)
TypeError: request() got an unexpected keyword argument 'config'

To temporarily solve this issue, I commented out passing in the config parameter. But, I have no idea if this is correct or not.

@ccorbacho

Actually, it seems that boto is incompatible with the entire requests 1.x series, as the config parameter was removed back in requests 1.0 - see kennethreitz/requests@4e5c4a6

@bryanweingarten

I see this still hasn't been fixed in the master branch. I'm very confused how people are working with an old version of the requests library when it's a popular (possibly the most popular trending HTTP) library that is a dependency in many other libraries. This incompatibility prevents boto cloud search from working at all without commenting out the config parameter. Are people just forking boto and patching it themselves? Or are people not having dependencies on a current requests library? I would have assumed that this would have prevented it from working for just about everyone. If what @ccorbacho said is correct, then boto cloudsearch is not even using a released 1.0+ version of the requests library, it's still depending on a pre 1.0 version.

@pasc I believe I met you at PyCon and you promised me this would be fixed right away. Can you please see if this can be fixed soon? I'm unable to deploy without having my own personal version of boto and commenting out the config parameter. It just seems kind of critical to me and most people will eventually run into it.

@kopertop
Owner
@toastdriven

I'll add this to the list of things to dive into tomorrow. We'll post an update here when we have a fix.

@bryanweingarten

@toastdriven Thanks... I and many others would appreciate that very much.

@pcn

My selfish interest in this is that I need requests > 1.1.0 for gssapi auth to work, and I need gssapi auth with requests in order to get STS credentials via federation, which I need to use aws via boto. Since 1.2.0 is released and the APIs required for requests-kerberos are there, would it be OK to put requests version 1.2.0 as the dependency for the next boto version?

@bryanweingarten

@pcn, If the new requirements for boto would be >= 1.1, the you should still be able to use the 1.2 library. I don't see why 1.2.0 needs to be the minimum requirement.

@pcn

I actually said > 1.1.0, not >=1.1.0. I can see how that's confusing. My current internal package is a cut of >1.1.0 and < 1.2.0 so I am being highly specific WRT my own needs.

@bryanweingarten

@pcn, I understand what you said... I was just not understanding if the boto's minimum requirements is >=1.1.0, why you couldn't use 1.2. Is there something that would break?

@pcn

I'm completely OK with 1.2. I want 1.2. I just don't want a version < 1.2.0.

@toastdriven

The reason this hadn't been addressed sooner is that, in dropping that config=... parameter from the commit @ccorbacho mentioned, Requests lost the ability to easily specify the options located here: https://github.com/boto/boto/blob/develop/boto/cloudsearch/document.py#L190-L195

Without connection pooling & retries, adding lots of documents at once could be very painful. It's possible to replicate these settings with a more modern requests, but it involves more work (& more to test) than the previously simple config options.

When I have a working implementation that doesn't sacrifice the functionality, I'll post here.

@toastdriven

Fixed in SHA: 0af8c95. The requirements.txt file has been updated, so you should be able to use a modern version of Requests. We require requests>=1.1.0, so any version (including 1.2.0) is fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.