Can't use params in a get() with OAuth creds

[michaelhelmick]

So, seeing that the params being handled was fixed https://github.com/kennethreitz/requests/pull/712

I tried to update Twython to use requests 0.13.3
Params are still not being handled correctly with OAuth.

app_key = u'***'
app_secret = u'***'
oauth_token = u'***'
oauth_token_secret = u'***'

auth = OAuth1(app_key, app_secret, oauth_token, oauth_token_secret,
                               signature_type='auth_header')

client = requests.session(auth=auth)
client.get('http://api.twitter.com/1/statuses/home_timeline.json', params={'count':2}) # Returns a 401

But this works fine:

client.get('http://api.twitter.com/1/statuses/home_timeline.json') # Returns a 200

[gulopine]

It looks like https://github.com/kennethreitz/requests/pull/712 was only for URLs with fragments, which doesn't match your case. I haven't had much of a chance to look at it again since you commented on https://github.com/kennethreitz/requests/pull/684 except to verify that I was testing with LinkedIn, which apparently verifies the signature differently, so it wasn't erroring out. I took that as confirmation that this was no longer an issue, without testing it with other providers, such as your case with Twitter.

I'm starting to wonder if there might be a bug in oauthlib, which provides the signature functionality. I'll have to do some more digging to determine if that's the case.

[michaelhelmick]

@gulopine Ah, thanks! I couldn't find my comment in 684! Haha, I was looking for it. And what does Linkedin use? Query or body?

[gulopine]

I guess I have to look at it some more. I've looked at so many providers for foauth.org that I think I'm getting really confused. I know I have Dropbox working with params, and that may be what I'm thinking of, not LinkedIn. Dropbox and Twitter both use the Authorization header for the signature payload, but Dropbox uses the plaintext approach, which doesn't take parameters into account at all. So it's not surprising to me now that it would behave very differently from Twitter.

[michaelhelmick]

Oh, I didn't know you were behind foauth. Haha. Nice service.
Anyways, yeah, I wish that there was more of a standard behind all API of which OAuth should be used. The amount of services that use OAuth2 is terrible compared to OAuth1, not sure why major companies are not deprecating OAuth1 support. But yes, Dropbox does use plaintext instead of HMAC >.>

Mike Helmick

On Friday, July 13, 2012 at 1:14 PM, Marty Alchin wrote:

I guess I have to look at it some more. I've looked at so many providers for foauth.org (http://foauth.org) that I think I'm getting really confused. I know I have Dropbox working with params, and that may be what I'm thinking of, not LinkedIn. Dropbox and Twitter both use the Authorization header for the signature payload, but Dropbox uses the plaintext approach, which doesn't take parameters into account at all. So it's not surprising to me now that it would behave very differently from Twitter.

---

Reply to this email directly or view it on GitHub:
https://github.com/kennethreitz/requests/issues/722#issuecomment-6969266

[gulopine]

So I figured out what's going on here, but I'm not sure yet how to best fix it. The fix I put in for https://github.com/kennethreitz/requests/pull/684 does help it generate the correct signature, but then requests rewrites the URL based on the result from oauthlib, in case it added OAuth parameters to the QueryString. Unfortunately, it rewrites the base URL, which then has all the parameters added to it later, before firing it off to the provider.

So in a nutshell, here's the URL that requests is actually sending Twitter, in this case:

http://api.twitter.com/1/statuses/home_timeline.json?count=2&count=2

Notice how count=2 shows up twice, even though the signature assumes it only shows up once. So of course it fails verification on Twitter's end.

The trouble is, we need it to rewrite the URL in case OAuth parameters get added to it, but we can't let it add non-OAuth parameters to it, because requests will add those later. So it seems like there are a couple options:

1. Remove non-OAuth parameters from the URL retrieved from oauthlib before assigning to the request in requests.
2. Make request.full_url a writable property, which will parse the QueryString and assign things back to params.
3. Something else I'm not thinking of right now.

I like the second option the best overall, because it seems more useful in the long run, in case something else like this happens elsewhere. But it might be more practical to go with the first option since it probably won't come up anywhere else anyway. I'll sleep on it and see what tomorrow brings.

[michaelhelmick]

I agree, second does sound best, but the first is more practical.. but for some reason I feel like it's #3

I just woke up, so let me think about this to.

[gulopine]

After trying out a few alternatives, I talked with @kennethreitz briefly about this on IRC, and he came up with the obvious solution:

gulopine@cf24d37

I think I screwed up my fork, so I don't know if I can submit a proper pull request, much less attach it to this ticket. That commit should illustrate the point, though.