New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FailureError('Unable to retrieve token secret from storage!',) #11

Closed
petertirrell opened this Issue Jun 21, 2013 · 21 comments

Comments

Projects
None yet
4 participants
@petertirrell

petertirrell commented Jun 21, 2013

Some users are reporting lost session after redirect within the login procedure.

the original issue as opened by @petertirrell was:

Pyramid Adapter and example

Implement an adapter for the Pyramid framework with example.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jun 21, 2013

I started trying to get something going on this, I figured it might be straightforward since Pyramid is based on WebOb, too. Finally just got it to actually compile with the module included, realized I was missing the openid module in my virtualenv.

petertirrell commented Jun 21, 2013

I started trying to get something going on this, I figured it might be straightforward since Pyramid is based on WebOb, too. Finally just got it to actually compile with the module included, realized I was missing the openid module in my virtualenv.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jun 22, 2013

I'm getting there I think, but am running into errors working with Twitter. Things seem to be hooked up, but I'm getting an error returned after the .login() method.

This is what I'm getting in the logs from my login handler:

2013-06-22 14:16:52,504 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Starting OAuth 1.0a authorization procedure.
2013-06-22 14:16:52,505 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Fetching for request token and token secret.
2013-06-22 14:16:52,948 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Got request token and token secret
2013-06-22 14:16:52,949 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Redirecting user to https://api.twitter.com/oauth/authorize?oauth_token=MyTokenHere(Didn't know if it was safe to paste or not?).

[At this point it appears that it's re-entering my login page. I'm not sure if this has anything to do with the callback URL set up on Twitter or not?]

2013-06-22 14:16:55,439 INFO [authomatic.core][Dummy-3] authomatic: Twitter: Continuing OAuth 1.0a authorization procedure after redirect.
2013-06-22 14:16:55,439 ERROR [authomatic.core][Dummy-3] authomatic: Twitter: Reported suppressed exception: FailureError('Unable to retrieve token secret from storage!',)!
2013-06-22 14:16:55,439 INFO [authomatic.core][Dummy-3] authomatic: Twitter: Procedure finished.

Then after that it never gets past the LoginResult error. I'm not sure if it's something with how I wired up the PyramidAdapter or if it's something with how I've set up my Twitter app dev settings? Any thoughts? Thanks!

petertirrell commented Jun 22, 2013

I'm getting there I think, but am running into errors working with Twitter. Things seem to be hooked up, but I'm getting an error returned after the .login() method.

This is what I'm getting in the logs from my login handler:

2013-06-22 14:16:52,504 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Starting OAuth 1.0a authorization procedure.
2013-06-22 14:16:52,505 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Fetching for request token and token secret.
2013-06-22 14:16:52,948 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Got request token and token secret
2013-06-22 14:16:52,949 INFO [authomatic.core][Dummy-2] authomatic: Twitter: Redirecting user to https://api.twitter.com/oauth/authorize?oauth_token=MyTokenHere(Didn't know if it was safe to paste or not?).

[At this point it appears that it's re-entering my login page. I'm not sure if this has anything to do with the callback URL set up on Twitter or not?]

2013-06-22 14:16:55,439 INFO [authomatic.core][Dummy-3] authomatic: Twitter: Continuing OAuth 1.0a authorization procedure after redirect.
2013-06-22 14:16:55,439 ERROR [authomatic.core][Dummy-3] authomatic: Twitter: Reported suppressed exception: FailureError('Unable to retrieve token secret from storage!',)!
2013-06-22 14:16:55,439 INFO [authomatic.core][Dummy-3] authomatic: Twitter: Procedure finished.

Then after that it never gets past the LoginResult error. I'm not sure if it's something with how I wired up the PyramidAdapter or if it's something with how I've set up my Twitter app dev settings? Any thoughts? Thanks!

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jun 23, 2013

So it seems that it's losing it's session value ('token_secret') after the the oauth 1.0a redirect. The session setting and getting does seem to work, as I've added some logging statements to try and see what's going on. It does set it and I'm able to immediately pull the value back out. But somehow after it redirects and re-enters login() 'token_secret' is no longer there to get.

petertirrell commented Jun 23, 2013

So it seems that it's losing it's session value ('token_secret') after the the oauth 1.0a redirect. The session setting and getting does seem to work, as I've added some logging statements to try and see what's going on. It does set it and I'm able to immediately pull the value back out. But somehow after it redirects and re-enters login() 'token_secret' is no longer there to get.

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jun 25, 2013

Member

@petertirrell Sorry for delayed response. I never get notifications on new issues.
Does this happen only with Twitter?
Could you please provide a gist with your code so I can take a look at it?

Member

peterhudec commented Jun 25, 2013

@petertirrell Sorry for delayed response. I never get notifications on new issues.
Does this happen only with Twitter?
Could you please provide a gist with your code so I can take a look at it?

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jun 26, 2013

Ugh - I responded last night and apparently it didn't post. Sorry! I haven't tried with anything but Twitter but I'll try setting it up with Facebook or Google.

I added a repo (https://github.com/petertirrell/LoginProject) with the Pyramid project I've been working with to set this up if you're interested.

petertirrell commented Jun 26, 2013

Ugh - I responded last night and apparently it didn't post. Sorry! I haven't tried with anything but Twitter but I'll try setting it up with Facebook or Google.

I added a repo (https://github.com/petertirrell/LoginProject) with the Pyramid project I've been working with to set this up if you're interested.

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jun 26, 2013

Member

Great. I will have time to look at it not sooner than next week.

Member

peterhudec commented Jun 26, 2013

Great. I will have time to look at it not sooner than next week.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jun 28, 2013

Thanks for your time! For what it's worth, I get the same behavior on Twitter, Facebook, and Google. At this point I think it might be related to Pyramid's session or cookie management....like the cookie that is used to set the token_secret isn't the same that is checked after the redirect. I'm trying to read up more on what is going on because it seems so close to working. Thanks again,

petertirrell commented Jun 28, 2013

Thanks for your time! For what it's worth, I get the same behavior on Twitter, Facebook, and Google. At this point I think it might be related to Pyramid's session or cookie management....like the cookie that is used to set the token_secret isn't the same that is checked after the redirect. I'm trying to read up more on what is going on because it seems so close to working. Thanks again,

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jul 3, 2013

Member

@petertirrell This is really strange.

I have created a simple pyramid example just with the WebObAdapter and it works like a charm.

But I recieved an email from a guy who is getting the same error with Django just by running the django example. He tried it also with the django session but with the same result.

The problem is that I cannot reproduce the error. Could you please provide as many details as you can about your environment (OS, python version, browser etc.)?

Member

peterhudec commented Jul 3, 2013

@petertirrell This is really strange.

I have created a simple pyramid example just with the WebObAdapter and it works like a charm.

But I recieved an email from a guy who is getting the same error with Django just by running the django example. He tried it also with the django session but with the same result.

The problem is that I cannot reproduce the error. Could you please provide as many details as you can about your environment (OS, python version, browser etc.)?

@expl0ratory

This comment has been minimized.

Show comment
Hide comment
@expl0ratory

expl0ratory Jul 3, 2013

Collaborator

Have you tried clearing your cookies on twitter? I ran into this with Facebook today after changing our route for sn authorizations. There were some old cookies hanging around with the wrong pathing causing the error.

Inspect your cookies both on your domain AND on the SN and make sure everything has the right path.

Collaborator

expl0ratory commented Jul 3, 2013

Have you tried clearing your cookies on twitter? I ran into this with Facebook today after changing our route for sn authorizations. There were some old cookies hanging around with the wrong pathing causing the error.

Inspect your cookies both on your domain AND on the SN and make sure everything has the right path.

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jul 4, 2013

Member

I have tried everything to reproduce it. I have changed the routes, switched frameworks, browsers, sessions and their backends but it still works 😬

Member

peterhudec commented Jul 4, 2013

I have tried everything to reproduce it. I have changed the routes, switched frameworks, browsers, sessions and their backends but it still works 😬

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jul 4, 2013

Interesting, thanks for the feedback! Your example looks like what I was trying to do so I'm going to go back through my code and see where I'm straying. At least it's good to know that it does work, just that I'm doing something wrong.

I did eventually get it to save the token, but by passing my beaker session to the login() function....which worked the first time but then started throwing me errors on subsequent tries :-). I'll check my cookies, too...I wondered if maybe I needed to clear things out.

petertirrell commented Jul 4, 2013

Interesting, thanks for the feedback! Your example looks like what I was trying to do so I'm going to go back through my code and see where I'm straying. At least it's good to know that it does work, just that I'm doing something wrong.

I did eventually get it to save the token, but by passing my beaker session to the login() function....which worked the first time but then started throwing me errors on subsequent tries :-). I'll check my cookies, too...I wondered if maybe I needed to clear things out.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jul 4, 2013

Frustrating, I'm still having problems! I'm basically mirroring the simple pyramid example now, just that it was based on the SQLAlchemy pyramid scaffold. My example is here, everything should be in the views.py file. For everything it's back to "Damn that error: Unable to retrieve token secret from storage!".

I tried running it in Chrome Incognito mode, too, and it gives the same behavior. My environment is Ubuntu 13.04, Pyramid 1.4, Python 2.7.4 in a virtualenv. The browser I'm testing with is Chrome stable on Ubuntu.

--ugh. So as I'm writing this I just tried in Firefox on here and it actually works. Both Facebook and Twitter. So I'm still going to post the comment in case something jumps out, but apparently there's something wrong with my Chrome session...not sure why the incognito doesn't fix that but at least that's a lead I can go on! Always PEBKAC isn't it!

petertirrell commented Jul 4, 2013

Frustrating, I'm still having problems! I'm basically mirroring the simple pyramid example now, just that it was based on the SQLAlchemy pyramid scaffold. My example is here, everything should be in the views.py file. For everything it's back to "Damn that error: Unable to retrieve token secret from storage!".

I tried running it in Chrome Incognito mode, too, and it gives the same behavior. My environment is Ubuntu 13.04, Pyramid 1.4, Python 2.7.4 in a virtualenv. The browser I'm testing with is Chrome stable on Ubuntu.

--ugh. So as I'm writing this I just tried in Firefox on here and it actually works. Both Facebook and Twitter. So I'm still going to post the comment in case something jumps out, but apparently there's something wrong with my Chrome session...not sure why the incognito doesn't fix that but at least that's a lead I can go on! Always PEBKAC isn't it!

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jul 5, 2013

Member

@petertirrell I was playing with the previous version of your LoginProject and the problem was that the login view didn't return the response object when there was not yet a LoginResult which means taht the login procedure is still pending. Athomatic uses the response internally and the view must return it to take effect.

This however didn't solve the issue with the lost session. I have tried the last version of the LoginProject with Pyramid 1.4.2 in venv but on Ubuntu 12.04 and it works in every browser I have. Also the previous version worked after I fixed the login view.

Member

peterhudec commented Jul 5, 2013

@petertirrell I was playing with the previous version of your LoginProject and the problem was that the login view didn't return the response object when there was not yet a LoginResult which means taht the login procedure is still pending. Athomatic uses the response internally and the view must return it to take effect.

This however didn't solve the issue with the lost session. I have tried the last version of the LoginProject with Pyramid 1.4.2 in venv but on Ubuntu 12.04 and it works in every browser I have. Also the previous version worked after I fixed the login view.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jul 6, 2013

Yeah, I saw after coming back that my response wasn't set up right. Thanks for looking into this - feel free to close this issue out. I looks like Authomatic works just fine in Pyramid, too, so it's just a matter of me figuring out what's going on with my system to cause me fits. Thanks again for all of your help!

petertirrell commented Jul 6, 2013

Yeah, I saw after coming back that my response wasn't set up right. Thanks for looking into this - feel free to close this issue out. I looks like Authomatic works just fine in Pyramid, too, so it's just a matter of me figuring out what's going on with my system to cause me fits. Thanks again for all of your help!

@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jul 10, 2013

Member

I have renamed the issue so it reflects the problem with the session. I will keep it open till we find out what's causing it so it can help other people having the same problem.

Member

peterhudec commented Jul 10, 2013

I have renamed the issue so it reflects the problem with the session. I will keep it open till we find out what's causing it so it can help other people having the same problem.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jul 10, 2013

I haven't had a chance to test it yet, but I think from reading it might have something to do with my chrome setting to 'continue where I left off' on startup. I think when I was messing with creating an actual PyramidAdapter (unnecessary now) I polluted my session cookie and it is still persisting. And from a chromium bug I found this persists even in incognito mode. When I get home I want to try and blow away my cache and everything in Chrome and see if that makes it work.

petertirrell commented Jul 10, 2013

I haven't had a chance to test it yet, but I think from reading it might have something to do with my chrome setting to 'continue where I left off' on startup. I think when I was messing with creating an actual PyramidAdapter (unnecessary now) I polluted my session cookie and it is still persisting. And from a chromium bug I found this persists even in incognito mode. When I get home I want to try and blow away my cache and everything in Chrome and see if that makes it work.

@jmurty

This comment has been minimized.

Show comment
Hide comment
@jmurty

jmurty Jul 15, 2013

Contributor

I hit this issue today, and with much trial-and-error I have found some work-arounds that seem to help, at least for Chrome on OSX.

The root problem seems to be buggy cookie-setting behaviour by WebKit-based browsers on redirects, per http://stackoverflow.com/questions/1144894/safari-doesnt-set-cookie-but-ie-ff-does Basically, Authomatic's Set-Cookie directive is incorrectly ignored by the browser in some circumstances when performed as part of a redirect, as it is for the Twitter auth flow (and likely many others)

In my particular case, running my dev site on "localhost" seemed to trigger the problem. Each of the following changes caused Chrome to set the cookie as it should:

  • use a full domain name instead of "localhost", e.g. by adding 127.0.0.1 mysite.com to my hosts file. This is workable but annoying
  • adjust the cookie template in authomatic/core.py:Session.create_cookie to leave out the Domain portion altogether. This is an awful hack
  • add special-case domain name handling to add a period to the beginning of any domains without any periods, e.g. to convert localhost to .localhost. This also feels like a hack but is perhaps an acceptable work-around?

I'll follow up shortly with a pull request containing the latter "fix".

Hopefully with the details I've given above others will be able to reproduce the issue and confirm/contest the fix.

Contributor

jmurty commented Jul 15, 2013

I hit this issue today, and with much trial-and-error I have found some work-arounds that seem to help, at least for Chrome on OSX.

The root problem seems to be buggy cookie-setting behaviour by WebKit-based browsers on redirects, per http://stackoverflow.com/questions/1144894/safari-doesnt-set-cookie-but-ie-ff-does Basically, Authomatic's Set-Cookie directive is incorrectly ignored by the browser in some circumstances when performed as part of a redirect, as it is for the Twitter auth flow (and likely many others)

In my particular case, running my dev site on "localhost" seemed to trigger the problem. Each of the following changes caused Chrome to set the cookie as it should:

  • use a full domain name instead of "localhost", e.g. by adding 127.0.0.1 mysite.com to my hosts file. This is workable but annoying
  • adjust the cookie template in authomatic/core.py:Session.create_cookie to leave out the Domain portion altogether. This is an awful hack
  • add special-case domain name handling to add a period to the beginning of any domains without any periods, e.g. to convert localhost to .localhost. This also feels like a hack but is perhaps an acceptable work-around?

I'll follow up shortly with a pull request containing the latter "fix".

Hopefully with the details I've given above others will be able to reproduce the issue and confirm/contest the fix.

@jmurty

This comment has been minimized.

Show comment
Hide comment
@jmurty

jmurty Jul 15, 2013

Contributor

Ugh, further testing shows that I was mistaken about the localhost to .localhost work-around. This doesn't actually work.

This leaves me with removing the Domain portion of the cookie template as the next-best option. Perhaps doing this only for domains that don't contain periods, such as localhost, isn't too awful?

What a mess...

Contributor

jmurty commented Jul 15, 2013

Ugh, further testing shows that I was mistaken about the localhost to .localhost work-around. This doesn't actually work.

This leaves me with removing the Domain portion of the cookie template as the next-best option. Perhaps doing this only for domains that don't contain periods, such as localhost, isn't too awful?

What a mess...

peterhudec added a commit that referenced this issue Jul 16, 2013

Merge pull request #18 from jmurty/issue/11
Make WebKit-based browsers set cookies on redirect in more cases, fixes #11
@peterhudec

This comment has been minimized.

Show comment
Hide comment
@peterhudec

peterhudec Jul 16, 2013

Member

Thanks James. I'm not sure whether removing the Domain lowers the security of the cookie but as this affects only domains without a dot this doesn't seem to be an issue. 👍

Member

peterhudec commented Jul 16, 2013

Thanks James. I'm not sure whether removing the Domain lowers the security of the cookie but as this affects only domains without a dot this doesn't seem to be an issue. 👍

@jmurty

This comment has been minimized.

Show comment
Hide comment
@jmurty

jmurty Jul 16, 2013

Contributor

You're welcome @peterhudec, thanks very much for Authomatic!

Removing the Domain attribute shouldn't affect security since the browser will automatically apply the domain from which the cookie was received if there is no explicit attribute.

Why removing the explicit Domain attribute fixes this issue at all is a mystery, but hopefully the tweak will help @petertirrell also.

Contributor

jmurty commented Jul 16, 2013

You're welcome @peterhudec, thanks very much for Authomatic!

Removing the Domain attribute shouldn't affect security since the browser will automatically apply the domain from which the cookie was received if there is no explicit attribute.

Why removing the explicit Domain attribute fixes this issue at all is a mystery, but hopefully the tweak will help @petertirrell also.

@petertirrell

This comment has been minimized.

Show comment
Hide comment
@petertirrell

petertirrell Jul 20, 2013

Thanks @jmurty! That did do the trick - works like a dream on my Chrome now, too.

petertirrell commented Jul 20, 2013

Thanks @jmurty! That did do the trick - works like a dream on my Chrome now, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment