Switch from cookies to bearer tokens #288

michielbdejong · 2014-06-09T17:03:45Z

This PR takes the auth token from the couchdb cookie and exposes it as a bearer
token at the HAPI level:

When couchdb sets a cookie, it is stripped, and the authToken is added to
the JSON body.
When a request comes in, the 'Bearer ...' token is read from the Authorization
header, and converted to a cookie for couchdb.

This happens at the hapi level.
The solution is based on hapijs/hapi#1637
and https://github.com/spumko/hapi/blob/master/docs/Reference.md

TODO:

~~create a couchdb mock in the test framework~~ (added test/unit/api-plugin-test.js)
use this to create a test for the mapProxyPath function
use this to create a test for the addCorsAndBearerToken method
add tests for the extractToken function
fix tests that rely on callback being called
fix clashing of integration test suites that both start couchdb on port 6006

michielbdejong · 2014-06-09T17:14:30Z

I could use some advice about

how to make the tests pass (they now fail because of https://github.com/hoodiehq/hoodie-server/blob/master/test/integration/assets.js#L18-L19)
~~how/where to add tests for the mapProxyPath and addCorsAndBearerToken functions~~
how to make sure callbacks get called in unit tests; see FIXMEs in test/unit/api-plugin-tests.js

michielbdejong · 2014-06-10T07:15:41Z

btw, the way I implemented this now, no OAuth dialog is necessary. The app just gets to see the user's real password and then calls hoodie.account.signIn the normal way. Much simpler! :)

Acconut · 2014-06-10T12:34:21Z

Will this be an integrated protection against CSRF?

svnlto · 2014-06-10T12:44:23Z

@Acconut there is a separate hapi module for CSRF protection https://github.com/spumko/crumb

Acconut · 2014-06-10T12:46:09Z

@svnlto Yes, but you're only vulnerable to CSRF if you use cookies. And if this PR removes all cookies between the browser and hoodie-server, we won't be vulnerable by default so no protection is needed.
Or am I missing something?

svnlto · 2014-06-10T12:49:55Z

@Acconut correct, but this makes me think that we should add CSRF protection by default? /cc @janl @caolan

Acconut · 2014-06-10T12:51:17Z

@svnlto The question is, if this PR removes cookie usage. :) But if not we should totally do! 👍

michielbdejong · 2014-06-10T12:58:08Z

@Acconut yes, it tries to strip all outgoing as well as incoming cookies, but I'm not sure whether it does so successfully, there may be edge cases I forgot. If there are no bugs in the cookie-stripping then your point about CSRF being solved by that sounds plausible. But IANASE. :)

And this PR introduces a new attack surface: the hoodie.account.authToken variable which will be available in memory, and persisted to config (localStorage?), so that also requires some extra pairs of eyes.

It's not a trivial change. But it makes new Hoodie('https://a.url.of.my.own.choice.com/'); work, which is I think an important part of the hoodie philosophy.

Acconut · 2014-06-10T13:02:42Z

@michielbdejong It's still possible for XSS attacks to get the token but before your PR, using cookies, XSS could also make own request, so I see this as an improvement 👍.

janl · 2014-06-10T13:14:29Z

Let’s keep the XSS/CSRF question out of this PR as the scope and intent are entirely different.

michielbdejong · 2014-07-02T09:49:13Z

It seems same-origin signup may be broken; i get a 409 and it displays "undefined: undefined" in the dialog. Will investigate further once I fix my Dockerfile for this branch.

michielbdejong · 2014-07-03T09:04:22Z

The code is working now, but before this gets merged I could use some help with:

fix tests that rely on callback being called
fix clashing of integration test suites that both start couchdb on port 6006
testing whether same-origin signup still works correctly

would someone be available to pair on that, maybe?

Also, Michiel Leenaars from nlnet pointed out I could offer to do a guest post on the Hoodie blog about "per-user Hoodie" once it's merged - for my part I think that would be a great idea! Let me know if that would work for you guys as well.

janl · 2014-07-03T11:40:40Z

@michielbdejong great progress. I’m meaning to look into this, but realistically, it won’t be before two weeks from now, so maybe someone can beat me to it :)

Also, yes, we would totally blog about this, if you want to supply a blog draft, let us know :)

Acconut · 2014-07-03T16:32:43Z

@michielbdejong I would enjoy helping with this PR but I'm currently busy and I need some time to get used to your changes but I'll try to contribute.

gr2m · 2014-07-14T10:56:24Z

lib/server/plugins/api/index.js

+      resp.headers = res.headers;
+
+      resp.headers['Access-Control-Allow-Origin'] = request.headers.origin || '*';
+      resp.headers['Access-Control-Allow-Headers'] = 'Authorization, Content-Length, Content-Type, If-Match, If-None-Match, Origin, X-Requested-With';


can you please make Access-Control-Allow-Headers dynamic, based on the request coming in, just like you do for Access-Control-Allow-Origin? Otherwise plugins with hooks depending on own headers won't work with CORS

good idea! although I guess that will only work if the headers are already sent in the OPTIONS request

this still seems unresolved?

I guess that will only work if the headers are already sent in the OPTIONS request

Shouldn't be a problem. When browsers need custom headers for CORS requests, they send the headers in the OPTIONS request as far as I know

ah ok, I could make it add all headers that are on the request, plus all headers from the Access-Control-Request-Headers request header. OK, I'll add that functionality. Anything else?

It looks good to me, I gave it another test with the latest version of your hoodie-server/hoodie.js pull requests.

Just one thought. Is there a reason to call the token "bearer token" and "auth token" at different places? Maybe it would reduce potential confusion to call it bearer token all over? Or do I miss something?

agreed, I'll make that change as well.

michielbdejong · 2014-07-22T11:29:16Z

If someone wants to pair on this in Berlin, I'm here until Monday evening

gr2m · 2014-09-16T11:10:39Z

@michielbdejong I just rebased it on master, there were a few conflicts. I know get an error when starting up, you also get it when running npm test. I pushed the rebased branch to https://github.com/hoodiehq/hoodie-server/tree/bearer-token-support-rebased

Any chance you could help me get this fixed? I want to get this merged, everything else is set from my side

michielbdejong · 2014-09-16T11:39:29Z

i'll have a look after lunch!

gr2m · 2014-09-16T11:49:17Z

@svnlto helped out me to get hoodie-server running again, it was a hapi problem. But I know see some of the addCorseAndBearerToken errors fail

gr2m · 2014-09-16T13:06:48Z

I fear I broke something with your hoodie-server work, the response to POST /_api/_session has no bearerToken property. If you could have a look at it, that'd be great

michielbdejong · 2014-09-16T17:10:00Z

OK, should be easy to fix. I lost some time getting couchdb running again on my devbox, but I'll continue tomorrow.

This PR takes auth token from the couchdb cookie and exposes it as a bearer token at the HAPI level: * When couchdb sets a cookie, it is stripped, and the authToken is added to the JSON body. * When a request comes in, the 'Bearer ...' token is read from the Authorization header, and converted to a cookie for couchdb. This happens at the hapi level. The solution is based on hapijs/hapi#1637 and https://github.com/spumko/hapi/blob/master/docs/Reference.md TODO: * [ ] create a couchdb mock in the test framework * [ ] use this to create a test for the `mapProxyPath` function * [ ] use this to create a test for the `addCorsAndBearerToken` method * [ ] add tests for the `extractToken` function add basic tests for api plugin tests for mapProxyPath add test for extractToken first draft of addCorsAndBearerToken tests use done function in tests use port number that is different from the one used in other integration tests fix hoodiehq#288 (comment) fix hoodiehq#288 (comment) jshint and typo fix merge errors from rebase try to fix https://travis-ci.org/hoodiehq/hoodie.js/builds/34471428 fix typo in test

michielbdejong · 2014-09-17T09:22:23Z

The latest changes from my branch are missing in your rebase, for instance the addAllowedHeader function.

I just rebased my branch on master, can you maybe test with my one?

I'm having a hard time getting hoodie-server and hoodie.js working on my machine, I have been trying for hours, but it just fails to install in different ways.

I'm going to start from scratch using the Ubuntu install instructions from hood.ie, instead of using the two git repos as a starting point.

gr2m · 2014-09-17T13:24:43Z

@michielbdejong Try to remove all karma packages from hoodie.js package json. Then it should run the npm install command at least. Sorry for the trouble, got bit a lot by this myself in the past. It's fixed in the new version that is currently in the works, but we have to get this merged into the old one.

Another problem I've found: the bearer token gets sent after you sign out. It gets removed form localStorage, but is still set on hoodie.account.bearerToken

michielbdejong · 2014-09-18T06:08:05Z

@gr2m ok, that should be fixed in michielbdejong/hoodie.js@893f999. Is everything else working now, if you use my rebase?

gr2m · 2014-09-18T10:01:56Z

Unfortunately not, when running the integration tests, the server crashes repeatedly with this error:

Debug: hapi, internal, implementation, error 
    SyntaxError: Uncaught error: Unexpected end of input
    at Object.parse (native)
    at /Users/gregor/JavaScripts/hood.ie/hoodie-server/lib/server/plugins/api/index.js:51:19
    at finish (/Users/gregor/JavaScripts/hood.ie/hoodie-server/node_modules/nipple/lib/index.js:202:20)
    at wrapped (/Users/gregor/JavaScripts/hood.ie/hoodie-server/node_modules/nipple/node_modules/hoek/lib/index.js:560:20)
    at onReaderFinish (/Users/gregor/JavaScripts/hood.ie/hoodie-server/node_modules/nipple/lib/index.js:255:16)
    at g (events.js:180:16)
    at EventEmitter.emit (events.js:117:20)
    at finishMaybe (_stream_writable.js:360:12)
    at endWritable (_stream_writable.js:367:3)
    at Writable.end (_stream_writable.js:345:5)

Have you seen that? Maybe we can pair on this later today or tomorrow?

To setup the integration tests locally, checkout https://github.com/hoodiehq/hoodie-integration-test/, do npm install, change port numbers in hosts.json file to point to your hoodie app that you test with (make sure admin password is set to "12345"), make sure latest selenium is running, and then run grunt intern:tests.

I also found a problem in hoodie.js that I fixed (https://gist.github.com/c109b87f27dacdd90a11), can you add me as collaborator to your fork?

gr2m · 2014-09-18T16:01:47Z

@michielbdejong Here is a script you can run against your hoodie test app using the bearer branches of hoodie-server and hoodie.js

// make sure localStorage is empty. If not, clear an reload page
var username = 'user' + Date.now();
var password = 'secret';
window.events = [];

[
  'signin',
  'signup',
  'signout',
  'changeusername',
  'changepassword',
  'passwordreset',
  'reauthenticated',
  'error:unauthenticated',
  'error:passwordreset'
].forEach(function(eventName) {
  hoodie.account.on(eventName, function() {
    window.events.push(eventName);
  });
});

hoodie.account.signIn('foo', 'bar').fail(function() {
  console.log('✓ sign in error message');
  hoodie.account.signUp(username, password).done(function() {
    console.log('✓ sign up');
    var oldPassword = password;
    password += '2';
    hoodie.account.changePassword(oldPassword, password).done(function() {
      console.log('✓ change password');
      username += '2';
      hoodie.account.changeUsername(password, username).done(function() {
        console.log('✓ change username');
        hoodie.account.signOut().done(function() {
          console.log('✓ sign out');
          hoodie.account.signIn(username, password).done(function() {
            console.log('✓ sign in');
            hoodie.account.destroy().done(function() {
              console.log('✓ destroy');
              var allEvents = events.join(',');
              var allEventsExpected = 'signup,changepassword,changeusername,signout,signin,signout';

              console.log('checking for events with 2s timeout to catch unwanted error:unauthenticated events');
              setTimeout(function() {
                if (allEvents === allEventsExpected) {
                  console.log('✓ events');
                } else {
                  console.log('Events failed. Expected ' + allEventsExpected + ' but was ' + allEvents);
                }
              }, 2000);
            });
          });
        });
      });
    });
  });
});

When running this code in latest Firefox stable on Mac, I see the hapi error above. I don't see it in Chrome. But even if wrap the line with a try / catch and fallback data to {}, the script above does not succeed, there is an error:unauthenticated triggered somewhere around the account.destroy call

Hope that helps.

gr2m · 2014-09-19T18:01:41Z

I've git pushed -d'd to the hoodie.js pull request, which resolves the problem with the error:unauthenticated event. Looking into this hapi error now

@svnlto

see hoodiehq#288 (comment) @svnlto could you review please?

gr2m · 2014-09-19T18:28:41Z

Okay, all integration tests pass now, but I had to hack the hapi exception mentioned above, see a69586a

I'd like someone review this before we merge.

This PR does two simple things: * look for an 'authToken' field in the sign-in response, and store this as hoodie.account.authToken * send an 'Authorization' header with 'Bearer ' + that token in each request to the hoodie-server. This PR goes together with a PR on the hoodie-server repo, which at the HAPI level, will: * strip the couchdb cookie and add it into the JSON response of signIn requests. * read the Authorization header to look for a token of the form 'Bearer ...', and create a cookie header from the '...', to send to the couchdb layer. set authToken from config at page load fix hoodiehq/hoodie-server#288 (comment)

This PR takes auth token from the couchdb cookie and exposes it as a bearer token at the HAPI level: * When couchdb sets a cookie, it is stripped, and the authToken is added to the JSON body. * When a request comes in, the 'Bearer ...' token is read from the Authorization header, and converted to a cookie for couchdb. This happens at the hapi level. The solution is based on hapijs/hapi#1637 and https://github.com/spumko/hapi/blob/master/docs/Reference.md TODO: * [ ] create a couchdb mock in the test framework * [ ] use this to create a test for the `mapProxyPath` function * [ ] use this to create a test for the `addCorsAndBearerToken` method * [ ] add tests for the `extractToken` function add basic tests for api plugin tests for mapProxyPath add test for extractToken first draft of addCorsAndBearerToken tests use done function in tests use port number that is different from the one used in other integration tests fix hoodiehq#288 (comment) fix hoodiehq#288 (comment) jshint and typo fix merge errors from rebase try to fix https://travis-ci.org/hoodiehq/hoodie.js/builds/34471428 fix typo in test

@svnlto

see hoodiehq#288 (comment) @svnlto could you review please?

Switch from cookies to bearer tokens This PR takes auth token from the couchdb cookie and exposes it as a bearer token at the HAPI level: * When couchdb sets a cookie, it is stripped, and the authToken is added to the JSON body. * When a request comes in, the 'Bearer ...' token is read from the Authorization header, and converted to a cookie for couchdb. This happens at the hapi level. The solution is based on hapijs/hapi#1637 and https://github.com/spumko/hapi/blob/master/docs/Reference.md BREAKING CHANGE: Hoodie is not using cookies for authentication anymore and relies on hoodie.js@^1.0.0

see hoodiehq#288 (comment)

hoodiehq#322

see #288 (comment)

gr2m · 2014-09-23T15:05:37Z

@michielbdejong are you around by chance? We have 4 of the unit tests failing now and are quite puzzled

michielbdejong · 2014-09-24T09:50:03Z

Sorry, was out of town for a few days. Will be on irc!

gr2m · 2014-09-24T18:14:55Z

any update? Sorry for pushing, I just need to coordinate with @boennemann when we both have time to finish the merge and release the new versions, and I'll be gone Friday - Monday. Would be great to get it done by tomorrow evening.

michielbdejong mentioned this pull request Jun 9, 2014

bearer token support hoodiehq/hoodie#323

Merged

michielbdejong mentioned this pull request Jun 9, 2014

Implement allowedOrigins setting #267

Closed

michielbdejong mentioned this pull request Jun 18, 2014

NLnet milestone 8: New feature - Hood.ie integration unhosted/u36#11

Closed

gr2m reviewed Jul 14, 2014
View reviewed changes

gr2m mentioned this pull request Jul 21, 2014

Pocket, Futon and the App all overwrite each other's sessions #134

Closed

davidpfahler added Type: Feature labels Jul 21, 2014

michielbdejong force-pushed the bearer-token-support branch from e86d489 to 45beb1c Compare September 17, 2014 09:08

gr2m added a commit to michielbdejong/hoodie-server that referenced this pull request Sep 19, 2014

workaround: prevent hapi, internal, implementation error

a69586a

see hoodiehq#288 (comment) @svnlto could you review please?

gr2m mentioned this pull request Sep 19, 2014

find cause and fix: "Debug: hapi, internal, implementation, error" #322

Closed

gr2m force-pushed the bearer-token-support branch from 84a402f to bf4f7c2 Compare September 23, 2014 13:55

gr2m added a commit to michielbdejong/hoodie-server that referenced this pull request Sep 23, 2014

workaround: prevent hapi, internal, implementation error

effb25f

see hoodiehq#288 (comment) @svnlto could you review please?

michielbdejong and others added 3 commits September 23, 2014 16:06

fix: prevent hapi, internal, implementation error

830f3de

see hoodiehq#288 (comment)

chore: added todo comment

1f50385

hoodiehq#322

gr2m force-pushed the bearer-token-support branch from bf4f7c2 to 1f50385 Compare September 23, 2014 14:09

gr2m added a commit that referenced this pull request Sep 23, 2014

fix: prevent hapi, internal, implementation error

6fab756

see #288 (comment)

gr2m closed this in hoodiehq/hoodie@0726487 Sep 23, 2014

Switch from cookies to bearer tokens #288

Switch from cookies to bearer tokens #288

Conversation

michielbdejong commented Jun 9, 2014

michielbdejong commented Jun 9, 2014

michielbdejong commented Jun 10, 2014

Acconut commented Jun 10, 2014

svnlto commented Jun 10, 2014

Acconut commented Jun 10, 2014

svnlto commented Jun 10, 2014

Acconut commented Jun 10, 2014

michielbdejong commented Jun 10, 2014

Acconut commented Jun 10, 2014

janl commented Jun 10, 2014

michielbdejong commented Jul 2, 2014

michielbdejong commented Jul 3, 2014

janl commented Jul 3, 2014

Acconut commented Jul 3, 2014

gr2m Jul 14, 2014

Choose a reason for hiding this comment

michielbdejong Jul 14, 2014

Choose a reason for hiding this comment

janl Sep 3, 2014

Choose a reason for hiding this comment

gr2m Sep 3, 2014

Choose a reason for hiding this comment

michielbdejong Sep 4, 2014

Choose a reason for hiding this comment

gr2m Sep 4, 2014

Choose a reason for hiding this comment

michielbdejong Sep 4, 2014

Choose a reason for hiding this comment

michielbdejong commented Jul 22, 2014

gr2m commented Sep 16, 2014

michielbdejong commented Sep 16, 2014

gr2m commented Sep 16, 2014

gr2m commented Sep 16, 2014

michielbdejong commented Sep 16, 2014

michielbdejong commented Sep 17, 2014

gr2m commented Sep 17, 2014

michielbdejong commented Sep 18, 2014

gr2m commented Sep 18, 2014

gr2m commented Sep 18, 2014

gr2m commented Sep 19, 2014

gr2m commented Sep 19, 2014

gr2m commented Sep 23, 2014

michielbdejong commented Sep 24, 2014

gr2m commented Sep 24, 2014