Avoid asking user to re-authorize client by persisting access and refresh tokens

[seanh]

Summary

• In the future we want the client to use OAuth token-based authorization (instead of cookie-based authorization) for normal, first-party accounts (as well as for the third-party accounts that it currently uses OAuth for).
• When using OAuth authorization the client needs to do a "flow" with the hypothes.is server to get an access and refresh token which it then uses to access the API.
• The client would have to do this flow every time it loads (e.g. on every page load) which would be annoying
• So we'd instead like the client to save the access and refresh token in some sort of persistent storage and re-use them as much as possible, minimising the amount of OAuth login flows that happen.

Problem that we're trying to solve with this card

Asking the user to re-authorize the client too frequently when using first-party accounts.

It's not implemented yet (at the time of writing) but in the future we intend for first-party accounts (e.g. normal seanh@hypothes.is users using a page-embedded or a browser extension instance of the client) to also use OAuth for authorization. See for example this card: Use OAuth authorization flow in client.

When this is implemented the client will end up doing an OAuth login flow - a popup window that may ask the user to login to hypothes.is, may ask the user to authorize the client, or may just immediately close automatically - every time it loads. So this popup window will happen every time a user loads a page that has an embedded client in it, or every time a user launches their Chrome extension client, or browses to a new page with their Chrome extension client activated. The result of the popup window is that the client gets a new access and refresh token.

That sounds potentially annoying so we'd like to reduce how often the popup window needs to be used to get a new access and refresh token, by having the client hold on to and reuse the tokens it got the first time for as long as possible.

Not for third-party accounts

It's important to note that the embedded client should not persist token when using third-party accounts. When using third-party accounts the partner site provides a grant token to the client which the client then exchanges for an access and refresh token pair. The client is relying on the partner site to tell it which user is logged-in (by providing a grant token for that user account). If the user logs out then the partner-provided grant token would change to null, and if the client had saved and continued using its access token then the client would still be logged in when it should be logged out. If another user logged-in to the partner site then the grant token would change to something else, and if the client had saved the access token from the first grant token and continued to use it, then the client would still be logged in as the first user. So it's important that persisting and reusing access and refresh tokens applies to first-party accounts only.

Solution

• When the client receives a first-party access and refresh token pair from the server it should save them to some form of persistent storage.

• Persistent here means that the stored tokens can be retrieved again after the browser has reloaded the page or navigated to another page and the client loads again; and after the browser tab or window is closed (or the browser app is closed, or the computer shut down...) and then opened again and the client launched again.

• When the client starts up and it needs an access token it should use the previously-stored token from persistent storage. The client should only do an OAuth login flow and get fresh tokens if there are no tokens in storage or if the stored tokens have expired and can't be refreshed.

• When there are multiple instances of the client in different browser tabs and windows, the idea is that they will all share the one access and refresh token pair.

• Nick has suggested that the client should use a pluggable list of persistent storage mechanisms and try one after the other: chrome.storage, localStorage, cookies (note: this means using a cookie as a place to store tokens, not using cookies directly for auth), indexeddb. Different forms of storage will be available or not available to the client under different circumstances.

  For example I think chrome.storage is always available to our browser extension? (We already request the storage permission.) Although it'd require some mechanism to communicate between client and extension.

  I think localStorage is not available in Safari when in private browsing mode, in Firefox when dom.storage.enabled is set to false in about:config, or in Chrome when "Block third-party cookies" is turned on (?)

  Cookies may not be available when we're in a third-party browsing context.

  If somehow no form of persistent storage is available the client should just store the tokens in (non-persistent) memory and otherwise keep working normally.

  Rob has suggested that implementing only localStorage and in-memory should be enough at first

• Take care when testing whether localStorage is available or not. For example I think Safari in private browsing mode there will be a window.localStorage and you can call setItem() on it but you will always get an "out of space" exception raised.

Potential issues with this solution

• If the access token expires while the client doesn't have an internet connection, then the client should try to use its refresh token to get a new access token ASAP after the internet connection comes back. This happens for example if the user puts the laptop to sleep, the access token expires, and then the user wakes up the laptop. It's important for the client to get a new access token ASAP because as soon as the user tries to do something that requires authentication (create, edit or delete an annotation, change the focused group...) an access token will be needed.

  Currently, when the client is using OAuth for third-party accounts, it checks every 30 seconds to see whether the access token has expired. This won't work for first-party accounts because it could take up to 30 seconds, after waking a laptop, before getting a new access token.

  One way to solve this might be to use the Document.ononline event.

• When there are multiple instances of the client in different tabs, and they're all sharing the same access and refresh token, they may all try to refresh the access token at the same time. There are two cases to worry about:

  1. The tabs are active when it comes time to refresh the access token, and all the tabs try to refresh it at once.
  2. The laptop is asleep when the access token expires. On wake all the tabs try to refresh it at once.

  Rob has suggested that this can be solved by having each instance of the client apply a random "jitter" to the time when it tries to refresh the token while also listening (using StorageEvents) for successful refreshes from other instances running in the same browser.

  In case 1 we have plenty of time to refresh the access token. We could for example refresh 7.5 minutes before it expires, with a random jitter of +/- 5 minutes. So it seems like a random jitter would work in this case. (Actually, each tab currently works by polling every 30 seconds to see whether the access token is within 5 mins of expiring, and that 30 second timeout starts for each tab when you open the tab, and you probably didn't open all your tabs at exactly the same time, so the times at which the tabs will attempt to refresh are already staggered. However, once one tab does a successful refresh and all the other tabs receive that new token pair via a StorageEvent this may bring all the tabs into sync in terms of when they will next try to refresh...)

  In case 2 we don't have as much time to refresh the access token. The laptop has just woken up and the client's access token has already expired, and it's necessary to get a new access token ASAP before the user tries to do something that needs it. Adding a random jitter means it will take longer for the client to get a new access token, how much longer depends on the jitter size and random chance. So this case seems like a potential problem for the jitter solution.

  We decided to look into implementing a fast mutex algorithm locally in JavaScript, instead of applying a random jitter to refresh times, see Slack discussion. Lamport's fast mutex algorithm, as described in this Medium Engineering post, sounds promising.

  @seanh recommends: implement the access and refresh token sharing and persistence first and then then look in to the problem of all the tabs trying to refresh at once.

See Also

• Original ticket this one was based on: Persist OAuth access & refresh tokens in client #315

[robertknight]

This is implemented.