Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Storage Access API #3338

Open
johnwilander opened this issue Jan 10, 2018 · 151 comments
Open

Proposal: Storage Access API #3338

johnwilander opened this issue Jan 10, 2018 · 151 comments

Comments

@johnwilander
Copy link

@johnwilander johnwilander commented Jan 10, 2018

Details to be discussed in the W3C Privacy CG

We've moved this to the W3C Privacy CG where you can file individual issues on the things you want to discuss: https://github.com/privacycg/storage-access


Original issue

Hi! John Wilander from WebKit here. We hope this can extend existing specifications rather than create some whole new spec. It was originally filed under whatwg/dom but I was advised to move it here.

Storage Access API

Problem

Tl;dr: Browsers that block access to third-party cookies break authenticated embeds such as commenting widgets and subscribed video services.

In the context of cross-origin resource loads, cookies are popularly referred to as third-party cookies. In reality, these cookies are often the same as the first-party cookies so what we really mean with third-party cookies is access to first-party cookies in a third-party context.

A browser may have rules for third-party cookies that go beyond cookie policy rules such as scheme, host, path, secure attribute etc. These additional rules may be:

  • Third-parties aren't allowed to set cookies,
  • Third-parties have their cookies partitioned or double-keyed, or
  • Third-parties have no cookie access.

However, certain services are intended to be embedded as third-party content and need access to first-party cookies for authentication. Examples are commenting widgets, video embeds, payment provider integration, document embeds, and social media action widgets. These break if the third-party content has no access to its first-party cookies.

The same problem exists for other kinds of storage such as IndexedDB and LocalStorage, except they are not tied to the HTTP protocol and are typically not used for authentication purposes. From here on we will refer to cookies except when the distinction between cookies and other storage makes sense.

Proposed Solution

Tl;dr: A new API with which cross-origin iframes can request access to their first-party cookies when processing a user gesture such as a tap or a click. This allows third-party embeds to authenticate on user interaction.

We propose two new functions on the document:

partial interface Document {
    Promise<bool> hasStorageAccess();
    Promise<void> requestStorageAccess();
};

The reasons these are on the document is that 1) storage access is granted to the particular document (see Access Removal) and 2) it changes document.cookie.

hasStorageAccess() can be called at any time to check whether access is already granted and it doesn't require user interaction.

requestStorageAccess() should only be called on user interaction such as a tap or a click. It will check a set of rules and grant access if the rules are fulfilled. Access to first-party cookies in the given iframe can be assumed if the returned promise resolves. From that point, any sub resource load in the iframe will have first-party cookies sent and incoming cookies will be set in the first-party cookie jar.

Note that no other third-party resources on that webpage are affected by the storage access status of an individual iframe.

Algorithm for requestStorageAccess()

  1. If the document already has been granted access, resolve.
  2. If the document has a null origin, reject.
  3. If the document's frame is the main frame, resolve.
  4. If the sub frame's origin is equal to the main frame's, resolve.
  5. If the sub frame is not sandboxed, skip to step 7.
  6. If the sub frame doesn't have the token "allow-storage-access-by-user-activation", reject.
  7. If the sub frame's parent frame is not the top frame, reject.
  8. If the browser is not processing a user gesture, reject.
  9. Check any additional rules that the browser has. Examples: Whitelists, blacklists, on-device classification, user settings, anti-clickjacking heuristics, or prompting the user for explicit permission. Reject if some rule is not fulfilled.
  10. Grant the document access to cookies and store that fact for the purposes of future calls to hasStorageAccess() and requestStorageAccess().

Access Removal

Storage access is granted for the life of the document and as long as the document's frame is attached to the DOM. This means:

  • Access is removed when the sub frame navigates.
  • Access is removed when the sub frame is detached from the DOM.
  • Access is removed when the top frame navigates.
  • Access is removed when the webpage goes away, such as a tab close.

In addition, the browser may decide to remove access on a timeout basis or on some dedicated user action such as switching cookie policy.

WebKit Specifics

WebKit's implementation of Storage Access API will be available in Safari Technology Preview soon and on by default. It only covers cookie access, i.e. no other storage mechanisms and the partitioning of them is affected by a call to requestStorageAccess() at this point.

@ppeg34
Copy link

@ppeg34 ppeg34 commented Jan 10, 2018

Can you please elaborate on what is meant by...

the partitioning of them is affected by a call to requestStorageAccess() at this point.

How is the partitioning of the cookies affected by calling requestStorageAccess()? Thanks in advance.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 10, 2018

Can you please elaborate on what is meant by...

the partitioning of them is affected by a call to requestStorageAccess() at this point.

How is the partitioning of the cookies affected by calling requestStorageAccess()? Thanks in advance.

That is, as you can see, a WebKit-specific detail and not really part of what we propose. But sure.

Let's say example.com has an iframe from social.org and social.org's cookies are partitioned under example.com. If the social.org iframe calls document.requestStorageAccess() upon a user gesture, and the rules say that access should be granted, the returned promise will resolve.

From that point, social.org's cookies are no longer partitioned under example.com in that iframe. Instead it has access to whatever cookies it would have if the user went directly to social.org as a first-party website.

Access goes back to its partitioned state when social.org's document in that iframe goes away or its frame is detached from the DOM.

The same would apply to any other partitioned storage that the browser supports in Storage Access API.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 10, 2018

Small note on "access to whatever cookies it would have if the user went directly to social.org as a first-party website": That obviously wouldn't apply to SameSite cookies. WebKit don't support those yet, but I wanted to mention that exception.

Loading

@dlongley
Copy link

@dlongley dlongley commented Jan 11, 2018

We ran into the partitioned storage problem when writing polyfills for the Payment Handler API and Credential Handler APIs and running them on Safari. These needed to load content via a third-party iframe that maintained previously registered payment and credential handlers that were kept track of via localStorage/IndexedDB.

When the relying party website (that loaded the polyfill iframe) attempted to request payment or credentials on Safari, the iframe would fail to see what the user previously registered -- and there was no way to fulfill the request. It sounds like this API would address the problem, which would be fantastic.

Loading

@jeisinger
Copy link
Member

@jeisinger jeisinger commented Jan 15, 2018

As I mentioned at TPAC, Chrome offers granular cookie & site data controls and UI that allow for users configuring this without any special APIs, so I'm a bit surprised that we'd need to change the spec for something that other browsers already can offer.

I'm also not sure why we'd need a separate API for this as opposed to using the permissions API.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 15, 2018

Are you saying Chrome has cookie controls that allow the user to block or partition all or some cross-origin resources’ cookies, and then allow those subresources cookie access on a per-iframe or per page basis?

Loading

@mikewest
Copy link
Member

@mikewest mikewest commented Jan 15, 2018

As I mentioned at TPAC, Chrome offers granular cookie & site data controls and UI that allow for users configuring this without any special APIs, so I'm a bit surprised that we'd need to change the spec for something that other browsers already can offer.

Browsers can offer the functionality natively, but it might be reasonable to give sites the ability to ask for more permission than they're granted as-configured. In the same way that the permission API gives sites the ability to ask for geolocation, even when the browser is configured to deny-by-default, the problem statement above posits that it might be reasonable to allow sites to ask for permission to access locally stored data, even if the existing configuration would deny that to them.

If we model storage as a permission, as @jyasskin suggested elsewhere, it doesn't seem out of the question to me that developers could call something like navigator.permissions.query({name:'first-party-storage'})) to determine whether their first-party storage is available (perhaps behind a prompt), and call something like navigator.storage.requestFirstParty() (or something similar, but navigator.storage feels like a better spot than Document if we mean for this to cover more than cookies) to cause the prompt.

Also, as @annevk noted in the DOM bug, and @raymeskhoury has noted in a Permission Delegation proposal prompting from a third-party context is hard. It's interesting to think about some of the secondary consequences of modeling this as a permission, if some browsers are shifting to a world where we deny third-party permissions by default and allow/require the parent to delegate. Requiring allow on a frame seems pretty reasonable when we're talking about additional privileges beyond what developers have traditionally expected by default (geolocation, etc). I imagine the breakage (and other side-effects, like the tacit encouragement for third-party scripts to execute in their parents' contexts) would be greater if we shifted things like storage into a deny-by-default mode.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 15, 2018

Mike, do you still think the API should be on navigator given that any access is scoped to the document?

Loading

@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Jan 15, 2018

To explain the context for this a bit more, I'd like to explain some things about Safari's current storage policy for third-party contexts, and also some model use cases for this proposed storage access API. I think some of the suggestions here may be due to lack of clarity about these things.

Safari's third-party storage policy

Safari has long partitioned or denied all types of storage except cookies, including even incidental storage like the cache, to prevent tracking via so-called super cookies like evercookie. Partitioning means keying based on both the origin of the resource and the top-level document's origin, effectively making third-party contexts see completely different storage universes on each site they were embedded into.

Cookies were an exception, for web compatibility reasons. Cookies were partially blocked via our long-standing policy that prevents setting new third-party cookies. But if a third-party cookie already existed, third-party origins could read and set it.

In 2017, we extended this to also partition cookies in third-party contexts (with a few limited exceptions). As a result, most content has access to only a partitioned view of its storage including cookies.

The reason for all this is privacy. We don't want embedded content to be able to passively track users across the web. We're most interested in preventing passive tracking, not necessarily incidental tracking when a user chooses to engage with a cross-site embed.

Use cases

  1. YouTube videos are commonly embedded in third-party sites. When the user plays a YouTube video embedded in another site, it would be good to respect their global YouTube preferences, for example caption settings. It would also be good if YouTube Red subscribers were not shown ads, as per the terms of their subscription. But if YouTube's cookies are partitioned, the video player frame cannot see the user's first party

  2. A number of sites have Facebook "like" buttons. These buttons register that a user has "liked" the article with their Facebook account but without leaving the page. If Facebook's cookies are partitioned, then FB doesn't know who the user is and has to navigate or pop up a window to ask. (Facebook "share" buttons and Twitter "tweet this" buttons are unaffected, because they always open a new first-party page on the relevant site to complete the action).

How the Storage Access API helps with cases like this

When the user clicks on a YouTube video to play it, or a Facebook like button to like an article, the embedded content may requestStorageAccess(). The API can only be called within the scope of a user action. Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient. We will prompt only if we suspect some funny business is going on, and even then we may just choose to always deny. The API is asynchronous just to enable the possibility of prompting. First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

In the examples above, this would allow YouTube to respect the user's settings, and Facebook to know who they are to register the "like". However, it would not enable passive tracking. The user has to click every time to expose first-party state.

Why some other ideas in this issue would not work

  1. Manual fine-grained cookie settings do not solve this problem. They do not enable the user to make Facebook "like" buttons and YouTube video preferences work, but without enabling these sites to passively track them even on pages where they don't engage with these features. IF the user allowed third-party cookies for youtube.com or facebook.com on all domains, this would enable passive tracking. If the user allowed third-party cookies for these domains when embedded in particular other domains, they'd have to predict exactly the sites where they will watch youtube videos or click like buttons, which is not reasonable.

  2. Treating this as a generic permission that is delegated via Feature Policy. We don't want sites to be able to grant their iframes first-party storage access by adding markup to the iframe. This would enable passive tracking even when the user does not engage. And embed providers would be incentivized to add this permission to their standard embed markup. The idea is that embeds only get to identify the user when the user engages with them.

How this might apply to other browsers

Other browsers, including Mozilla and Brave, have expressed interest in denying storage to third-party origins either by default or in optional modes. We believe this API could allow embedded gadgets to function correctly under these types of policies.

Loading

@dlongley
Copy link

@dlongley dlongley commented Jan 15, 2018

@othermaciej,

First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

Would the choice be remembered on the same site in the future without user interaction? I imagine not -- which has me wondering how this would impact embedded content that only loads after the user has engaged in some activity on the top-level domain.

For example, consider a payment handler polyfill that shows the user their registered payment handlers for selection after they've hit a pay button on the top level domain.

With storage partitioning enabled and this new API, the registered payment handlers would be inaccessible (being stored in localStorage/IndexedDB by a third-party polyfill domain) until the user clicks on the embedded content. This may be acceptable the first time it happens, by adding a one-time prompt requesting that the user enable payment on the site. But introducing an extra click to the payment process every time they buy something from the same site would be unfortunate.

It would also be unfortunate to force the top-level domain to use a specialized payment button that is served from the third-party polyfill domain as a mitigation to this problem. One of the advantages of standardizing payment on the Web is to avoid the Nascar problem and forcing top-level domains to use specialized buttons.

I wonder if some kind of one-time use "delegated user interaction object" could be passed to the iframe via postMessage if specifying delegated permissions in the iframe markup is considered unacceptable. That sort of primitive may also be helpful/reusable in other situations on the Web platform, particularly those where a Promise must resolve prior to "consuming" a user interaction to call some guarded API.

Loading

@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Jan 15, 2018

Would the choice be remembered on the same site in the future without user interaction?

No.

I imagine not -- which has me wondering how this would impact embedded content that only loads after the user has engaged in some activity on the top-level domain.

I don't think it would help in such cases (unless the user clicks again on the embedded content).

For example, consider a payment handler polyfill that shows the user their registered payment handlers for selection after they've hit a pay button on the top level domain. With storage partitioning enabled and this new API, the registered payment handlers would be inaccessible (being stored in localStorage/IndexedDB by a third-party polyfill domain) until the user clicks on the embedded content.

Do you know a real site or embedded gadget that actually works this way? It would specifically need to:

  • Load an iframe after the user initially clicks.
  • Show a set of payment providers based on info specific to the user, with that set being accessed from the iframe.
  • Not involve any clicks prior to selecting a payment provider in its normal flow.
  • Not make the initial payment button itself an iframe.

I don't know of any setups like this, but if there are, we can certainly think about opportunities to help them in the face of limitations on third-party storage.

Loading

@dlongley
Copy link

@dlongley dlongley commented Jan 16, 2018

@othermaciej,

Other than the payment handler and credential handler polyfills -- I don't know of any at the moment. There may be some some third-party payment providers that provide this kind of flow, but I'm unaware of them.

This is the payment handler polyfill demo that was presented at TPAC that I'm talking about (note that this won't run in Safari because of this very issue, but will run in Chrome, Firefox, and Edge):

You install a payment handler and add payment instruments (credit cards) here at a "wallet" website:

https://payment-handler.demo.digitalbazaar.com/

And then you use the registered handler here via a payment instrument of your choice here at a "merchant" website:

https://payment-merchant.demo.digitalbazaar.com/

This follows the flow you described above.

Loading

@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Jan 16, 2018

It does seem like that demo would need to be modified even with this new API in place. It will either need an extra click to look up the default payment method, or the buy button itself needs to be an iframe.

Since this demo is not (as far as I know) in wide actual use, I am not sure it should be a priority to design around its requirements rather than to look at ways to modify the demo.

Loading

@mikewest
Copy link
Member

@mikewest mikewest commented Jan 16, 2018

Thanks, @johnwilander and @othermaciej for the detailed background. That's helpful information. I'll pull out a few points below:

Mike, do you still think the API should be on navigator given that any access is scoped to the document?

If we model things as a permission, then I think hooking into the existing navigator.permissions and navigator.storage makes more sense than adding one-off methods to Document. I'd also push back a bit on the notion that "access is scoped to the document", as, at least for cookies, HttpOnly cookies wouldn't be accessible to the document, but instead sent on requests to the origin via HTTP requests which initiate from the document (or, presumably, from its parent?). Ideally, those cookies would never be exposed to the document context.

When the user clicks on a YouTube video to play it, or a Facebook like button to like an article, the embedded content may requestStorageAccess(). The API can only be called within the scope of a user action.

I think this addresses some of the use cases you noted above, while partially addressing others. Video sites are good examples of embeds that need to make decisions about what content to show a given user prior user interaction in order to comply with a user's preferences regarding mature content (Vimeo's viewing preferences, YouTube's restricted mode, etc.). It seems like this mechanism might push some of those sites into a client-side rendering model whereby content is locked behind a clickthrough that calls out to the mechanism y'all are proposing before rendering anything. That seems doable from a technical perspective, but would have performance (and aesthetic!) impacts on users.

Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient. We will prompt only if we suspect some funny business is going on, and even then we may just choose to always deny. The API is asynchronous just to enable the possibility of prompting. First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

All of these restrictions would be compatible with treating storage access as a permission that the browser makes decisions about whether to grant or restrict, with or without user interaction. The scoping and persistence is a bit different than we've done with other permissions in the past (and different still from what Chrome folks are proposing), but the core notion of asking first, and then doing, seems like a reasonable fit.

In the examples above, this would allow YouTube to respect the user's settings, and Facebook to know who they are to register the "like". However, it would not enable passive tracking. The user has to click every time to expose first-party state.

It's somewhat tangential to the core proposal, but I'd suggest that y'all will want to consider whether or not the user gesture is consumed by the call to obtain first-party storage. I know that Chrome's gesture implementation has sometimes caused developers some annoyance by disallowing combinations of actions with a single gesture. I can imagine that applying here as well: perhaps a widget would ask for credentials on click, and use them to make a decision about whether to call window.open() (which I believe Safari also gates on a gesture requirement).

Manual fine-grained cookie settings do not solve this problem. They do not enable the user to make Facebook "like" buttons and YouTube video preferences work, but without enabling these sites to passively track them even on pages where they don't engage with these features.

Chrome's content settings are certainly capable of making this distinction between "B in A" and "B in C" (see the Primary and Secondary Patterns section of the chrome.contentSettings extension API docs, for example), and allows the user to do so in certain cases via the blocked-cookie UX in our omnibox. I suspect we could do a better job exposing those granular options to the user in our settings UI, but that in some ways comes back to third-party contexts being difficult to expose to users in a way that they can be expected to generally understand.

IF the user allowed third-party cookies for youtube.com or facebook.com on all domains, this would enable passive tracking. If the user allowed third-party cookies for these domains when embedded in particular other domains, they'd have to predict exactly the sites where they will watch youtube videos or click like buttons, which is not reasonable.

I know that some users (like @jeisinger above :) ) have developed workflows whereby they toggle Chrome's "Block third-party cookies and site data" option, and then make use of the omnibox UX and the reload button to fix sites after visiting them. It seems to me that that's not a terrible model.

Treating this as a generic permission that is delegated via Feature Policy. We don't want sites to be able to grant their iframes first-party storage access by adding markup to the iframe. This would enable passive tracking even when the user does not engage. And embed providers would be incentivized to add this permission to their standard embed markup. The idea is that embeds only get to identify the user when the user engages with them.

Another way of looking at the same mechanism would be that the top-level document would be able to use Feature Policy to deny access to certain permissions to its children by fiat, the embeddee's wishes notwithstanding.

Moreover, even in a fully realized delegation model, the browser remains in a position of mediating the permission requests, and could certainly make a decision about whether to grant permissions on the basis of information to which the page isn't privy. The permission model means that the request would fall into a well-paved API path for developers, not that you'd be locked into adhering to a page's desires without the potential for override.

Other browsers, including Mozilla and Brave, have expressed interest in denying storage to third-party origins either by default or in optional modes. We believe this API could allow embedded gadgets to function correctly under these types of policies.

/cc @TanviHacks, @dveditz, @diracdeltas from those browsers for feedback. Also tagging in @patrickkettner to get Edge in the loop.

Loading

@diracdeltas
Copy link

@diracdeltas diracdeltas commented Jan 17, 2018

Hi, Yan from Brave here! Brave does indeed block 3rd party storage (cookies, localStorage, etc.) and referrer by default.

Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient.

We (Brave) would almost certainly want to prompt by default with a 'never prompt again' option in the permissions UX. I think this API would be useful but we would want to be careful to not make our current cookie restrictions more lax without alerting the user.

Loading

@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Jan 17, 2018

@diracdeltas The API is designed to allow all the prompting you want (since it's async) so it would be easy to fulfill it with a prompt-always (or prompt-once-per-origin-pair) policy.

Loading

@hillbrad
Copy link

@hillbrad hillbrad commented Jan 17, 2018

Yes, it would be great to allow for never prompting again. Many integrations that involve data sharing are moving towards a more granular model of permissions, with an emphasis on prompting only in context and as needed, rather than asking for all permissions that might ever be needed up front. The more friction the browser adds to this, the less likely sites are to adopt these granular and just-in-time models.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 17, 2018

From today's W3C WebAppSec call, as I interpreted it:

  • Mozilla is “cautiously interested” in implementing. They will monitor the uptake of the API in Safari and if it seems to work, they will try it out.
  • Edge said their status is similar to Mozilla’s.
  • Chrome has no plans but might implement in Chromium for other ports, especially if this gets standardized.

Loading

@diracdeltas
Copy link

@diracdeltas diracdeltas commented Jan 17, 2018

@johnwilander For now, Brave's status is that we would use this if it were already implemented in Chromium. We could be interested in implementing it independent of Chromium depending on adoption.

Loading

@mikewest
Copy link
Member

@mikewest mikewest commented Jan 18, 2018

Chrome has no plans but might implement in Chromium for other ports, especially if this gets standardized.

What I hope I suggested on the call was that I don't think Chrome has a concrete interest in implementing this, but that if a Chromium port (like Brave) wanted to upstream a patch to reduce their maintenance burden, I'd be happy to help them do so.

The WebAppSec discussion's draft minutes are available at https://www.w3.org/2018/01/17-webappsec-minutes.html#item04 if folks here are interested in the points raised there. @dveditz and I re-raised the suggestion that this might fit into the Permissions API rather than building a new one-off on Document, and @jyasskin suggested that if the Permissions API spec contains restrictions on the permission model that make it incompatible with this usage, he'd see it as a bug and fix it.

Loading

@annevk
Copy link
Member

@annevk annevk commented Jan 18, 2018

The "has" method can be modeled as a permission I think. "Request" not so much.

FWIW, there's quite a few people within Mozilla that have a problem with prompts on behalf of third-parties and I'm not really sure how to reconcile that position with this API.

Loading

@mikewest
Copy link
Member

@mikewest mikewest commented Jan 18, 2018

The "has" method can be modeled as a permission I think. "Request" not so much.

I agree. One of the earlier suggestions in this thread was to tie the request bit to a new method on navigator.storage rather than Document.

FWIW, there's quite a few people within Mozilla that have a problem with prompts on behalf of third-parties and I'm not really sure how to reconcile that position with this API.

That's also something Chrome folks are reluctant to do more of.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Jan 18, 2018

As for prompting, it’s optional in the proposal. Are you saying you’re against optional prompting too? Brave expressed that they’ll definitely prompt. We want the ability if we see abuse of the API down the road.

Loading

@dlongley
Copy link

@dlongley dlongley commented Dec 17, 2019

In this thread, there's a suggestion that Safari may be considering making all popups run in a 3rd party context:

@snyderp points out that Safari may be taking steps to make popups default to 3P context.

If this is done, I'm concerned it will make it impossible for flows whereby the user never visits a certain website in a first party context but that site needs unpartitioned storage. This flow is currently possible by using a popup to cause the user to visit the site and set a first party cookie -- and, subsequently, the Request Storage Access API can be successfully called from the 3rd party context.

The W3C Credentials Community Group has a "credential handler polyfill" work item that is relying upon this approach to experiment with future standard API. With a standard API the above flow could be avoided. However, disabling this experimentation would make it impossible to demonstrate the appropriate levels of incubation and community consensus that is often requested by browser manufacturers to start standardizing -- thus creating a chicken/egg problem.

Loading

@pes10k
Copy link

@pes10k pes10k commented Dec 17, 2019

Just to be clear, I don't work on Safari or WebKit. The above was just my observation last time I tested (and its not impossible I did something wrong there).

Loading

@dlongley
Copy link

@dlongley dlongley commented Dec 17, 2019

@snyderp -- Understood. Thanks for the information and bringing more attention to it, should it be the case.

Loading

@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Dec 17, 2019

@johnwilander can answer questions about his, but I think it’s off-topic for this issue. If no already appropriate issue, and on Twitter or by email on webkit-dev@webkit.org

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Dec 17, 2019

@johnwilander,

Small note on "access to whatever cookies it would have if the user went directly to social.org as a first-party website": That obviously wouldn't apply to SameSite cookies. WebKit don't support those yet, but I wanted to mention that exception.

We shipped support for SameSite cookies in September 2018 and support for SameSite=none in September this year (2019).

It seems that cookies set with SameSite=None are not available in Safari once first party storage access has been granted. I realize that you said Safari doesn't support SameSite cookies yet (as of Jan 2018 anyway), but does that mean that Safari will block access to cookies with the SameSite attribute regardless of the attribute's value? If so, is that intentional?

If it's just an artifact of not supporting SameSite cookies, could Safari be updated to at least treat None values as if SameSite were not present when determining storage access?

Given the information about Safari's support for SameSite cookies, are you seeing unexpected behavior with the Storage Access API?

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Dec 17, 2019

In this thread, there's a suggestion that Safari may be considering making all popups run in a 3rd party context:

@snyderp points out that Safari may be taking steps to make popups default to 3P context.

I was alerted to this late yesterday and I'm still reading up on that thread. That sentence on its own does not make sense to me so either I'm missing some context or there is a misunderstanding of our plans or beta functionality.

If this is done, I'm concerned it will make it impossible for flows whereby the user never visits a certain website in a first party context but that site needs unpartitioned storage.

This is already the case and has been since 2013. Third-parties cannot set cookies unless they have already set cookies as first parties, third-party LocalStorage is permanently partitioned, and IDB is blocked for third-parties.

This flow is currently possible by using a popup to cause the user to visit the site and set a first party cookie -- and, subsequently, the Request Storage Access API can be successfully called from the 3rd party context.

Yes, this is a supported flow, although I'd point out that "popups" don't really exist on the mobile web since they're just tabs there.

Successfully using the Storage Access API is gated on 1) previous user interaction as first-party, 2) current user interaction as third-party (in the iframe), and 3) user opt in if there's a prompt.

The W3C Credentials Community Group has a "credential handler polyfill" work item that is relying upon this approach to experiment with future standard API. With a standard API the above flow could be avoided. However, disabling this experimentation would make it impossible to demonstrate the appropriate levels of incubation and community consensus that is often requested by browser manufacturers to start standardizing -- thus creating a chicken/egg problem.

With the information provided above, do you still have concerns about the utility of the Storage Access API?

Loading

@dlongley
Copy link

@dlongley dlongley commented Dec 18, 2019

@johnwilander,

Given the information about Safari's support for SameSite cookies, are you seeing unexpected behavior with the Storage Access API?

At the time (October 13rd, 2019) that I wrote this:

It seems that cookies set with SameSite=None are not available in Safari once first party storage access has been granted.

I was seeing that behavior. I'd have to go back and see what version of Safari I was experiencing the issue on and try again. At the time, I was trying to resolve warnings from Chrome about future blocking of cross-domain cookies not marked with SameSite=None and Secure. Adding these cookie attributes fixed the Chrome warning but caused Safari to break. The behavior was as I quoted above -- setting a SameSite=None and Secure cookie via a first party context followed by a call to requestStorageAccess() in a third party context failed to grant access to the cookie. Removing those attributes from the cookie restored the functionality.

This is already the case and has been since 2013. Third-parties cannot set cookies unless they have already set cookies as first parties, third-party LocalStorage is permanently partitioned, and IDB is blocked for third-parties.

Right, I didn't mean to imply anything different, apologies for any confusion there.

This flow is currently possible by using a popup to cause the user to visit the site and set a first party cookie -- and, subsequently, the Request Storage Access API can be successfully called from the 3rd party context.

Yes, this is a supported flow, although I'd point out that "popups" don't really exist on the mobile web since they're just tabs there.

Yes, this "popup" (or "tab") flow is vital to our use case and I was concerned it may be in jeopardy. It's currently the only means that works for us by which user interaction can be captured and a first party cookie can be set to allow subsequent storage access. But, if you're looking for feedback, we'd probably be happier with a direct user prompt when we request storage access w/o a previous first party visit... rather than sending the user on this little quest.

With the information provided above, do you still have concerns about the utility of the Storage Access API?

No, since you said the flow we're using continues to be a supported flow I'm not concerned. Thank you for your time and responses!

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Dec 18, 2019

@johnwilander,

Given the information about Safari's support for SameSite cookies, are you seeing unexpected behavior with the Storage Access API?

At the time (October 13rd, 2019) that I wrote this:

It seems that cookies set with SameSite=None are not available in Safari once first party storage access has been granted.

I was seeing that behavior. I'd have to go back and see what version of Safari I was experiencing the issue on and try again.

SameSite=none is supported from Safari 13 on macOS and Safari on iOS/iPadOS 13. Versions prior to that interpret none as an unknown value and defaults to SameSite=strict. A strict SameSite cookie will not be available in a third-party iframe regardless of calls to the Storage Access API so this might have been what you saw.

At the time, I was trying to resolve warnings from Chrome about future blocking of cross-domain cookies not marked with SameSite=None and Secure. Adding these cookie attributes fixed the Chrome warning but caused Safari to break. The behavior was as I quoted above -- setting a SameSite=None and Secure cookie via a first party context followed by a call to requestStorageAccess() in a third party context failed to grant access to the cookie. Removing those attributes from the cookie restored the functionality.

This is already the case and has been since 2013. Third-parties cannot set cookies unless they have already set cookies as first parties, third-party LocalStorage is permanently partitioned, and IDB is blocked for third-parties.

Right, I didn't mean to imply anything different, apologies for any confusion there.

No worries. We get a lot of questions on these things so I just wanted to be clear.

This flow is currently possible by using a popup to cause the user to visit the site and set a first party cookie -- and, subsequently, the Request Storage Access API can be successfully called from the 3rd party context.

Yes, this is a supported flow, although I'd point out that "popups" don't really exist on the mobile web since they're just tabs there.

Yes, this "popup" (or "tab") flow is vital to our use case and I was concerned it may be in jeopardy. It's currently the only means that works for us by which user interaction can be captured and a first party cookie can be set to allow subsequent storage access. But, if you're looking for feedback, we'd probably be happier with a direct user prompt when we request storage access w/o a previous first party visit... rather than sending the user on this little quest.

It's a deliberate design choice of the Storage Access API to require prior user interaction as first party. We don't want the tens or hundred third-parties on websites to be able to prompt the user for storage access and users will be very unlikely to recognize a domain name that they have never interacted with while visible in the URL bar.

With the information provided above, do you still have concerns about the utility of the Storage Access API?

No, since you said the flow we're using continues to be a supported flow I'm not concerned. Thank you for your time and responses!

You're welcome!

Loading

@dlongley
Copy link

@dlongley dlongley commented Dec 19, 2019

SameSite=none is supported from Safari 13 on macOS and Safari on iOS/iPadOS 13. Versions prior to that interpret none as an unknown value and defaults to SameSite=strict. A strict SameSite cookie will not be available in a third-party iframe regardless of calls to the Storage Access API so this might have been what you saw.

Given that Safari 13 was only released a few weeks prior to my testing, this is almost certainly true. Thanks.

It's a deliberate design choice of the Storage Access API to require prior user interaction as first party. We don't want the tens or hundred third-parties on websites to be able to prompt the user for storage access and users will be very unlikely to recognize a domain name that they have never interacted with while visible in the URL bar.

This explanation makes a lot of sense, thank you.

Loading

@jyasskin
Copy link
Member

@jyasskin jyasskin commented Dec 19, 2019

Given that w3c/payment-handler#351 (comment) hopes to call into the requestStorageAccess() algorithm, I'd like to encourage the folks here to write a PR and get it into HTML.

Loading

@hober
Copy link
Collaborator

@hober hober commented Feb 4, 2020

The @privacycg has adopted this as a work item; instead of trying to cover all of the issues with the API in this one issue, everyone can now file issues over there: https://github.com/privacycg/storage-access/issues

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Feb 5, 2020

👆🏼What Tess said. We've moved this to the W3C Privacy CG where you can file individual issues on the things you want to discuss: https://github.com/privacycg/storage-access

Thanks for all the engagement we got on this lengthy issue! I will now close it. Eventually we should expect a pull request on the HTML spec.

Loading

@annevk
Copy link
Member

@annevk annevk commented Feb 5, 2020

Reopening as WHATWG likes to keep track of activity that will eventually be upstreamed through open issues. Hope that's understandable.

Loading

@annevk annevk reopened this Feb 5, 2020
@othermaciej
Copy link
Collaborator

@othermaciej othermaciej commented Feb 5, 2020

Maybe a better approach would be to edit the initial comment to prominently point to the privacycg issue tracker where work is happening for now?

Loading

@annevk
Copy link
Member

@annevk annevk commented Feb 5, 2020

Done.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Feb 5, 2020

Thanks, Anne!

Loading

@valipopescu
Copy link

@valipopescu valipopescu commented Sep 28, 2021

After reading all of the above specification and pondering on its effects on real applications I think that if one needs non ephemeral non partitioned local storage for any purpose other thank tracking there's no real way of having it granted even if the user would otherwise want to grant it. LocalStorage purpose is to save large amounts of data and have the data not leave the browser (be it for logistical purposes or security for that matter ... and I won't go in details on how important is the latter...) There's a plethora of applications that work based on this presumption ... say for example an application that has multiple subdomains and users have copyrighted art they work on with various tools presented to them on different websites. The purpose is that the data never leaves their browser so the data is not copied nor exposed to any party attempting to copy it, not to mention that such data is large enough to begin with that it would not make sense to send over the wire.

For all of the above and the presented example, I think that the only proper implementation is the one made in Firefox meanwhile the Safari version is at the very least abysmally unthought about ... if there's anything nice to say about it ...

And to make things even worse the Firefox implementation already has other security measures and partitioning in place the (read here TLS Security keys, HSTS etc ) which I don't know whether Apple and your Safari (speaking mainly to you @johnwilander) has even had a glimpse about, so the only thing Safari stops is properly implemented applications that have the security of their users at heart meanwhile protecting shamelessly the very thing they claim to work against.
If you need I can give you enough examples of how actual trackers can circumvent your implementation but I doubt you can find proper resolution to my example and a few others I won't go into detail here to not be the cause of any serious security outbreaks.

In all honesty I think that at the very least this initiative should be approached with giving the user the ability to grant localStorage access even if it is after being prompted for it.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Sep 28, 2021

Hi @valipopescu!

First of all, the Storage Access API is worked on in the W3C Privacy CG since a couple of years: https://github.com/privacycg/storage-access. It'll eventually find its way into specs such as HTML but the Privacy CG is the right place to discuss it.

Second, in standards repos like this, we try not to discuss browser-specific things that are not proposed standards. You can file a bug with the particular browser or engine if you want to raise such issues.

Finally, I believe you are mistaken on WebKit's partitioning and tracking prevention, based on what you write above. Please see https://webkit.org/tracking-prevention/.

Loading

@valipopescu
Copy link

@valipopescu valipopescu commented Sep 28, 2021

Hi @johnwilander

Thanks for the swift response, which I highly appreciate. I haven't been able to find a way to report the implementation of the Tracking prevention as you call it in webkit, as a bug per se. Since I found that you were the proposal main as well as responsible for the webkit tracking prevention, would appreciate a way for us to present in detail what's going on and maybe you can suggest the way Apple Webkit has thought about how to handle such situations. This is once again related to actual user security rather than tracking.

Loading

@johnwilander
Copy link
Author

@johnwilander johnwilander commented Sep 28, 2021

Hi @johnwilander

Thanks for the swift response, which I highly appreciate. I haven't been able to find a way to report the implementation of the Tracking prevention as you call it in webkit, as a bug per se. Since I found that you were the proposal main as well as responsible for the webkit tracking prevention, would appreciate a way for us to present in detail what's going on and maybe you can suggest the way Apple Webkit has thought about how to handle such situations. This is once again related to actual user security rather than tracking.

You are always welcome to file bugs at https://bugs.webkit.org and CC me. Please use the Security bug category for privacy/security bugs. Thanks!

(The other engines have open source bug trackers too in case you wanted to file for them too.)

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet