Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Localstorage Support #44

Closed
mrdokenny opened this issue Jun 14, 2017 · 98 comments

Comments

@mrdokenny
Copy link
Member

commented Jun 14, 2017

Follow up issues for IndexDB and other types of storage when we get there.

Future Solutions

Proper API for deleting Localstorage by hostname (both of these bugs seem to want to extend the browsingData API)

Pros

  • Clear localstorage by site rather than all or nothing
  • Should be easily added to the extension code since it seems like the extended API uses a list of hostnames which Cookie Cleanup conveniently returns

Cons

  • No support yet
  • Unless there's a way to enumerate all localstorage, there's a way for this extension to miss some localstorage
    • Ex. A site that uses no cookies and only localstorage (as cookie cleanup outputs site names for cookies that have been deleted to be sent into the localstorage cleanup)
      • I find this very unlikely since almost all sites have cookies
    • Ex. A current Cookie AutoDelete user who upgrades to a version that supports deleting localstorage will have their existing localstorage still there
      • The user would have to clear all their localstorage after upgrade to get a "clean slate"

Firefox Blockers

Chrome Blockers

Current solutions

browsingData API

Pros

  • Actual API for removing localstorage

Cons

Firefox Blockers

Inject a content script that clears localstorage on page load (courtesy of reddit)

Code:

window.localStorage.clear();
window.sessionStorage.clear();

Pros

  • Works in all browsers
  • Better than the current browsingData API
  • Will work with the whitelists

Cons

  • Seems very hackish and its only a workaround
  • Localstorage stays until the user visits the website again
  • Probably have to come up with an algorithm to find the diff of the current tabs (between page loads) and only inject if there is a change (otherwise the injection would happen every time on site navigation)
    • Because of this, this workaround will be on the backburner until I have some strategies about the best way to only inject the content script when it is needed

I'm thinking of doing the content script route as an experimental feature until the browsingData gets extended. Anyone else is free to give their input on how best to implement this.

@crssi

This comment has been minimized.

Copy link

commented Jun 16, 2017

The workaround you found seems quite OK, at least until Mozilla does not sort out the API.
Do you have any ETA for the workaround to be implemented?

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 16, 2017

@crssi Probably once #20 is done and stable so hopefully by the end of summer. Things are subject to change like the workaround might not work as well as I thought but we'll see.

@Stebalien

This comment has been minimized.

Copy link

commented Jun 18, 2017

The tricky part here will be figuring out when to clear local storage. The content script will have to ask the backend if/when it should clear localStorage before it lets the rest of the page load. Unfortunately, you can't use a port as ports are asynchronous.

The best method I can think of is to inject a "clear-localStorage" cookie with onHeadersReceived and then read/delete it from the content script.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 18, 2017

@Stebalien I was planning on doing tabs.executeScript() at document_start and see how well that does.

Edit: There also the problem of injecting the same content script multiple times on a new page load on the same website.

@Stebalien

This comment has been minimized.

Copy link

commented Jun 18, 2017

Where will you call that function? Unless I'm mistaken, the only place one can synchronously "catch" a page load (pause it while you do something) is from onHeadersReceived and I'm not sure if you can programatically inject scripts at that point.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 18, 2017

There's the tabs.onUpdated event which I use to update the icon and calculate the number of cookies for that site.

@Stebalien

This comment has been minimized.

Copy link

commented Jun 18, 2017

I'm pretty sure that event is asynchronous so there's no way to guarantee that the content script will get injected before the page loads.

@ghost

This comment has been minimized.

Copy link

commented Jun 27, 2017

Ex. A site that uses no cookies and only localstorage (as cookie cleanup outputs site names for cookies that have been deleted to be sent into the localstorage cleanup)

I find this very unlikely since almost all sites have cookies

Not frequent indeed yet sites using only localStorage do exist.
I have two in mind because bookmarked, the first is modest, the second is far more important since it is a search engine: ClockTab and Qwant

Cleaning the localStorage is indeed as imperious as cleaning cookies, for non white-listed sites of course.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 27, 2017

@zymase The only way I see this happening for those sites without an API to enumerate localstorage is to store the store the current domains in memory and find the diff between sites on page load. Another option that could work is to artificially set a temporary cookie, so that the cleanup would still get the hostname of the site. Personally 2nd one might be better for performance,

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@mrdokenny I know the idea is to be constructive, always. I'm not at all into coding but from what I read concerning the possibility to read/analyze/edit localStorage with a Webextension it seems to be Mission:Impossible if the proper API is not made available. From there on, considering Mozilla's commitment to privacy, considering the highly privacy concern related to localStorage, I remain reasonably optimistic that Mozilla will work on that API. Should I be wrong, should Firefox 57 appear to consider localStorage as an exotic quest not worth being taken care of that I'd remain deaf to the company's credo of users' privacy and consider definitely an alternative to Firefox. Period.

If programmers in the light of Webextensions are required to support extensions as Atlas the weight of the planet on his shoulders then where are we leading to? Counter-progress? For the sake of what, "universal" extensions valid on all platforms on the basis of a leverage on the cheapest possibilities granted to extensions?

Good luck to all, and to extension programmers in particular, they'll need it.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

Is it possible to clear all local storage on startup?

SDC is able to do it even with e10s enabled.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@zeepob , the Firefox SDC (Self-Destructing Cookies) add-on's developer faces the same problems:

Q: Will this add-on ever be multi-process (e10s) compatible?
A: Add-ons can't monitor sites' LocalStorage usage in e10s mode. This functionality will probably never be restored for legacy add-ons such as SDC. This means that the answer is "very likely never". You can still force-enable e10s and SDC should clean your cookies just fine, but it can only clean your LocalStorage when the browser starts.

Q: Will this add-on make the jump to the WebExtension world?
A: I don't have the time for a full rewrite as a WebExtension. Enjoy it while it lasts.

As explained on the add-on's AOM page

The Cookie AutoDelete add-on handles e10 I think, SDC doesn't, at least not completely.
Cookie Autodelete handles localStorage, SDC doesn't

Cookie Autodelete is a WebExtension, SDC is not and will likely never be.
I'm still running SDC (on Firefox ESR 52.2.0) but one day or another (at latest at the EOL of FF52ESR) SDC will be obsolete, so the target is Cookie Autodelete and the wish is to have it handle localStorage.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@zymase I don't understand your answer.

SDC can't clear individual site local storage when e10s is enabled but it can clear local storage globally on startup. My question is if it's technically possible for Cookie Autodelete to do.

That ability is clear advantage of SDC running on latest firefox versions.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@zeepob ,

SDC can't clear individual site local storage when e10s is enabled but it can clear local storage globally on startup.

That is correct. That is why I added "at least not completely.". I block e10 altogether here and now so I don't have to face the e10 restrictions when it comes to e10 incompatibility.

My question is if it's technically possible for Cookie Autodelete to do

SDC's developer says it's not possible. I have no idea, not being a coder myself.
In fact it appears that Cookie Autodelete is facing two walls when it comes to handling localStorage: the WebExtension format and the e10 implications. Need I say both bother me and many of us?

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@zymase

SDC's developer says it's not possible

Well, you just quote him saying that's possible with e10s:

You can still force-enable e10s and SDC should clean your cookies just fine, but it can only clean your LocalStorage when the browser starts.

I wonder if it's possible for webextension too.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@zeepob , I quoted above SDC' developer:

Add-ons can't monitor sites' LocalStorage usage in e10s mode. This functionality will probably never be restored for legacy add-ons such as SDC. This means that the answer is "very likely never". You can still force-enable e10s and SDC should clean your cookies just fine, but it can only clean your LocalStorage when the browser starts.

Seems explicit to me.
This functionality will probably never be restored for legacy add-ons such as SDC.
The point is to know if this functionality will or will not be made available on Webextensions by means of a dedicated API. Clear enough?

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

It's very explicit and contradicts you saying that's not possible. I understand you have no knowledge to answer my question so please refrain from posting unrelated answers to me. Thanks.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

I tried to bring my contribution to your question by referring to what is known. I linger to find a contradiction when emphasizing on what the developer of an add-on similar to Cookie Auto delete wrote, explicitly. To force enable e10 which will clean cookies but will clean localStorage only when browser starts is not what I call monitoring localStorage, neither is it SDC's developer opinion when he states "Add-ons can't monitor sites' LocalStorage usage in e10s mode. "

Is it a language problem of that of basic logic and understanding? Good luck.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

Maybe it's language. In my every post I was crystal clear that I talk about:

clearing all local storage on startup

Monitoring local storage and clearing it per domain is out of my question. Thank you.

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

OK; @zeepob I have to agree that indeed your quest concerned clearing all local storage on startup
I will have missed that probably because, if it is for you a point of interest it is for me so far from what I expect from localStorage monitoring that unconsciously I misunderstood. From there on arguments mismatched. Neither a language nor a logic problem, obviously a wrong dialog triggered by an initial psychological bias.

I think we got it clear now :)

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 28, 2017

@zymase

"Add-ons can't monitor sites' LocalStorage usage in e10s mode. "

That's only for legacy extensions because Mozilla broke the XUL API for monitoring localstorage when they were implementing e10s and don't want to fix it because they were moving on to WebExtensions.

See 1130859 and 1043081. It's not that with e10s extensions can't clear localstorage ever again, because there is an API to do so, but it's too general of an approach.

@zeepob

Is it possible to clear all local storage on startup?

Yes this should be possible with the general API (browsingData).

How the browsingData works currently is you pass in which type of storage you want to clean and the removal options. The only problem is that it only has the since property which means that I could pass 0 in it and it would clear all localstorage (as well as any other data that I specify).

What I want is another property hostname that I can pass in to delete data by site rather than only by time. The proper API is nice but this is probably the best for right now.

@spinda

This comment has been minimized.

Copy link

commented Jun 28, 2017

Just so everyone is on the same page now: what you need is a hostname option to be added to browsingData's removal options, and that's it?

@ghost

This comment has been minimized.

Copy link

commented Jun 28, 2017

@mrdokenny thanks for the reply.

So what do you think about adding localstorage clearing at startup as opt-in feature? I think it would be better than nothing as currently we have.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Jun 29, 2017

Just so everyone is on the same page now: what you need is a hostname option to be added to browsingData's removal options, and that's it?

@spinda Yes

Also one interesting to note that that Chrome bug is 6 years old by now, but hopefully Mozilla won't take that long.

So what do you think about adding localstorage clearing at startup as opt-in feature? I think it would be better than nothing as currently we have.

@zeepob They haven't added localstorage cleanup to browsingData yet. The related bug is under Current Solutions ->browsingData API -> Firefox Blockers and it's getting some activity from what I see. :)

@willsALMANJ

This comment has been minimized.

Copy link

commented Jun 29, 2017

Just following up on @zeepob's request -- I want to transition from SDC to a WebExtension (so thank you for working on this!). What practice would you recommend in the mean time to preserve privacy at a level similar to SDC? If I clear "Offline website data" in the "Clear history" option in Preferences->Privacy, will that clear localStorage and indexDB? I could do that by hand periodically (not quite as good as SDC doing it when a tab closes but close).

Or maybe I should just keep using SDC for now and wait for 1355576 so you can implement @zeepob's suggestion.

@ihateregs

This comment has been minimized.

Copy link

commented Dec 10, 2017

The current title of Bug 1329745 is "WE API to add/change localStorage items on a per-site basis" and the last comment is "Renaming, because removing is possible." So is this a wrong link to describe possibility of enumerating?

@curiosity-seeker

This comment has been minimized.

Copy link

commented Dec 16, 2017

Can we do a full cleanup from inside Firefox (or maybe from the profile folder)? A delete all is probably an easy way out for most people.

I agree. The StoragErazor add-on claims that it "automatically removes data stored in DOM Storage (local storage) and IndexedDB when the browser restarts". This might be a way to go.

@Kagami

This comment has been minimized.

Copy link

commented Dec 16, 2017

automatically removes data stored in DOM Storage (local storage) and IndexedDB when the browser restarts

I think this contradicts with the purpose of the extension - delete cookies/data that you don't need. Obviously I don't want to remove data of the sites which I reguarly use.

@EchoDev

This comment has been minimized.

Copy link

commented Dec 16, 2017

I think this contradicts with the purpose of the extension - delete cookies/data that you don't need. Obviously I don't want to remove data of the sites which I reguarly use.

Ofcourse but it is a response to "It will not clear localstorage that was previously there". CAD won't be able to delete old data with one click, a complete delete should give us the option to start with a fresh/clean localstorage and start whitelisting/blacklisting from there.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Dec 24, 2017

I uploaded 2.1.0b1 to the AMO beta channel which has localstorage support.

Some notes:

  • The API for localstorage cleaning doesn't appear to support Containers. So if cookies from one container are cleared, then all of the site's localstorage from all containers are deleted regardless of whitelisting rules.
    • You are still able to enable both Containers and Localstorage cleaning but a warning will appear.
    • TBH, I haven't noticed anything major with both of them on, but still better to put a warning there.
  • Due to the way this extension works, I am placing a temporary cookie on sites that have no cookies if you have the localstorage setting on.
    • The cookie's name and content are very obvious to what the purpose is.
    • Fortunately, not many sites only use localstorage, so this shouldn't be a major issue.
      capture
  • This can't clear localstorage that was there previously
    • I'll eventually add a popup action that clears all of them manually or you can use something like StorageErazor

Some Test sites

A test site that uses only localstorage and no cookies:
https://mdn.github.io/dom-examples/web-storage/

Soundcloud uses localstorage for the volume control.
https://soundcloud.com/

@ruv

This comment has been minimized.

Copy link

commented Dec 24, 2017

BYW, why don't set Path to some special value to avoid sending this CookieAutoDelete cookie on each request to the given host?
For example
Path: /cookie-for-localstorage-cleanup

@grenzor

This comment has been minimized.

Copy link

commented Dec 24, 2017

Due to the way this extension works, I am placing a temporary cookie on sites that have no cookies if you have the localstorage setting on.

How does this affect users who would like to use something like #95?

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Dec 28, 2017

@grenzor

How does this affect users who would like to use something like #95?

My other option is to store which websites you visited and pass that along to the localstorage API, which might not be ideal privacy-wise.

@anewuser

This comment has been minimized.

Copy link

commented Feb 5, 2018

Using a fixed path name makes it easy to fingerprint Cookie AutoDelete users.

Can you set it to a different random string every time a new dummy cookie is created?

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Feb 6, 2018

Since localstorage "support" is there, FF version 58 is out, and this thread is getting long, I'm going to close this issue. Any future issues with localstorage should be new issue. Thanks for the support.

@zero77

This comment has been minimized.

Copy link

commented Feb 6, 2018

@mrdokenny
Was this thread just for FF and not Chrome.

@mrdokenny

This comment has been minimized.

Copy link
Member Author

commented Feb 7, 2018

@zero77 Technically there is "Clear localstorage for this domain" in the "Clean" dropdown menu of the popup. This does not require an API since CAD injects a content script, so it works in Chrome.

I also doubt that this bug will be resolved anytime soon.

https://bugs.chromium.org/p/chromium/issues/detail?id=78093

@zero77

This comment has been minimized.

Copy link

commented Feb 7, 2018

I see, ok thanks.

@zero77

This comment has been minimized.

Copy link

commented Feb 12, 2018

@mrdokenny

Technically there is "Clear localstorage for this domain" in the "Clean" dropdown menu of the popup. This does not require an API since CAD injects a content script, so it works in Chrome.

Dose this happen automatically when the tab is closed, if not is it possible.

@anewuser

This comment has been minimized.

Copy link

commented Feb 12, 2018

Dose this happen automatically when the tab is closed, if not is it possible.

He has said:

I'm going to close this issue. Any future issues with localstorage should be new issue.

And he has also already pointed you to the bug that needs to be solved by Chromium developers for it to be possible automatically: https://bugs.chromium.org/p/chromium/issues/detail?id=78093

Please don't post comments here anymore. Many people had subscribed to this issue and are still receiving notifications about your posts.

@mrdokenny Consider to restrict comments for this issue to contributors-only.

@Cookie-AutoDelete Cookie-AutoDelete locked as resolved and limited conversation to collaborators Feb 12, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
You can’t perform that action at this time.