network unavailable may cause filter become "0 used out of 0"

[keuucy]

Prerequisites

• I verified that this is not a filter issue
  • Filter issues MUST be reported at filter issue tracker
• This is not a support issue or a question
  • Support issues and questions are handled at /r/uBlockOrigin
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
  • Your issue may already be reported.
• I tried to reproduce the issue when...
  • uBlock Origin is the only extension
  • uBlock Origin with default lists/settings
  • using a new, unmodified browser profile
• I am running the latest version of uBlock Origin
• I checked the documentation to understand that the issue I report is not a normal behavior

Description

(Sorry, my English is very poor)
I use a socks5 proxy server to connect to the network, sometimes the proxy server may not be available,
if uBlock try to update filter lists at this time, some filter may will become "0 used out of 0"

A specific URL where the issue occurs

[A specific URL is MANDATORY for issue happening on a web page, even if it happens "everywhere"]

Steps to Reproduce

1. set firefox's proxy to an invalid proxy server address
2. open the uBlock's filter lists setting and click the "Purge all caches" button
3. click the "Update now" button
4. wait 20-30 seconds, some filter will become "0 used out of 0"

Expected behavior:

[What you expected to happen]

Actual behavior:

[What actually happened]

Your environment

• uBlock Origin version: uBlock Origin v1.16.4
• Browser Name and version: Firefox 59.0.3
• Operating System and version: Windows 8.1

[uBlock-user]

Waited 5 mins, no change. Can't reproduce on Firefox 59

[keuucy]

I did more tests. this problem only occurs in private browsing mode (browser.privatebrowsing.autostart=true)
and only affects certain non-default filters like:
Adblock Warning Removal List
Adguard’s Annoyance List
(Regions, languages)
ara: Liste AR
BGR: Bulgarian Adblock list
CHN: CJX's EasyList Lite
etc.

[uBlock-user]

FF bug - https://bugzilla.mozilla.org/show_bug.cgi?id=1313401

gorhill/uBlock#2925 (comment)

[gorhill]

uBO does not save filter lists in window.localStorage. I first want to investigate to find out what is the real issue. A filter list should never be empty, uBO has code to avoid trashing the cache when the returned response is empty.

[uBlock-user]

Hmmm, so it's not related to the FF bug ?

[uBlock-user]

This is how it looks like if I reproduce it on Nightly -

https://i.gyazo.com/92d169fac37656938a8092c6693f4294.png

[gorhill]

Only private browsing mode is needed to reproduce?

[gorhill]

I can't reproduce: I enabled private browsing mode, and all the filter lists which are fetched from a remote server are properly fetched.

[uBlock-user]

No, not just that, you need to setup a proxy in Firefox, with an invalid server address like 0.0.0.0 like it's mentioned in the STR.

[gorhill]

set firefox's proxy to an invalid proxy server address

I am going to need exact, detailed steps for this.

[uBlock-user]

need exact, detailed steps for this.

1. Run Firefox, open about:preferences#general, scroll down to the end of the page where you will find a preference named Network Proxy

2. Click on Settings button and select Manual proxy configuration, fill in the HTTP Proxy field with 0.0.0.0 and Port can be set as 80.

3. Tick Use this proxy server for all protocols if it's not ticked.

4. Click on OK and restart Firefox.

[gorhill]

I can't reproduce with that proxy config. However I can reproduce if I disable my access to internet -- something which is not described anywhere in the repro steps.

Also, something is wrong in OP's report, or I still just can't reproduce as described. The only way I can get "0 out of 0" is if I try to fetch the remote filter list when it is not yet present in the cache -- and in such case, it is indeed expected, as per https://bugzilla.mozilla.org/show_bug.cgi?id=1313401.

Thus marking as unable to reproduce as per OP's repro steps.

[uBlock-user]

If you set your proxy address to 0.0.0.0, then you're basically cutting off Firefox's access to the Internet. you won't be able to open any websites at all.

[gorhill]

My internet access was not cut off, which means Firefox was somehow ignoring the proxy and connecting using an available connection (using linux here).

[uBlock-user]

This is how it should look like

Otherwise, it's like you said, it's ignoring the proxy settings set by yourself.

[gorhill]

Yes I get that page. I also get successful network requests by Firefox itself and uBO:

It could be because on FF59/linux extensions are not yet in their own content process, and maybe FF main process does not go through the designated proxy. In any case, just turning off the connection should accomplish the same thing.

However at this point, issue should be closed assuming OP reported wrongly -- can't never emphasize enough "exact and detailed". uBO does not overwrite the cache with empty content. The only way to get "0 out of 0" is if the cache was empty in the first place -- and no cache are removed when clicking "Purge all caches".

[uBlock-user]

To me it seems more and more of a browser bug if any.

[gorhill]

When a filter list is not in the cache and there is no connection available, there is no way for uBO to download the list, hence "0 out of 0" -- you will get the same issue with any browser. If you want to assume OP described the issue improperly (likely) and that is issue is that uBO loses the cached lists upon browser restart because of private browsing mode, then yes, it is a known Firefox bug.

[keuucy]

Sorry, now I understand that this is a bug of the Firefox's private browsing mode.
I only know that sometimes the proxy server may not be available, and sometimes the filter rules become an empty list (it takes about one to two months) so I try to set an invalid address to reproduce the problem, but the invalid proxy address triggers another problem...

About invalid proxy settings with "0 out of 0":

1. launch Firefox 59.0.3 with new profile (firefox.exe -profile "F:\newprofile")
2. install uBlock Origin v1.16.4
3. close Firefox
4. overwrite prefs.js file with the following: (10.7.0.3:1080 is a valid proxy)

user_pref("network.proxy.socks", "10.7.0.3");
user_pref("network.proxy.socks_port", 1080);
user_pref("network.proxy.socks_remote_dns", true);
user_pref("network.proxy.type", 1);
user_pref("browser.privatebrowsing.autostart", true);

5. launch Firefox
6. subscribe to some filters and click the "update now" button
7. wait some seconds, make sure all filters have been updated
8. change Socks5 port ("network.proxy.socks_port") to 1081 (10.7.0.3:1081 is an invalid proxy)
9. click the "purge all caches" and "update now" button
10. some filters become empty content
    GIF: https://i.imgur.com/XsMilsI.gif