"$all" and "$doc" currently don't seem to be supported #229

DandelionSprout · 2020-03-07T11:35:44Z

Prerequisites

I verified that this is not a filter issue
- Filter issues MUST be reported at filter issue tracker
This is not a support issue or a question
- Support issues and questions are handled at /r/uBlockOrigin
I performed a cursory search of the issue tracker to avoid opening a duplicate issue
- Your issue may already be reported.
I tried to reproduce the issue when...
- uBlock Origin is the only extension
- uBlock Origin with default lists/settings
- using a new, unmodified browser profile
I am running the latest version of uBlock Origin (1.16.4.19 stable)
I checked the documentation to understand that the issue I report is not a normal behavior

Description

Currently, neither $all or $doc seem to be supported by Firefox Legacy, as any strict blocking prompts don't show up.

Fixing $doc would be as simple as adding 'doc': 'main_frame' to

uBlock-for-firefox-legacy/src/js/static-net-filtering.js

Lines 1379 to 1409 in 755a8e0

    
           FilterParser.prototype.toNormalizedType = { 
        
                       'beacon': 'other', 
        
                          'css': 'stylesheet', 
        
                         'data': 'data', 
        
                     'document': 'main_frame', 
        
                     'elemhide': 'generichide', 
        
                         'font': 'font', 
        
                        'frame': 'sub_frame', 
        
                 'genericblock': 'unsupported', 
        
                  'generichide': 'generichide', 
        
                        'ghide': 'generichide', 
        
                        'image': 'image', 
        
                  'inline-font': 'inline-font', 
        
                'inline-script': 'inline-script', 
        
                        'media': 'media', 
        
                       'object': 'object', 
        
            'object-subrequest': 'object', 
        
                        'other': 'other', 
        
                         'ping': 'other', 
        
                     'popunder': 'popunder', 
        
                        'popup': 'popup', 
        
                       'script': 'script', 
        
                   'stylesheet': 'stylesheet', 
        
                  'subdocument': 'sub_frame', 
        
                          'xhr': 'xmlhttprequest', 
        
               'xmlhttprequest': 'xmlhttprequest', 
        
                       'webrtc': 'unsupported', 
        
                    'websocket': 'websocket' 
        
           }; 
        
           /******************************************************************************/

; but $all would possibly have to be converted into two different entries at once, one for nothing at all and one for $popup. Arguably even a third one for $document would have to be added, leading to complexity that makes me wonder if it's more ideal to just add native support for $all instead.

A specific URL where the issue occurs

Just about anywhere on the internet, but let's assume https://www.nrk.no/ as a very, very safe test site for this.

Steps to Reproduce

Add ||nrk.no^$all and ||nrk.no^$doc to My Filters.
Visit https://www.nrk.no/.
See that the strict warning page doesn't show up, and that the site is loaded normally.

Expected behavior:

The strict blocking page shows up.

Actual behavior:

The page is loaded normally.

Your environment

uBlock Origin version: 1.16.4.19
Browser Name and version: Pale Moon 28.8.4 64-bit
Operating System and version: Windows 10 November 2019 Update

The text was updated successfully, but these errors were encountered:

JustOff · 2020-03-07T11:47:31Z

~~Could you please point me to the wiki or smth about $all keyword?~~

Well, I found it myself:

$all - gorhill/uBlock@1888033
$doc - gorhill/uBlock@dfcd231

JustOff · 2020-03-12T13:52:34Z

Here is the test version - uBlock0_1.16.4.20b1.firefox-legacy.xpi.zip (rename zip to xpi).

What is the best way to test $all operator?

DandelionSprout · 2020-03-13T01:38:45Z

I can confirm that $doc works as it should in 1.16.4.20b1, especially since force-updating my Nordic list in that version bumped the amount of entries in uBO's settings from ~900 to ~1,130.

I simply use nrk.no as my test site of choice for just about everything ever, as it has zero ads, ≤5 trackers, no region-locking (except for sports videos), virtually zero hostility towards adblockers, no fullpage overlays (that I'm aware of), and has a modern GUI that is also pretty lightweight by PC browser standards. Although that site has no pop-up elements to test for whether $all covers $popup, I can confirm that $all covers just about everything else (incl. $document), so I feel very certain that it also covers $popup as well.

JustOff · 2020-03-13T10:38:27Z

Thanks, I will put this beta in my main browser and watch how it behaves.

@THEtomaso, you may be interested in this as well.

THEtomaso · 2020-03-13T17:15:21Z

Thanks, JustOff.
I can also confirm that $all and $doc works in your latest beta ($all successfully covering $document and $popup)! 👍

--

@DandelionSprout:
You can observe $all covering the $popup syntax, by applying this rule:
google.com/*/ChromeSetup.exe$all

..and then try to download one of the worst spyware packages in existence, from here:
https://www.google.com/chrome/index.html
:)

DandelionSprout · 2020-03-15T21:55:02Z

Having only now noticed that edit, I can confirm that google.com/*/ChromeSetup.exe$all does indeed cause the Chrome browser download process to fail to ever download anything, and that it's registered as a $popup blocking. [👍/😅]

Related discussion: - https://www.reddit.com/r/uBlockOrigin/comments/bqnsoa/ The `all` option is equivalent to specifying all network-based types + `popup`, `document`, `inline-font`, `inline-script`. Example from discussion: ||bet365.com^$all Above will block all network requests, block all popups, prevent inline fonts/scripts from `bet365.com`. EasyList- compatible syntax does not allow to accomplish that semantic when using only `||bet365.com^`. If using specific negated type options along with `all`, the order in which the options appear is important. In such case `all` should always be first, followed by the negated type option(s).

THEtomaso · 2020-06-21T11:32:32Z

Issue:

While troubleshooting this problem: https://github.com/wolfbeast/lunarblocklist/issues/6/ I think I might have discovered a bug related to uBO Legacy's $all and $doc implementation:

How to reproduce (using uBO Legacy in Pale Moon):

1 - Add this blocking rule to your filters (found in 'Lunar Blocklist by Moonchild'):
||pixel.

2 - Open this page, and observe as all article images are blocked:
https://www.thecut.com/2016/08/winona-ryder-c-v-r.html

3 - Add one of the following whitelisting rules:
@@||pixel.nymag.com/imgs/$doc,image
or
@@||pixel.nymag.com/imgs/$document,image

4 - Reload the page, and observe as all article images are now shown.

5 - Now, right-click on one of the images, and select "View Image"..
The direct-linked image is now blocked!

6 - Add this whitelisting rule:
@@||pixel.nymag.com/imgs/$all

7 - Right-click on one of the images, and select "View Image" again..
The direct-linked image is now shown, while uBO's logger clearly states that it's allowed by the $doc variable.
So, why doesn't $doc or $document work?

JustOff · 2020-06-21T11:57:50Z

The above problem also occurs with uBO-legacy 1.16.4.19, i.e. before all and doc filter options were added, so please fill a separate issue for this.

PS: I would not advise you to use Lunar Blocklist with uBlock, just saying.

THEtomaso · 2020-06-21T12:16:11Z

The above problem also occurs with uBO-legacy 1.16.4.19, i.e. before all and doc filter options were added, so please fill a separate issue for this.

OK, good thing that we can rule that out then.
Thanks for troubleshooting!
Separate issue report posted here:
#237

--

I would not advise you to use Lunar Blocklist with uBlock

Why not?
If I didn't, then who else would be filing issue reports for it? :)

--

btw; when testing this particular issue in other browsers, I discovered yet again how flawed and potentially unsafe Chromium is.
The $doc / $document syntax aren't even needed for Chromium, in this regard, as it somehow seems to convert the subdomain from pixel.nymag.com to pyxis.nymag.com natively, before uBO ever gets a chance to block it!

gwarser · 2020-06-21T13:14:50Z

The $doc / $document syntax aren't even needed for Chromium, in this regard, as it somehow seems to convert the subdomain from pixel.nymag.com to pyxis.nymag.com natively, before uBO ever gets a chance to block it!

pixel.nymag.com returns 301 moved permanently, so it may be understandable to "optimize" by requesting directly pyxis.nymag.com when resource is requested next time.

THEtomaso · 2020-06-21T13:23:28Z

"optimize"

Yeah, those quotation marks should indeed be included.
I've noticed a lot of that going on in Chromium, and I'm definitely not a fan of it!

gwarser · 2020-06-21T14:44:51Z

For 301 this is in specification.

THEtomaso · 2020-06-21T14:54:53Z

For 301 this is in specification.

Of course, Pale Moon also redirects from pixel.nymag.com to pyxis.nymag.com.
But in this case, the redirection doesn't occur until after uBO has had a chance to block the original URL.
Browsers shouldn't assume anything, but rather threat any website like it's visiting it for the first time!

THEtomaso · 2020-06-21T20:18:25Z

@gwarser:
Although, you seem to be on to something in that other thread, I stand corrected in regards to uBO Legacy's behaviour, in this matter:
#237 (comment)
😊

JustOff added the something to address label Mar 12, 2020

JustOff referenced this issue Mar 17, 2020

Add support for doc filter option

4c646ce

JustOff added fixed and removed something to address labels Mar 17, 2020

JustOff closed this as completed Mar 17, 2020

THEtomaso mentioned this issue Jun 21, 2020

Too general blocking rules can't be exceptioned in some cases #237

Closed

gwarser mentioned this issue Jun 21, 2020

Multiple entries for the same resources in the logger (Firefox only) uBlockOrigin/uBlock-issues#1122

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"$all" and "$doc" currently don't seem to be supported #229

"$all" and "$doc" currently don't seem to be supported #229

DandelionSprout commented Mar 7, 2020 •

edited

JustOff commented Mar 7, 2020 •

edited

JustOff commented Mar 12, 2020

DandelionSprout commented Mar 13, 2020 •

edited

JustOff commented Mar 13, 2020

THEtomaso commented Mar 13, 2020 •

edited

DandelionSprout commented Mar 15, 2020 •

edited

THEtomaso commented Jun 21, 2020 •

edited

JustOff commented Jun 21, 2020

THEtomaso commented Jun 21, 2020 •

edited

gwarser commented Jun 21, 2020

THEtomaso commented Jun 21, 2020

gwarser commented Jun 21, 2020

THEtomaso commented Jun 21, 2020

THEtomaso commented Jun 21, 2020 •

edited

"$all" and "$doc" currently don't seem to be supported #229

"$all" and "$doc" currently don't seem to be supported #229

Comments

DandelionSprout commented Mar 7, 2020 • edited

Prerequisites

Description

A specific URL where the issue occurs

Steps to Reproduce

Expected behavior:

Actual behavior:

Your environment

JustOff commented Mar 7, 2020 • edited

JustOff commented Mar 12, 2020

DandelionSprout commented Mar 13, 2020 • edited

JustOff commented Mar 13, 2020

THEtomaso commented Mar 13, 2020 • edited

DandelionSprout commented Mar 15, 2020 • edited

THEtomaso commented Jun 21, 2020 • edited

Issue:

How to reproduce (using uBO Legacy in Pale Moon):

JustOff commented Jun 21, 2020

THEtomaso commented Jun 21, 2020 • edited

gwarser commented Jun 21, 2020

THEtomaso commented Jun 21, 2020

gwarser commented Jun 21, 2020

THEtomaso commented Jun 21, 2020

THEtomaso commented Jun 21, 2020 • edited

DandelionSprout commented Mar 7, 2020 •

edited

JustOff commented Mar 7, 2020 •

edited

DandelionSprout commented Mar 13, 2020 •

edited

THEtomaso commented Mar 13, 2020 •

edited

DandelionSprout commented Mar 15, 2020 •

edited

THEtomaso commented Jun 21, 2020 •

edited

THEtomaso commented Jun 21, 2020 •

edited

THEtomaso commented Jun 21, 2020 •

edited