Cosmetic filter not working after refreshing the page

[bbeardsley]

I'm able to add a rule to block a div but after I refresh the page it comes back. How do I go about debugging this? Is there a way to turn on logging to see what divs are being processed?

thesite.domain###comments

is the rule that works initially...

Thanks.

[chrisaljoudi]

Please follow the issue template.

[gorhill]

I need to know:

• Browser/version
• Filter lists enabled
• The exact name of the filter
• Other settings: dynamic filtering rules for the site (if any), whitelist state for the site, etc.

[bbeardsley]

Thank you for the quick replies - I was looking more for guidance on debugging it myself and was wondering what facilities are built into ublock already to assist in debugging (ie: if I gave you the site - how would you go about debugging it?)

[gorhill]

For cosemtic filtering, you have to jump into the content script code.

[LordJZ]

Can confirm Cosmetic filters not working on some domains.

• Firefox 36 / Nightly 39
• All defaults + Fanboy's Social & Annoyance
• example: habrahabr.ru###layout on habrahabr.ru should hide all content; does not
• no dynamics, no whitelists
• uBlock version: 0.8.8.2, 0.9.1

Cosmetics do work on some other sites though.

Additionally, sometimes some filters work, some not -- can see when 1 or 2 social buttons hidden some times. Most times all visible.

[gorhill]

@LordJZ Works fine from here:

So there is something different on your side.

[gorhill]

Do you have "My filters" selected in "3rd-party filters" pane?

[LordJZ]

Yes, and "parse cosmetic filters" as well. It's not "my filters" that aren't working stably -- it's the subscriptions too.

[gorhill]

Well I need a test case where I can reproduce. It works well from here.

Do you have other add-ons which are also messing with the DOM?

[LordJZ]

Doing a binary search to find which one is intercepting. Also, it's not e10s (I have it disabled).

[LordJZ]

Okay figured.

Two filters, first one (layout) works:

habrahabr.ru###layout
!habrahabr.ru##A\[attr="value"]

But now it won't:

habrahabr.ru###layout
habrahabr.ru##A\[attr="value"]

Looks like the \[ is messing up things. There's a whole lot of such filters in my adblock backup with all subscriptions, 9000 out of 125000.

[gorhill]

\ is to escape a character which is reserved in CSS. What is the original filter in ABP from which habrahabr.ru##A\[attr="value"] comes from?

[LordJZ]

It's habrahabr.ru##A[attr="value"]. Can only see that in ABP backups, not ABP UI itself. I had to manually copy over the filters from ABP backup to uBlock because all Import buttons I could find didn't work.

[gorhill]

Ok I see the problem. ABP doesn't save the filters as is, it escapes them in order to neutralize the backup-specific syntax. In an ABP backup, the [ is meaningful, thus it is escaped when encountered in a filter. Thus, cut & pasting the filters from the ABP backup into uBlock causes a bunch of potentially invalid filters to be pasted in uBlock.

Let me think if there is somethign I can do. Maybe I can do something in the "Import & append" code.

[LordJZ]

Sorry I don't know how Import & Append works -- it just did nothing for me.

However, I believe a broken filter shouldn't affect other filters, as it does in this case.

[gorhill]

I believe a broken filter shouldn't affect other filters, as it does in this case.

Easier said than done. That would require to test each filter before it is injected, and thus be a huge performance hit. When I say "huge", I mean it. Other blockers also fail when you give them an invalid cosmetic filter.

Sorry I don't know how Import & Append works

I will patch it in order to be able to import as is ABP backup. I will tell you when it's done, there will be a dev build.

[LordJZ]

Well, ABP is fine with this particular broken filter.

Anyway, since there's a workaround it's enough to let people know of this behavior for the impact to be much less. I've already switched to uBlock with my enormous collection of custom filters, and that was the only issue I've had so far. Thanks for everything!

[gorhill]

Well, ABP is fine with this particular broken filter

ABP will break with invalid cosmetic filters: https://issues.adblockplus.org/ticket/2065. Maybe ABP is just handling its own escaping syntax for A\[attr="value"]. Difference is I had no idea about this special backup syntax. I will provide a fix for importing seamlessly.

[gorhill]

I opened an issue specifically for supporting the import of filters from an ABP backup. It's unclear whether the issue opened by OP is related, as there was no further feedback.

[gwarser]

@gorhill commented on 15 mar 2015, 17:02 CET:

I believe a broken filter shouldn't affect other filters, as it does in this case.

Easier said than done. That would require to test each filter before it is injected, and thus be a huge performance hit. When I say "huge", I mean it. Other blockers also fail when you give them an invalid cosmetic filter.

I remember one ad blocking extension on old Opera doing that. This was just one loop that feeds all filters into document.querySelector enclosed by try-catch block and throws invalid ones out on exception. It was relatively fast.

[gorhill]

@gwarser Yes, I would know how to implement this. That's not what is stopping me, it's the fact that there is nothing stopping users from have MB worth of filters in the text area: there is no limit.

By the way, querySelector('A\[attr="value"]') does not throw, that's the first thing I tried. But if you insert it into a style tag, the whole style tag become invalid, i.e. good CSS selectors in that same style tag stops working.

Edit: Actually, my mistake, when I tried the querySelector at the console, I should have escaped the \ to get equivalent, which I did not.

[gorhill]

@gwarser Thinking more of it, this could be done at list-compile time (done only when a list's content changes), but I would definitely need to run benchmarks to see how much this affect at least the largest filter lists (EasyList).

[gorhill]

Benchmarking how long it takes to compile the filter lists^[1], 3 samples for both cases.

Without querySelector:

• µBlock.loadFilterLists(): 836.264 ms (2 samples)
• µBlock.loadFilterLists(): 919.076 ms (3 samples)
• µBlock.loadFilterLists(): 968.652 ms (4 samples)

With querySelector:

• µBlock.loadFilterLists(): 1733.726 ms (2 samples)
• µBlock.loadFilterLists(): 1726.392 ms (3 samples)
• µBlock.loadFilterLists(): 1690.265 ms (4 samples)

[1] EasyList, Fanboy Social, Fanboy Annoyance, Bitblock, RU Adlist.

[gwarser]

Can you try one more run with querySelectorAll? Looks stupid, but actually on old Opera was faster.

[gorhill]

I wouldn't spend time to benchmark querySelectorAll. If ever I want to spend more time on this I would test Element.matches.

[bbeardsley]

Guess I need more guidance on debugging myself...

Using Firefox 36.0.1 windows, uBlock v0.9.1.1, no whitelist or dynamic rules.
3rd Party Filters:

My filter: sdahymnals.com###comments
Url: http://sdahymnals.com/Hymnal/002-All-Creatures-of-Our-God-and-King/

If I use the picker to pick the latest comments section and hide it - it adds the rule
sdahymnals.com###comments and the comments are gone. If I refresh the page, the comments come back.

[gorhill]

I couldn't reproduce, the comments section does not come back.

If you have bad cosmetic filters applied on that page, they will break cosmetic filtering. This is probably what is happening. I see you 38 other custom filters. There is probably an invalid cosmetic filter in there.

[bbeardsley]

Ok, I removed all my filters added them back one at a time and it seems one that had a asterisk in it was the problem. I removed that one and the rest seemed to work. Would it be possible to flag invalid cosmetic rules somehow?

[gorhill]

Would it be possible to flag invalid cosmetic rules somehow?

The topic was brought forth here.

[gorhill]

Ok, with Element.matches I get the following timings (same benchmark as above):

With Element.matches:

µBlock.loadFilterLists(): 1063.859 ms (2 samples)
µBlock.loadFilterLists(): 1219.154 ms (3 samples)
µBlock.loadFilterLists(): 1315.536 ms (4 samples)

Without Element.matches:

µBlock.loadFilterLists(): 853.832 ms (2 samples)
µBlock.loadFilterLists(): 938.345 ms (3 samples)
µBlock.loadFilterLists(): 977.075 ms (4 samples)

[gorhill]

Given the time users have been wasting dealing with the issue of bad CSS selector, and given that the overhead of using Element.matche is not too bad, I will implement the Element.matches solution, at list compile time, which is the best place to use it as external filter lists too can have bad CSS selector (there is a specific one which I have to fix every times I update 3rd-party assets). The error will be reported in the browser console (Firefox) or in the extension error log (Chromium).

[gorhill]

After latest changes, with Element.matches:

µBlock.loadFilterLists(): 860.540 ms (2 samples)
µBlock.loadFilterLists(): 976.603 ms (3 samples)
µBlock.loadFilterLists(): 1003.348 ms (4 samples)