Third-party rules do not block 4-th level subdomains

[ameshkov]

The source:
AdguardTeam/AdguardFilters#1192

The problem:

1. We have a rule ||innity.net^$third-party
2. For some reason this rule is not applied to https://ssl-cdn.media.innity.net/lib/global.js

[Mizzick]

Content blocker prevented frame displaying http://cn.cari.com.my/portal.php from loading a resource from https://ssl-cdn.media.innity.net/lib/global.js
Content blocker prevented frame displaying http://cn.cari.com.my/portal.php from loading a resource from http://partner.googleadservices.com/gpt/pubads_impl_78.js

As I can see in console, global.js is blocked and on the next line an other 4-th level domain is blocked. So everything is fine?

[ameshkov]

No, it's not fine.

partner.googleadservices.com - is a 3rd level subdomain.
And ssl-cdn.media.innity.net currently is blocked due to the ||media.innity.net^$third-party I've added in AdguardTeam/AdguardFilters#1192

Just disable "Safari filter" and it won't be blocked again.

[ameshkov]

The same issue with exception rules:

Content blocker prevented frame displaying http://www.pornhub.com/ from loading a resource from http://cdn1b.static.pornhub.phncdn.com/images/categories/118x88/7.jpg

While here is the rule that should unblock it:

@@||pornhub.phncdn.com^$image,object-subrequest,domain=pornhub.com|redtube.com|tube8.com|tube8.es|tube8.fr|youporn.com|youporngay.com

[Mizzick]

Fixed, the issue was in incorrect regexp:
"url-filter": "^https?://[^.].?test-elemhide.com"
changed to:
"url-filter": "^https?://[^.].*?test-elemhide.com"

[ameshkov]

Hm, seems that the incorrect regexp has became even more incorrect...
Just use regexr.com and check how it works.

Also keep in mind, that the whole purpose of new regexp is to avoid using regexp groups.
Use testRegexpPerformance to check the speed of new regexps.

[Mizzick]

What?
The previous matches only one domain level up, cause it allows only one dot character before the domain name.

[ameshkov]

Right. The same as your new version.

[Mizzick]

https?://[^.][^/]*.?domain.com

[ameshkov]

verygooddomain.com is matched by this regexp:
http://www.regexr.com/3cg43

[ameshkov]

Huh, I think we had the same issue with a previous regexp too

[Mizzick]

I don't think it's a bug.

[ameshkov]

I am pretty sure this is wrong.
||domain.com should match domain.com and it's subdomains and SHOULD NOT match other domains.

[Mizzick]

This regexp preffix is used not only for || rules

[ameshkov]

In fact it is used for || prefixes only.

url-filter-rule:

    if (StringUtils.startWith(regex, UrlFilterRule.MASK_START_URL)) {
        regex = UrlFilterRule.REGEXP_START_URL + regex.substring(UrlFilterRule.MASK_START_URL.length);

converter:

urlFilter = StringUtils.replaceAll(urlFilter, UrlFilterRule.REGEXP_START_URL, URL_FILTER_REGEXP_START_URL);

[ameshkov]

We may ignore this for now and use even simpler regexp:

https?:\/\/[^/]*\.?domain.com

Or we can try a bit more complex version that works properly:

https?:\/\/([^/]*\.)?domain.com

The question is: what's the difference in terms of performance if we use the second regexp?

[Mizzick]

Okay changed to https?://([^/]*.)?domain.com

[ameshkov]

What with performance test? What's the difference?

[Mizzick]

var regExp3 = new RegExp('^https?://([^/]*.)?some-domain.com.com[/:&?]?', 'i');
base-test.js:52 2015-12-29T15:41:05.270Z DEBUG Elapsed: 127ms
base-test.js:52 2015-12-29T15:41:05.377Z DEBUG Elapsed: 107ms
base-test.js:52 2015-12-29T15:41:05.377Z DEBUG Performance gain: 16%

var regExp4 = new RegExp('^https?://[^/]*.?some-domain.com.com[/:&?]?', 'i');
base-test.js:52 2015-12-29T15:41:05.509Z DEBUG Elapsed: 131ms
base-test.js:52 2015-12-29T15:41:05.608Z DEBUG Elapsed: 98ms
base-test.js:52 2015-12-29T15:41:05.609Z DEBUG Performance gain: 26%

[ameshkov]

Ok then. You'd better increase converter version