french list too broad regex filter breaking domain names with a final dot. #7619

dascritch · 2020-07-02T06:39:46Z

URL(s) where the issue occurs

Any domain name with a final . in the name

Describe the issue

A new regex is preventing to visit websites when the entered domain name is written with a final dot, with is legit and sometimes used by webdevs.

The offending regex is :

/^(https?|wss?):\/\/([0-9a-z\._-]+)\.(accountant|bid|cf|click|club|com|cricket|date|download|faith|fun|ga|gdn|gq|info|link|loan|men|ml|net|network|ovh|party|pro|pw|racing|review|rocks|ru|science|site|space|stream|tk|top|trade|webcam|win|xyz|zone)\.\/(.*)/$document

the problems is probably in the \.

The regex is from : FRA: EasyList Liste FR

Versions

Browser/version: Firefox Nigthly 80
uBlock Origin version: uBlock Origin v1.27.10

Settings

only standard settings

The text was updated successfully, but these errors were encountered:

mapx- · 2020-07-02T06:44:18Z

french list issues => https://forums.lanik.us/viewforum.php?f=91

@smed79

dascritch · 2020-07-02T06:45:59Z

Why there is no issue report howto file in the original github repo linked from the addon ?

I'm fed of, it's the third time

dascritch · 2020-07-02T06:47:02Z

OK, I note the very offensive way to treat an issue, especially if you are not french speaking.

mapx- · 2020-07-02T06:48:34Z

You should provide test cases , sites broken by that filter, steps to reproduce the issue

dascritch · 2020-07-02T06:51:17Z

Easy :
google.com. is blocked.

It should not, as writing a final dot in a domain name is absolutely legit.

uBlock-user · 2020-07-02T06:52:51Z

I added /^(https?|wss?):\/\/([0-9a-z\._-]+)\.(accountant|bid|cf|click|club|com|cricket|date|download|faith|fun|ga|gdn|gq|info|link|loan|men|ml|net|network|ovh|party|pro|pw|racing|review|rocks|ru|science|site|space|stream|tk|top|trade|webcam|win|xyz|zone)\.\/(.*)/$document to my filters, opened google.com and it works as expected. Google is not blocked.

dascritch · 2020-07-02T06:54:31Z

https://user-images.githubusercontent.com/482505/86326199-a2e47a00-bc41-11ea-827a-7c32a9cb9429.png

mapx- · 2020-07-02T06:55:47Z

@smed79 you should remove such filters, lots of issues were reported using french list

mapx- · 2020-07-02T06:56:18Z

@dascritch already fixed

uBlock-user · 2020-07-02T06:56:22Z

You're going to google.com./ instead of google.com/. Why ?

dascritch · 2020-07-02T07:00:30Z

Why not ?

As I told, it is a really used trick by webdevs on websites to use many profiles on browsers. And use the Fully Qualified Domain Name notation, instead of the Relative as your usually do, may be really useful in some DNS issues.

For more details : https://webmasters.stackexchange.com/questions/73989/how-should-websites-handle-hostname-with-trailing-dot

uBlock-user · 2020-07-02T07:06:50Z

Why not ?

Because google.com./ redirects to google.com/ in the end. So not a use case.

dascritch · 2020-07-02T07:53:19Z

It was an example : numerous websites don't redirect their FQDN to their RDN , and it permits to go thru misconfigured DNS servers.

If you cannot accept tech people are using a final dot in domain names for testing purposes, you're just over complicating their job. So mine.

dascritch · 2020-07-02T10:03:16Z

Well, even after refreshing all the caches and filters, there is too much limitations when we have FRA: EasyList Liste FR activated for my team : by example any JS or CSS are still blocked. As we usually use FQDN for multiple account testing on websites, I recommended to not uninstall uBlock, but to remove FRA: EasyList Liste FR.

I am not able to tell which rule is blocking, as i don't have enough information in dev tools (capture of a personal website, due to NDA).

I still cannot understand why final dotted domain names are blocked, this is really problematic.

poke @uBlock-user

uBlock-user · 2020-07-02T10:13:54Z

~~Did you update the uBlock unbreak list ? The RegExp filter was disabled in that list.~~

Nevermind, all other network types are also blocked.

gorhill · 2020-07-02T12:16:11Z

That is such a bad generic filter that we should fix this with no question asked really -- a trailing dot is entirely valid, indiscriminately blocking any hostname which has a trailing dot is really bad. I would like to know what was the reason to add such filter in a widely used list.

gwarser · 2020-07-02T12:42:11Z

I think this filter was created to block mining scripts using this FQDN trick.

https://hg.adblockplus.org/listefr/rev/b7608d8b8bbb
hoshsadiq/adblock-nocoin-list@e2c961f

Is this really used for good purposes in the "normal" web? Or just one developer is "exploiting" this in local development environment?

dascritch · 2020-07-02T12:47:54Z

It is really used in generic environment as it is a legal way to force a DNS request, or to make cleaned request without cookies or cleaning cache.

And it seems also used here by malicious code to circumvent a poorly written regex.

gorhill · 2020-07-02T12:50:12Z

A trailing dot would not work around the filters which were replaced, not in uBO as far as I know, and I would be surprised other blockers would not handle properly the trailing dot, so I still fail to understand what issue this solves.

gorhill · 2020-07-02T12:55:05Z

Ok so it appears that filter was added back then to accomodate ABP, which was not properly dealing with the trailing dot (and as a result this shortcoming was being leveraged by miners), this was eventually fixed in ABP, so that bad regex filter should be reverted back to the original set of filters or outright removed.

gwarser · 2020-07-02T13:37:04Z

~~To block only the requests with trailing dot? (regex having good or bad syntax is not important)~~ I don't know.

In uBO .com blocks also .com.. Highlighting in logger is missing

In ABP match is exact.

gorhill · 2020-07-02T14:05:53Z

uBO drops the trailing dot, since it's essentially the same, i.e. example.com is same as example.com.. I don't think filters should care about the trailing dot. Missing highlighting in the logger should be fixed.

Edit: I need to clarify, when I say uBO drops the trailing dot, I mean for hostname-matching purpose (which is what the original filters were), the trailing dot is not dropped in case of URL pattern-matching purpose.

Related feedback: - uBlockOrigin/uAssets#7619 (comment) Also fixed strict-blocking of URL using FQDN.

maxxyme · 2020-08-06T09:01:11Z

If you cannot accept tech people are using a final dot in domain names for testing purposes, you're just over complicating their job. So mine.

Or maybe "tech people" (I am) shouldn't clutter their web browser with so much extensions when "testing"... 🤔

mapx- closed this as completed Jul 2, 2020

mapx- reopened this Jul 2, 2020

mapx- closed this as completed in 2d739b2 Jul 2, 2020

mapx- changed the title ~~Try to not block domain names with a final dot.~~ french list too broad regex filter breaking domain names with a final dot. Jul 2, 2020

uBlock-user added a commit that referenced this issue Jul 2, 2020

update #7619

f638b13

gorhill added a commit to gorhill/uBlock that referenced this issue Jul 3, 2020

Fix highligthing of FQDN match in logger

941898e

Related feedback: - uBlockOrigin/uAssets#7619 (comment) Also fixed strict-blocking of URL using FQDN.

gwarser mentioned this issue Nov 22, 2021

General filter chit-chat №2 DandelionSprout/adfilt#63

Closed

gwarser mentioned this issue Sep 4, 2023

Domains ending in . are rejected, despite being valid FQDNs uBlockOrigin/uBlock-issues#2800

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

french list too broad regex filter breaking domain names with a final dot. #7619

french list too broad regex filter breaking domain names with a final dot. #7619

dascritch commented Jul 2, 2020

mapx- commented Jul 2, 2020

dascritch commented Jul 2, 2020

dascritch commented Jul 2, 2020

mapx- commented Jul 2, 2020

dascritch commented Jul 2, 2020

uBlock-user commented Jul 2, 2020

dascritch commented Jul 2, 2020 •

edited by mapx-

Loading

mapx- commented Jul 2, 2020

mapx- commented Jul 2, 2020

uBlock-user commented Jul 2, 2020

dascritch commented Jul 2, 2020 •

edited

Loading

uBlock-user commented Jul 2, 2020 •

edited

Loading

dascritch commented Jul 2, 2020

dascritch commented Jul 2, 2020 •

edited

Loading

uBlock-user commented Jul 2, 2020 •

edited

Loading

gorhill commented Jul 2, 2020 •

edited

Loading

gwarser commented Jul 2, 2020

dascritch commented Jul 2, 2020

gorhill commented Jul 2, 2020

gorhill commented Jul 2, 2020 •

edited

Loading

gwarser commented Jul 2, 2020 •

edited

Loading

gorhill commented Jul 2, 2020 •

edited

Loading

maxxyme commented Aug 6, 2020

french list too broad regex filter breaking domain names with a final dot. #7619

french list too broad regex filter breaking domain names with a final dot. #7619

Comments

dascritch commented Jul 2, 2020

URL(s) where the issue occurs

Describe the issue

Versions

Settings

mapx- commented Jul 2, 2020

dascritch commented Jul 2, 2020

dascritch commented Jul 2, 2020

mapx- commented Jul 2, 2020

dascritch commented Jul 2, 2020

uBlock-user commented Jul 2, 2020

dascritch commented Jul 2, 2020 • edited by mapx- Loading

mapx- commented Jul 2, 2020

mapx- commented Jul 2, 2020

uBlock-user commented Jul 2, 2020

dascritch commented Jul 2, 2020 • edited Loading

uBlock-user commented Jul 2, 2020 • edited Loading

dascritch commented Jul 2, 2020

dascritch commented Jul 2, 2020 • edited Loading

uBlock-user commented Jul 2, 2020 • edited Loading

gorhill commented Jul 2, 2020 • edited Loading

gwarser commented Jul 2, 2020

dascritch commented Jul 2, 2020

gorhill commented Jul 2, 2020

gorhill commented Jul 2, 2020 • edited Loading

gwarser commented Jul 2, 2020 • edited Loading

gorhill commented Jul 2, 2020 • edited Loading

maxxyme commented Aug 6, 2020

dascritch commented Jul 2, 2020 •

edited by mapx-

Loading

dascritch commented Jul 2, 2020 •

edited

Loading

uBlock-user commented Jul 2, 2020 •

edited

Loading

dascritch commented Jul 2, 2020 •

edited

Loading

uBlock-user commented Jul 2, 2020 •

edited

Loading

gorhill commented Jul 2, 2020 •

edited

Loading

gorhill commented Jul 2, 2020 •

edited

Loading

gwarser commented Jul 2, 2020 •

edited

Loading

gorhill commented Jul 2, 2020 •

edited

Loading