Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redirection fails for filters having * in the host part #572

Closed
6 of 8 tasks
kulfoon opened this issue May 13, 2019 · 9 comments
Closed
6 of 8 tasks

Redirection fails for filters having * in the host part #572

kulfoon opened this issue May 13, 2019 · 9 comments
Labels
bug Something isn't working fixed issue has been addressed

Comments

@kulfoon
Copy link

kulfoon commented May 13, 2019

Orginal polish issue: olegwukr/polish-privacy-filters#67
Orginal uassets issue: uBlockOrigin/uAssets#5360
The current issue perhaps somehow related to #568 and gorhill/uBlock#1597 and gorhill/uBlock#3590 not sure.

Prerequisites

  • I verified that this is not a filter issue
  • This is not a support issue or a question
  • I performed a cursory search of the issue tracker to avoid opening a duplicate issue
    • Your issue may already be reported.
  • I tried to reproduce the issue when...
    • uBlock Origin is the only extension
    • uBlock Origin with default lists/settings
    • using a new, unmodified browser profile
  • I am running the latest version of uBlock Origin
  • I checked the documentation to understand that the issue I report is not a normal behavior

Description

We were just trying to find a pure uBO solution (that doesn't need external solutions: (uBlockOrigin/uAssets#5360 (comment)) for anti-adblock for player.pl: olegwukr/polish-privacy-filters#67 and we succeeded (for Chrome-based browsers) but by the way discovered that some redirection filters with * does not work. We're not sure whether it's a bug in uBO or just a filter bug in domain syntax of the filter - perhaps only *. works and no other characters are allowed in the domain of the filter if * is used, like entity in static extended filters: https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#entity

A specific URL where the issue occurs

Example url: https://player.pl/programy-online/kuchenne-rewolucje-odcinki,114

Steps to Reproduce

Filter 1: ||n-*-*.dcs.redcdn.pl/*/TVN-Adserver/*.mp4$media,redirect=noopmp3-0.1s,domain=player.pl
Filter 2: ||n-*.dcs.redcdn.pl/*/TVN-Adserver/*.mp4$media,redirect=noopmp3-0.1s,domain=player.pl
Filter 3: ||*.dcs.redcdn.pl/*/TVN-Adserver/*.mp4$media,redirect=noopmp3-0.1s,domain=player.pl
Filter 4: ||dcs.redcdn.pl/*/TVN-Adserver/*.mp4$media,redirect=noopmp3-0.1s,domain=player.pl

  1. make sure you have also both regional PL filter lists enabled
  2. add the Filter 1 and Filter 2 to My Filters
  3. open the logger
  4. visit the website
  5. on the website notice the spinning wheel - the video won't load, and then an error is showing up instead because the redirection Filters 1 and 2 don't work
  6. go back to the logger and notice a lack of the redirection filter in the logger
  7. add the Filter 3 or Filter 4
  8. visit the website again
  9. on the website notice the ads are being skipped / redirected and then the video is playing fine
  10. go back to the logger and notice a presence of the redirection filter in the logger

Expected behavior:

Filters 1 and 2 should work just like Filter 3 and 4.

Actual behavior:

Filters 1 and 2 not working, the part n-*-*. or n-*. breaks the redirection.

Your environment

  • uBlock Origin version: 1.19.3b0
  • Browser Name and version: Opera 60 64bit
  • Operating System and version: Windows 7 Pro SP1 PL

Screenshoots:

Filters 1 and 2 (video not playing - an error showing up)

1

Filters 3 and 4 (video playing fine)

2

@gwarser
Copy link

gwarser commented May 13, 2019

Specifically, notice that the filter must start with || or *, otherwise no redirection directive will be created, although a blocking filter will be created. Essentially, a redirection filter must always have a destination hostname specified, or * if the filter is to apply to all destinations*.

https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#redirect

Domain with wildcard is not explicit enough.


I think, uBO thinks destination domain is n-.

@gorhill
Copy link
Member

gorhill commented May 13, 2019

As pointed out in the documentation excerpted by @gwarser, not supported.

@gorhill gorhill closed this as completed May 13, 2019
@gorhill gorhill added invalid not a uBlock issue and removed something to address something to address labels May 13, 2019
@gorhill
Copy link
Member

gorhill commented May 13, 2019

Let's say I take https://player.pl/programy-online/kuchenne-rewolucje-odcinki,114 as a typical case of why uBO should support wildcards inside a hostname for redirection purpose... I am unable to see network requests matching n-*-*.dcs.redcdn.pl, I rather see r-scale-*.dcs.redcdn.pl on my side. What do I need to do to reproduce the n-*-*.dcs.redcdn.pl ones?

Edit: Never mind, I was geo-blocked -- I can see the n-*-*.dcs.redcdn.pl requests with VPN.

@gorhill gorhill removed the invalid not a uBlock issue label May 13, 2019
@gorhill gorhill reopened this May 13, 2019
@gorhill
Copy link
Member

gorhill commented May 13, 2019

Sorry for my quick dismissal, giving more thoughts to this, uBO should still be able to extract a wildcard-less hostname from ||n-*-*.dcs.redcdn.pl^... and use this for the quick lookup of redirection directives.

gorhill added a commit to gorhill/uBlock that referenced this issue May 14, 2019
Related issue:
- uBlockOrigin/uBlock-issues#572

Wildcards are now allowed in the hostname part of redirect
filters. There will be an attempt to find the longest
right-hand portion of the hostname with no wildcard. If
no non-empty hostname can be extracted, `*` will be used.
@uBlock-user uBlock-user added bug Something isn't working fixed issue has been addressed labels May 14, 2019
@kulfoon
Copy link
Author

kulfoon commented May 14, 2019

I'm unable to verify whether the fix works or not at given specific case, as no TVN-Adserver requests in the logger anymore in Opera (however they are still present in Firefox), and even Filters 3 and 4 won't unblock the video now, perhaps something changed at the website's side, exactly at this time, a strange coincidence, perhaps player.pl follows us on github and adapting to bypass our filters.

so I've tried to redirect another request: https://s1-player5.cdntvn.pl/video/test.mp4 with a filter:
||s*-player*.cdntvn.pl/video/test.mp4$media,redirect=noopmp3-0.1s,domain=player.pl
but a redirection hasn't occured, and the request has been blocked.

when I've tried: ||cdntvn.pl/video/test.mp4$media,redirect=noopmp3-0.1s,domain=player.pl a redirection is working.

Could you verify whether the fix works.

As for the:

gorhill : #572 (comment) :

the are more such hostnames - a first is just above, and the others are:

https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/turbo.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-file-76.dcs.redcdn.pl/file/o2/web/lib/jquery/3.2.1/jquery.min.js

And there is a bunch of requests grabbed from the logger:

spoiler
https://j.dcs.redcdn.pl/dash/o2/tvnplayer/vod/19_400_13273_0011/SMOOTH_HD_PL/ea5c2609-5563-4331-952c-9d67911660b6/playlist.ism
https://j.dcs.redcdn.pl/dash/o2/tvnplayer/vod/19_400_13273_0011/SMOOTH_HD_PL/ea5c2609-5563-4331-952c-9d67911660b6/playlist.ism
https://j.dcs.redcdn.pl/file/o2/tvn/web-content/m/p1/v/73e0f7487b8e5297182c5a711d20bf26/5aec48a0-f9ef-4e68-aed7-49bbd5580503-720p.mp4
https://n-5-9.dcs.redcdn.pl/dash/o2/tvnplayer/vod/19_400_13273_0011/SMOOTH_HD_PL/ea5c2609-5563-4331-952c-9d67911660b6/playlist.ism
https://r-file-76.dcs.redcdn.pl/file/o2/web/lib/jquery/3.2.1/jquery.min.js
https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/kuchnia.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/player.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/ttv.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/turbo.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-http-d1.dcs.redcdn.pl/scale/o2/tvnplayer/portal/logo/tvn.png?type=0&srcmode=4&srcx=0%2F1&srcy=0%2F1&srcw=231&srch=80&dstw=231&dsth=80
https://r-scale-3d.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/77369e37b2aa1404f416275183ab055f/e42119a7-2c27-4302-897d-d34f57f23de1.jpg?srcx=0&srcy=0&srcw=2048&srch=1152&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-45.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/acab0116c354964a558e65bdd07ff047/5e3d5112-8368-4ab3-ba32-fd1101e1394b.jpg?srcx=0&srcy=106&srcw=2000&srch=1125&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-54.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/f45a1078feb35de77d26b3f7a52ef502/3ac8ed6e-39d5-4bb3-a57b-6681e89a2c67.jpg?srcx=166&srcy=134&srcw=1843&srch=1038&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-6c.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/acab0116c354964a558e65bdd07ff047/be98dac7-7eb3-4a03-9bad-0a6006286f77.jpg?srcx=0&srcy=156&srcw=2000&srch=1125&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-6e.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/acab0116c354964a558e65bdd07ff047/a80b3f02-53db-4fd5-b1b7-35d0a1fd0293.jpg?srcx=0&srcy=31&srcw=1975&srch=1112&type=1&srcmode=0&quality=65&dstw=1920&dsth=1080
https://r-scale-af.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/b3bbccd6c008e727785cb81b1aa08ac5/d2dee6db-8743-422c-8d97-353fa8e745fb.jpg?srcx=0&srcy=166&srcw=2035&srch=1145&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-ec.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/8725fb777f25776ffa9076e44fcfd776/0f908de6-abe8-46d9-a1ae-98519cbc6ea3.jpg?srcx=0&srcy=131&srcw=2000&srch=1125&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-f3.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/07a4e20a7bbeeb7a736682b26b16ebe8/8d5b2bb7-052c-4f03-95f4-0f7ea674bbd5.jpg?srcx=657&srcy=0&srcw=1390&srch=782&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://r-scale-fd.dcs.redcdn.pl/scale/o2/tvn/web-content/m/p24/i/8c8a58fa97c205ff222de3685497742c/a4d4a91b-c353-44d7-b9af-22902281537e.png?srcx=0&srcy=0&srcw=1899&srch=1071&type=1&srcmode=0&quality=65&dstw=315&dsth=177
https://s1-player5.cdntvn.pl/img/components/parental-rating/12.png
https://s1-player5.cdntvn.pl/img/player-pl/loader.png
https://s1-player5.cdntvn.pl/player.css
https://s1-player5.cdntvn.pl/player.js
​https://s1-player5.cdntvn.pl/js/components/parental-rating/main.js?bust=1557824108143
​https://s1-player5.cdntvn.pl/video/test.mp4

gorhill added a commit to gorhill/uBlock that referenced this issue May 14, 2019
@gorhill
Copy link
Member

gorhill commented May 14, 2019

Could you verify whether the fix works.

I made a mistake and tested only the one-wildcard case. This should work in the next dev build.

@kulfoon
Copy link
Author

kulfoon commented May 14, 2019

It fixed the two-wildcard case:
||s*-player*.cdntvn.pl/video/test.mp4$media,redirect=noopmp3-0.1s,domain=player.pl

But now I've noticed another broken case, a one-wildcard case - which was already present even before the last fix, only that I didn't check it before:
||s*-player5.cdntvn.pl/video/test.mp4$media,redirect=noopmp3-0.1s,domain=player.pl

@gorhill
Copy link
Member

gorhill commented May 14, 2019

||s*-player5.cdntvn.pl/video/test.mp4$media,redirect=noopmp3-0.1s,domain=player.pl

Ok, I overlooked that a wildcard could be positioned at another position than hostname label boundary. I will also fix for next dev build.

gorhill added a commit to gorhill/uBlock that referenced this issue May 14, 2019
@kulfoon
Copy link
Author

kulfoon commented May 14, 2019

Fixed, all filters works now, even those from the 1st comment (as TVN-Adserver request is back on Opera, so I've been able to test them too).

kowith337 referenced this issue in kowith337/PersonalFilterListCollection May 21, 2019
Now I've added GitLab mirror into any markdown documents that I've already written.
The 🐱 will link to file location on GitHub, while 💊 will link onto GitLab instead.

Filters update
- [Misc] Add some rules for `Google` to prevent some async ping with `ei` parameter
- [Test] Try prevent BingsuMaker script early for `Pantip`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fixed issue has been addressed
Projects
None yet
Development

No branches or pull requests

4 participants