New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ws-gateway websocket circumvention ? #1936

Closed
mapx- opened this Issue Aug 26, 2016 · 21 comments

Comments

Projects
None yet
4 participants
@GokuSenpai

This comment has been minimized.

Show comment
Hide comment
@GokuSenpai

GokuSenpai Aug 26, 2016

This only happens on Chromium based browsers. It works fine on Firefox and the browsers based on it.

Secondly, pornhub is now using invalid hostnames to bypass WebSocket blockage and it's also happening on Chromium based browsers only, it makes me wonder why on chromium only...

By the way, this is what pornhub is using as it's hostname domain for WS - ws://ws.00zasdf.pw**.**/nsoj

Notice that . at the end of the domain, that makes the domain invalid and that's how it's inserting ads on chrome since the past 48 hours...

GokuSenpai commented Aug 26, 2016

This only happens on Chromium based browsers. It works fine on Firefox and the browsers based on it.

Secondly, pornhub is now using invalid hostnames to bypass WebSocket blockage and it's also happening on Chromium based browsers only, it makes me wonder why on chromium only...

By the way, this is what pornhub is using as it's hostname domain for WS - ws://ws.00zasdf.pw**.**/nsoj

Notice that . at the end of the domain, that makes the domain invalid and that's how it's inserting ads on chrome since the past 48 hours...

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 26, 2016

Owner

I could reproduce.

Meanwhile I created a filter to at least prevent the ads and circumvention of WebSocket until I figure what is going on.

Quick look is that the WebSocket is created into an iframe with a data:-based URI as source.

Owner

gorhill commented Aug 26, 2016

I could reproduce.

Meanwhile I created a filter to at least prevent the ads and circumvention of WebSocket until I figure what is going on.

Quick look is that the WebSocket is created into an iframe with a data:-based URI as source.

@mapx-

This comment has been minimized.

Show comment
Hide comment
@lewisje

This comment has been minimized.

Show comment
Hide comment
@lewisje

lewisje Aug 26, 2016

By the way, this is what pornhub is using as it's hostname domain for WS - ws://ws.00zasdf.pw./nsoj

Notice that . at the end of the domain, that makes the domain invalid

Actually, the fullest expression of a hostname has a period at the end, to represent the root zone in the DNS, which has no name; it's just that most software allows us to keep it off, and it's much easier to not type the dot at the end, and then as a result, some software developers don't properly account for the case where the trailing dot is left on.

lewisje commented Aug 26, 2016

By the way, this is what pornhub is using as it's hostname domain for WS - ws://ws.00zasdf.pw./nsoj

Notice that . at the end of the domain, that makes the domain invalid

Actually, the fullest expression of a hostname has a period at the end, to represent the root zone in the DNS, which has no name; it's just that most software allows us to keep it off, and it's much easier to not type the dot at the end, and then as a result, some software developers don't properly account for the case where the trailing dot is left on.

@GokuSenpai

This comment has been minimized.

Show comment
Hide comment
@GokuSenpai

GokuSenpai Aug 26, 2016

Actually, the fullest expression of a hostname has a period at the end, to represent the root zone in the DNS, which has no name; it's just that most software allows us to keep it off,

Yeah aside from that technical mumbo jumbo, pornhub is using that as an exploit for bypassing the blockage of WS on chrome, that's the point here and because of this, uBO isn't able to detect the WS connection either!!

GokuSenpai commented Aug 26, 2016

Actually, the fullest expression of a hostname has a period at the end, to represent the root zone in the DNS, which has no name; it's just that most software allows us to keep it off,

Yeah aside from that technical mumbo jumbo, pornhub is using that as an exploit for bypassing the blockage of WS on chrome, that's the point here and because of this, uBO isn't able to detect the WS connection either!!

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 27, 2016

https://issues.adblockplus.org/ticket/4372#comment:6

Pornhub circumventing our wrapper with a WebWorker:

AdDelivery.prototype.createWW = function() { var b = "self.onmessage=function(a){self.debug=" + this.debug + ';self.wsurl="' + this.websocketURL + '";self.initWS=function(b){self.ws=new WebSocket(b);self.ws.onerror=function(c){self.log("Websocket error: "+c);postMessage(null)};self.ws.onopen=function(c){self.log("Websocket connected")};self.ws.onmessage=function(c){self.log("Websocket received msg.");postMessage(c.data)}};self.requestAds=function(b){if(self.ws.readyState!==1){setTimeout(function(){self.log("Waiting for connection");self.requestAds(b)},100)}else{ws.send(b)}};self.log=function(b){if(self.debug){console.log(b)}};if(!self.ws){self.initWS(self.wsurl);self.log("Initializing websocket")}else{self.log("Websocket already connected")}self.requestAds(a.data)};'; this.blob = new Blob([b], { type: "application/javascript" }); this.ww = new Worker(URL.createObjectURL(this.blob)); return };

mapx- commented Aug 27, 2016

https://issues.adblockplus.org/ticket/4372#comment:6

Pornhub circumventing our wrapper with a WebWorker:

AdDelivery.prototype.createWW = function() { var b = "self.onmessage=function(a){self.debug=" + this.debug + ';self.wsurl="' + this.websocketURL + '";self.initWS=function(b){self.ws=new WebSocket(b);self.ws.onerror=function(c){self.log("Websocket error: "+c);postMessage(null)};self.ws.onopen=function(c){self.log("Websocket connected")};self.ws.onmessage=function(c){self.log("Websocket received msg.");postMessage(c.data)}};self.requestAds=function(b){if(self.ws.readyState!==1){setTimeout(function(){self.log("Waiting for connection");self.requestAds(b)},100)}else{ws.send(b)}};self.log=function(b){if(self.debug){console.log(b)}};if(!self.ws){self.initWS(self.wsurl);self.log("Initializing websocket")}else{self.log("Websocket already connected")}self.requestAds(a.data)};'; this.blob = new Blob([b], { type: "application/javascript" }); this.ww = new Worker(URL.createObjectURL(this.blob)); return };

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 27, 2016

Owner

I have implemented blocking through Content Security Policy.

Verified that a filter such as *$websocket,domain=pornhub.com|tomshardware.com works:

a

Personally that will be my solution for now to this, this will put the blocker squarely ahead of the find-a-hole-in-the-wrapper game.

Owner

gorhill commented Aug 27, 2016

I have implemented blocking through Content Security Policy.

Verified that a filter such as *$websocket,domain=pornhub.com|tomshardware.com works:

a

Personally that will be my solution for now to this, this will put the blocker squarely ahead of the find-a-hole-in-the-wrapper game.

@gorhill gorhill closed this in 8586aee Aug 27, 2016

gorhill added a commit to uBlockOrigin/uAssets that referenced this issue Aug 27, 2016

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 27, 2016

Owner

pornhub is using that as an exploit for bypassing the blockage of WS on chrome, that's the point here and because of this, uBO isn't able to detect the WS connection either

uBO can handle the dot at the end of a FQDN -- this is not the issue here.

Owner

gorhill commented Aug 27, 2016

pornhub is using that as an exploit for bypassing the blockage of WS on chrome, that's the point here and because of this, uBO isn't able to detect the WS connection either

uBO can handle the dot at the end of a FQDN -- this is not the issue here.

@GokuSenpai

This comment has been minimized.

Show comment
Hide comment
@GokuSenpai

GokuSenpai Aug 28, 2016

I figured that since this was only happening on Chrome and not on uBO on Firefox. So what was the issue really ?

GokuSenpai commented Aug 28, 2016

I figured that since this was only happening on Chrome and not on uBO on Firefox. So what was the issue really ?

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 28, 2016

tested ubo 1.9 , chrome 53
page:
http://www.parentherald.com/articles/63071/20160824/the-pirate-bay-tpb-shut-down-imminent-after-service-partner-faces-piracy-lawsuit.htm

  • delete cookies / local storage in chrome settings before any test
  1. with ublock filters enabled: a lot of normal http / https addresses seen as websocket
    image
  2. without ublock filters:
partially visible ad reclaim crap on top right
clickable layer 
no websocket request detected
click the page ==> popups ==> ubo  closes some and finally will close even the main page (test page)

mapx- commented Aug 28, 2016

tested ubo 1.9 , chrome 53
page:
http://www.parentherald.com/articles/63071/20160824/the-pirate-bay-tpb-shut-down-imminent-after-service-partner-faces-piracy-lawsuit.htm

  • delete cookies / local storage in chrome settings before any test
  1. with ublock filters enabled: a lot of normal http / https addresses seen as websocket
    image
  2. without ublock filters:
partially visible ad reclaim crap on top right
clickable layer 
no websocket request detected
click the page ==> popups ==> ubo  closes some and finally will close even the main page (test page)
@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner

lot of normal http / https addresses seen as websocket

These are all from embedded frames. The new code does not detect WebSocket connections, it just injects a CSP directive preventing such connections from occurring at all, and when that policy is injected, it creates a log entry -- just like when inline-scripting is blocked, a log entry is created as well, even if uBO does not detect the execution of inline scripts.

However, the CSP directive should not be injected for embedded frames not being a target of the filter, this needs fixing (this is true for the CSP to prevent inline-script too, which means the bug has existed since a long time).

Owner

gorhill commented Aug 28, 2016

lot of normal http / https addresses seen as websocket

These are all from embedded frames. The new code does not detect WebSocket connections, it just injects a CSP directive preventing such connections from occurring at all, and when that policy is injected, it creates a log entry -- just like when inline-scripting is blocked, a log entry is created as well, even if uBO does not detect the execution of inline scripts.

However, the CSP directive should not be injected for embedded frames not being a target of the filter, this needs fixing (this is true for the CSP to prevent inline-script too, which means the bug has existed since a long time).

gorhill added a commit that referenced this issue Aug 28, 2016

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner

Fix verified working with the following test case:

  • Add custom filter *$websocket,domain=raymondhill.net;
    • CSP directive to block WebSocket on raymondhill.net will be injected;
  • In a new tab, open http://www.raymondhill.net/ublock/pageloadspeed.html;
  • Add https://www.websocket.org/echo.html to the field URL, click Reload;
    • This causes the embedded frame to be loaded with a document for which WebSocket connections should not be blocked;
  • Click Connect button in the Echo Test demo;

Result before fix: demo not working.

Result after fix: demo working.

Owner

gorhill commented Aug 28, 2016

Fix verified working with the following test case:

  • Add custom filter *$websocket,domain=raymondhill.net;
    • CSP directive to block WebSocket on raymondhill.net will be injected;
  • In a new tab, open http://www.raymondhill.net/ublock/pageloadspeed.html;
  • Add https://www.websocket.org/echo.html to the field URL, click Reload;
    • This causes the embedded frame to be loaded with a document for which WebSocket connections should not be blocked;
  • Click Connect button in the Echo Test demo;

Result before fix: demo not working.

Result after fix: demo working.

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner

For the popup, this should be added to EasyList: ||riowrite.com^$popup.

Owner

gorhill commented Aug 28, 2016

For the popup, this should be added to EasyList: ||riowrite.com^$popup.

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 28, 2016

For the popup, this should be added to EasyList: ||riowrite.com^$popup.

I know, but ubo should not close even the test page or it happens only for me ?

mapx- commented Aug 28, 2016

For the popup, this should be added to EasyList: ||riowrite.com^$popup.

I know, but ubo should not close even the test page or it happens only for me ?

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner

The test page did not close for me. When I clear the cookies and refresh the page and click anywhere on the page, I get an alert about "Chrome PDF viewer" and at the same time a nasty popup opens and does not close.

Owner

gorhill commented Aug 28, 2016

The test page did not close for me. When I clear the cookies and refresh the page and click anywhere on the page, I get an alert about "Chrome PDF viewer" and at the same time a nasty popup opens and does not close.

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 28, 2016

I'm talking about 1.9.0, unchecking "ublock filters", deleting cookies + local storage, all on that page above (parentherald)

mapx- commented Aug 28, 2016

I'm talking about 1.9.0, unchecking "ublock filters", deleting cookies + local storage, all on that page above (parentherald)

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner

I'm talking about 1.9.0, unchecking "ublock filters", deleting cookies + local storage, all on that page above (parentherald)

The page does not close for me, neither the popup (deleting cookies also delete local and session storage):

a

Owner

gorhill commented Aug 28, 2016

I'm talking about 1.9.0, unchecking "ublock filters", deleting cookies + local storage, all on that page above (parentherald)

The page does not close for me, neither the popup (deleting cookies also delete local and session storage):

a

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 28, 2016

it's weird. Without adding this filter
||riowrite.com^
I get this in logger (and the page closes):
http://pastebin.com/Zta8jhYj

using the filter it seems ok.

However, in both cases, I get ad reclaim stuff:
image

mapx- commented Aug 28, 2016

it's weird. Without adding this filter
||riowrite.com^
I get this in logger (and the page closes):
http://pastebin.com/Zta8jhYj

using the filter it seems ok.

However, in both cases, I get ad reclaim stuff:
image

@gorhill

This comment has been minimized.

Show comment
Hide comment
@gorhill

gorhill Aug 28, 2016

Owner
||myawesomecash.com^$third-party,popup  --  popunder  http://myawesomecash.com/new/upload.php?r=1167077&sub1=eyJzIjogIjExNzQwIiwgImIiOiAiMSIsICJ0IjogInAiLCAibCI6IDYzODJ9

Looks like the parentherald.com tab got closed as a popunder because it was replaced with something from myawesomecash.com. As if the page not only opened a nasty popup, but replaced itself with a nasty popunder as well. Looking at my log output above, it looks like the page is shuffling a lot the loaded document in the popup, including loading the popup with the document from http://www.parentherald.com/articles/[...] and replacing it again etc.

However, in both cases, I get ad reclaim stuff

Not sure I follow. That is why the *$websocket,domain=parentherald.com is needed, to completely block WebSocket using a CSP, all the AdReclaim stuff is pulled using a WebSocket connection from within a data: based iframe, and the received data is postMessage'd to the parentherald.com document, which inserts it into the DOM.

Owner

gorhill commented Aug 28, 2016

||myawesomecash.com^$third-party,popup  --  popunder  http://myawesomecash.com/new/upload.php?r=1167077&sub1=eyJzIjogIjExNzQwIiwgImIiOiAiMSIsICJ0IjogInAiLCAibCI6IDYzODJ9

Looks like the parentherald.com tab got closed as a popunder because it was replaced with something from myawesomecash.com. As if the page not only opened a nasty popup, but replaced itself with a nasty popunder as well. Looking at my log output above, it looks like the page is shuffling a lot the loaded document in the popup, including loading the popup with the document from http://www.parentherald.com/articles/[...] and replacing it again etc.

However, in both cases, I get ad reclaim stuff

Not sure I follow. That is why the *$websocket,domain=parentherald.com is needed, to completely block WebSocket using a CSP, all the AdReclaim stuff is pulled using a WebSocket connection from within a data: based iframe, and the received data is postMessage'd to the parentherald.com document, which inserts it into the DOM.

@mapx-

This comment has been minimized.

Show comment
Hide comment
@mapx-

mapx- Aug 28, 2016

well ... disabling "ublock filters" the filter *$websocket,domain=parentherald.com was disabled too.
Adding it I can't reproduce closing page anymore (and the local storage is not written too)

So, it was all generated by the websocket, sorry.

mapx- commented Aug 28, 2016

well ... disabling "ublock filters" the filter *$websocket,domain=parentherald.com was disabled too.
Adding it I can't reproduce closing page anymore (and the local storage is not written too)

So, it was all generated by the websocket, sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment