Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forcible /g00 adware insertion on newspaper websites #227

Closed
uBlock-user opened this issue Dec 13, 2016 · 198 comments
Closed

Forcible /g00 adware insertion on newspaper websites #227

uBlock-user opened this issue Dec 13, 2016 · 198 comments

Comments

@uBlock-user
Copy link
Member

@uBlock-user uBlock-user commented Dec 13, 2016

URL(s) where the issue occurs

orlandosentinel.com
sandiegouniontribune.com
sun-sentinel.com
mcall.com
boston.com

Those are the ones I have seen so far, there may be more.

Describe the issue

Forcibly inserts g00 adware content and abuses window.location API if blocked by a filter like /g00^$important until it turns into a bad request.

Screenshot(s)

https://i.gyazo.com/86ab54811f6aaa1785b3d308566d6af6.png

Versions

  • Browser/version: [here] Chromium 57
  • uBlock Origin version: [here] 1.10.0

Settings

Default

Notes

  1. This didn't happen when I visited the website few days ago, however it seems the website is infested with adware as of today, as it keeps trying to load the /g00 stuff when it fails the first time and it also inserts shitty adware cookies too.

  2. Blocking inline script does stop the onslaught attack of /g00 however breaks pictures from loading and possibly other things.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 13, 2016

The /g00 stuff is Instart Logic's crap. I have no problem with perceiving this as crapware -- their code goes out of its way to work against end users, doing its best to try to turn user agents (browsers) into proprietary devices.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 13, 2016

Well is there any solution for this, besides from blocking the inline scripts ? Blocking inline scripts would be the last thing I wanna do, that's why I posted here.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 13, 2016

is there any solution for this

I will be able to answer when I have the time to investigate.

@IsraeliAdblocker
Copy link

@IsraeliAdblocker IsraeliAdblocker commented Dec 14, 2016

I've investigated Instart Logic's crap for the past 3 hours, I now know how they work, how they communicate, how they implement on new customers and etc.
I have a lot of information to reveal and I know the best non "cat & mouse" solution to fight them that we can implement right now.

@gorhill , If you can arrange private channel maybe an invisible thread on issues.adblockplus.org (just give me access I am using the same username there) I will post there all the details.

I don't want Instart Logic people to see my research report.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 14, 2016

@IsraeliAdblocker Please do. If these guys find success, soon all other major blogs and websites will be infested with it and we will be forced to block inline scripts every now and then, so far folks at easylist forum came up with a filter which no longer works and only worsens the situation.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 18, 2016

example.com##script:inject(abort-on-property-write.js, I10C) works on my end.
Example:
Go to http://www.sandiegouniontribune.com
You will get peppered with g00 requests.
Now add sandiegouniontribune.com##script:inject(abort-on-property-write.js, I10C)
no more g00 requests, at least on my end.
Should work with all the mentioned domains.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 18, 2016

example.com##script:inject(abort-on-property-write.js, I10C) works on my end

Tried first site in list, orlandosentinel.com, and the site is rather broken, images won't display.

I suggest:

orlandosentinel.com##script:inject(wowhead.com.js)

Will await feedback.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 18, 2016

orlandosentinel.com displays fine on my end, nothing appears broken,
anyway, if I go to orlandosentinel.com, then clear cookies, add
orlandosentinel.com##script:inject(wowhead.com.js),
then reload this is the logger output filtered for "g00"
g002

and this after I add orlandosentinel.com##script:inject(abort-on-property-write.js, I10C)
g003
and like I said all the pictures display just fine on my end.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 18, 2016

The only mentioned site that doesn´t work on my end is boston.com,
but the issue there seems to be the filter boston.com##script:inject(i10c-defuser.js)
in uBlock filters list, if I disable it, it also works on my end.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 18, 2016

sandiegouniontribune.com##script:inject(abort-on-property-write.js, I10C)

That does stop the onslaught attack /g00 attack, however manipulates and adds /g00/refferr/i to the domain at the address bar, and still adds referrer tracking cookies.

sandiegouniontribune.com##script:inject(wowhead.com.js)

This one stops the attack from happening at the root page, however cookies are still created and inserted to the browser and occasional /g00 ads get loaded silently after few mins.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 18, 2016

Did you clear the cookies before you added the filter?
Because I don`t see any.
g004

Neither is there anything added to the domain.
g005

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 18, 2016

Yes I do, I have page opened in another tab. Please let me finish what I'm testing. Also by cookies I meant third-party cookies which are inserted as first party.

https://i.gyazo.com/d40c182c13f113fb41ddee2a4ac4d5fd.png

using Wowhead reduced the amount crap cookies being inserted however some are still inserted apart from the main domain, even when I'm blocking 3rd party cookies and site data.

Apparently wowhead isn't effective as I thought. I deleted all cookies/site data related to the site sandiegounion tribune and with wowhead filter reloaded again.

Website (after few secs) - https://i.gyazo.com/b5fd844104770562743a921908b52b26.jpg

Cookies - https://i.gyazo.com/513e196f14fa3587ec624ade2a5c3bcf.png

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 18, 2016

Tested with sandiegouniontribune.com##script:inject(i10c-defuser.js)

same result as wowhead, ads manage to load after few secs and crap cookies are being inserted.

@IsraeliAdblocker
Copy link

@IsraeliAdblocker IsraeliAdblocker commented Dec 18, 2016

@gorhill , Please tell me how can I privately share my research with you?

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 18, 2016

@gorhill I can, not reliably, sometimes it works, reproduce a broken orlandosentinel.com
with the filter
orlandosentinel.com##script:inject(abort-on-property-write.js, I10C).

So my previous post was inaccurate. The reason that I couldn´t reproduce was that I didn´t
realize that I used Chrome 49 on that computer.
I still can never reproduce a broken orlandosentinel.com with Chrome 49, but I can with Chrome 55.
Sorry for any confusion I caused.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

orlandosentinel.com##script:inject(wowhead.com.js) works fine on my side: the first load, the page will redirect to a non-g00 version eventually, as it thinks the console is opened, and as a result the Instart Logic code stops doing crappy things (not unlike cockroaches running for hiding spots when turning on the light):

a

There are instances of URL with g00 in it, but it's just the URL of the document itself.

gorhill added a commit that referenced this issue Dec 19, 2016
@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 19, 2016

I now tested Edit: orlandosentinel.com##script:inject(abort-on-property-write.js, I10C and
sandiegouniontribune.com##script:inject(abort-on-property-write.js, I10C)
with Chrome 49, Chrome 55, Firefox 50.1.0 and Microsoft Edge.
The problem you describe only occurs on Chrome 55 for me.
It works fine on all other browsers.
My OS is Windows 10.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 19, 2016

Tried this boston.com,mcall.com,sun-sentinel.com,sandiegouniontribune.com,orlandosentinel.com##script:inject(wowhead.com.js)

With that filter, I tested, both orlando and sandiego, the first load is very slow and takes a lot of time for the loading spinner to stop; still creates some g00 cookies. After refreshing the site upto 3 or 4 times, it becomes normal. /g00 redirection is still there, however it's like a popup defuser, it comes when you click and the URL resets back immediately like it never happened, atleast the website is browsable now. I have yet to test the remaining aforementioned ones for similar behaviour.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

The boston.com one still exhibits the issue with the wowhead.com.js scriplet (I will rename more appropriately eventually). I am investigating -- I added a scriptlet which defuses Instart Logic's ability to detect that the console is opened, so I can freely investigate using dev tools now.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 19, 2016

It seems spoofing user agent string to Firefox's ones works perfectly on Chromium. I'm using uMX for spoofing Firefox's UA and it does the job too. So only Chromium based browsers are affected by this.

Edit - sandiego one still loads slow and injects g00 cookies and other crapware cookies.

Edit2 - doesn't seem to work on sun-sentinel.com, loads ads even after spoofing the UA.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

So only Chromium based browsers are affected by this.

Yes: https://np.reddit.com/r/wow/comments/5exq2d/wowheadcom_sucking_bandwidth/dagbmie/. The server will serve a different document if Firefox (or "not Chrome").

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

Essentially, the g00 URLs are obfuscated URLs to 3rd-parties that would normally be blocked by blockers:

a

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 19, 2016

And those 3rd party urls leave their crap cookies with the help of the script which inserts the cookies as first party ? I already have the Block 3rd party cookies and site data activated, so that's the only way around to insert 3rd party data onto my browser.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

And those 3rd party urls leave their crap cookies with the help of the script which inserts the cookies as first party ?

Looks like this.

@gorhill
Copy link
Member

@gorhill gorhill commented Dec 19, 2016

I find ||boston.com^$inline-script seems to work fine.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Dec 19, 2016

There is no obvious site breakage but www.boston.com/video will not work, if you disable
inline-script.
There is no "g00" crap in the sourcecode of www.boston.com/video, so if you add
@@||www.boston.com/video$inline-script
the video site works.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Dec 19, 2016

Except for boston, the rest of the lot breaks at root page with thumbnails for the articles and videos at any individual article.

@ghost
Copy link

@ghost ghost commented Dec 19, 2016

timeanddate.com loads g00 too. Noticed when a video ad appeared. ##script:inject(abort-on-property-write.js, I10C) breaks the date selection menus that pop up for example on https://www.timeanddate.com/date/dateadd.html when clicking into a field. ||g00.timeanddate.com^$subdocument seems to work.

@jjohns71
Copy link

@jjohns71 jjohns71 commented Jun 28, 2017

https://realclearpolitics.com seems to be using g00 adware now. It doesn't seem to be on their other 'realclear' sites for now, but I wouldn't be surprised if it migrates to them at some point in the future if I were to guess.

Screenshot with a sample of the logger is below:

g00

@simo1994
Copy link

@simo1994 simo1994 commented Jul 30, 2017

I'm just leaving this here for future reference IF they manage to break even uBOExtra.
There is an official Google extension that permits users to spoof their user agent completely, overwriting the navigator.userAgent variable too.
https://chrome.google.com/webstore/detail/user-agent-switcher-for-c/djflhoibgkdhkhhcedjiklpkjnoahfmg

The usage is pretty straightforward. You insert a custom user-agent string, e.g. Firefox's, then you add to the "Permanent Spoof List" the domain . (just one dot, it will work on every domain) or you could add specific domains you know are using the g00 thing. Unfortunately, this has to be done manually.

However, you will probably miss out on some site improvements which are designed for Chrome and you might be logged out from some sites. Don't be creative with user agent strings as you may be blacklisted, please use an actual third-party browser UA string.

@ghajini
Copy link
Contributor

@ghajini ghajini commented Jul 30, 2017

element picker doesn't seem to work
steps to produce=
visit www.sandiegotribune.com
click on news
enter element picker mode,element picker not opened

video=
https://streamable.com/6bajw

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Jul 30, 2017

@ghajini That happens when the HTML is replaced by uBO-Extra.

@gotitbro
Copy link

@gotitbro gotitbro commented Aug 7, 2017

@jspenguin2017 Can you could please elaborate on your above comment?

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Aug 7, 2017

@gotitbro When uBO-Extra aborts the page to replace it, it has the side effect to break element picker. I'm not exactly sure why, I'm guessing that uBO will only inject the element picker once, and it gets overwritten.

@ghajini
Copy link
Contributor

@ghajini ghajini commented Aug 12, 2017

tv.com
hosting g00 .......make ublock extra updates a rolling release same as ublock protector

@Hrxn
Copy link

@Hrxn Hrxn commented Aug 26, 2017

Okay, so this g00 crap is to blame for extremely slowing down some sites, apparently.
I've been busy with restoring a couple of old computers, and it is ridiculous how drastic the difference is on some older system. With new hardware it is probably barely noticeable..

What is the best solution right now? Is there a specific filter? Or using uBO-Extra?
Or uBlock Protector? What would you recommend?

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Aug 26, 2017

For old computers, Firefox is probably better. Firefox is hard on CPU and Chrome is hard on RAM, but if you don't have enough RAM, then Firefox will be faster. g00 currently bails out on Firefox, not sure how long will that last though.

@Hrxn
Copy link

@Hrxn Hrxn commented Aug 26, 2017

Based on my tests, Chrome is definitely better in this scenario. RAM is less of an issue, it's actually the CPU you will notice the most. Unless you have really ridiculously low RAM.

Just one example, really big single HTML documents are a problem, Firefox starts choking very easily while Chrome is still running smoothly.
Just opening multiple tabs to quite heavy sites (news sites, nowadays) or sites with lots of embedded GIFs slow down Chrome noticeably.

Anyway, I was more interested in how to counter that g00 crap, actually 😄

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Aug 26, 2017

The best way to counter it is to use Firefox or, if you prefer Chrome, use uBO-Extra.

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Aug 26, 2017

@okiehsch I wouldn't say so, the script can totally affect Firefox, it is just intentionally aborting on Firefox. It is a switch flip for it to start affecting Firefox.

@okiehsch
Copy link
Contributor

@okiehsch okiehsch commented Aug 26, 2017

I agree, I meant it is the best way to deal with IL-sites at this moment.

@gorhill
Copy link
Member

@gorhill gorhill commented Aug 26, 2017

the script can totally affect Firefox

The IL scripts are not served with Firefox, just compare view-source:http://www.tv.com/ with Firefox vs Chrome.

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Aug 26, 2017

The IL scripts are not served with Firefox

You are saying they cannot serve it to Firefox? I highly doubt that. An user agent will accept anything it receives.

@gorhill
Copy link
Member

@gorhill gorhill commented Aug 26, 2017

You are saying they cannot serve it to Firefox?

Nowhere did I say this.

@Hrxn
Copy link

@Hrxn Hrxn commented Aug 26, 2017

Okay, to conclude, Firefox has the benefit of being unaffected by this plague, at least for now.

And with Chrome (which works better on the old systems here), it is uBlock Origin + uBO-Extra, in preference to just uBO or uBO + uBlock Protector.
Agreed?

@jspenguin2017
Copy link
Contributor

@jspenguin2017 jspenguin2017 commented Aug 26, 2017

uBlock Protector defuses anti-adblock and has uBO-Extra embedded in it. If websites you use don't have anti-adblock then uBlock Origin + uBO-Extra will be faster.
I'm not very sure if Firefox is fully unaffected, as sometime g00 fallbacks to anti-adblock.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Aug 27, 2017

g00 can now be blocked via abort-current-inline-script.js

Example - sandiegouniontribune.com##script:inject(abort-current-inline-script.js, atob, g00)

I haven't tested all but feel free.

PS - This is a scriptlet, uBO will have to win the race condition on every page and every refresh, so if it doesn't work for you or works once or twice it's the race condition at play here, nothing wrong with the filter itself.

@Hrxn
Copy link

@Hrxn Hrxn commented Aug 27, 2017

Does not seem to work for metacritic.com, for example.
metacritic.com##script:inject(abort-current-inline-script.js, atob, g00)

First load did work, I think, but doing a hard reload shows old behaviour.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Aug 27, 2017

I'm not receiving any g00 on metacritic.com even after disabling uBO-Extra, I'm on Chromium 62.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Aug 27, 2017

Tested on orlandosentinel.com and chicagotribune.com. Working as expected.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Aug 27, 2017

Tested on mcall.com, boston.com and sun-sentinal.com. Working as expected.

@Hrxn
Copy link

@Hrxn Hrxn commented Aug 27, 2017

Chrome 60.0.3112.113 (64-bit) (Stable, latest version), uBO 1.13.8, without uBO-Extra.

g00 can now be blocked via abort-current-inline-script.js

I assumed this meant uBO-Extra would not be required here, sorry if I misunderstood this.

@uBlock-user
Copy link
Member Author

@uBlock-user uBlock-user commented Aug 27, 2017

this meant uBO-Extra would not be required here

Yes, that's exactly what I meant, however you may need uBO-Extra for blocking WebRTC connections,although you can use CSP to block them too.

@gorhill
Copy link
Member

@gorhill gorhill commented Aug 27, 2017

I haven't tested all but feel free.

It does not work, that's the entire point of uBO-Extra. That kind of inaccurate information just add noise and confuse further anybody reading this now too huge thread. I already explained in the past why uBO-Extra is needed.

Locking this thread. For any issue with IL stuff, open a new issue at uBO-Extra repo.

To everybody: stick to observed facts please, otherwise this just contribute to propagate myths out there.

This is what we have:

  • IL stuff is not served if user agent is Firefox (correction, not longer true, see below):
    • I have also observed in the past that sometimes IL stuff is not served with development version of Chrome.
    • It seems to be a site-specific server setting. I actually just observed that IL stuff is served for Firefox here:
      • view-source:http://www.orlandosentinel.com/
      • view-source:http://www.sandiegouniontribune.com/
      • view-source:http://www.wowhead.com/
    • But not here:
      • view-source:http://www.tv.com/
    • In any case, easy to actually verify, no need to speculate: view-source:[IL-infested site URL].
  • uBO-Extra is needed to deal with IL stuff because the IL inline script appears at the top of the served HTML page -- before any other secondary resource-pulling tag.
    • uBO-Extra injects its content script declaratively, i.e. on every page, hence it will always run before IL stuff.
  • uBO-Extra no longer deals with WebRTC (gorhill/uBO-Extra@6239f39).
@uBlockOrigin uBlockOrigin locked and limited conversation to collaborators Aug 27, 2017
@gorhill
Copy link
Member

@gorhill gorhill commented Aug 27, 2017

@jspenguin2017 Sorry, you are correct about IL stuff being served on Firefox, I just tested many sites, and I see the IL script at the top with Firefox. I believe this is a new development, I am pretty sure this was not the case until no long ago. So possibly uBO-Extra will be needed for Firefox in the near future.

In any case, I rather discuss all this at uBO-Extra itself, the issue has grown too large and anyways it was solved long ago with the release of uBO-Extra.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet