Fixed adblocking on SFGate.com

[lukemulks]

Test plan

1. Shields Up on Desktop or Tablet (landscape orientation)
2. Go to: http://www.sfgate.com/news/article/before-after-storm-California-flood-ca-drought-10851105.php?google_editors_picks=true#item-44548
3. There should NOT be ads shown

Original issue description

Did you search for similar issues before submitting this one?
Yes

Describe the issue you encountered:
On desktop, and from my Nexus9 Android Tablet (landscape orientation), the following SFGate URL loaded ads:
http://www.sfgate.com/news/article/before-after-storm-California-flood-ca-drought-10851105.php?google_editors_picks=true#item-44548

Ads loaded in tablet: landscape
Ads loaded on desktop (windows10)
Ads did not load on my Nexus5 phone

Expected behavior:
Ads should be blocked, but SFGate is calling the ad library and subsequent logic through two separate, custom calls.

• Platform (Win7, 8, 10? macOS? Linux distro?):

• Windows 10 (desktop - 64bit OS)

• Android OSv7.1.1 (Nexus 9 tablet - landscape orientation)

• Brave Version (revision SHA):
  Windows 10: 0.12.15
  Android: 1.0.10

• Steps to reproduce:

  1. Shields Up on Desktop or Tablet (landscape orientation)
  2. Go to: http://www.sfgate.com/news/article/before-after-storm-California-flood-ca-drought-10851105.php?google_editors_picks=true#item-44548
  3. Ads!

Resolution to test:

SFGate is using 2 separate calls, both custom, to load the request library and load/form the actual ad requests. Details below to resolve.

Request 1 calls Hearst custom ad library _(peas)_: http://aps.hearstnp.com/Scripts/loadAds.js
Request 1: RESPONSE

Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Server: Microsoft-IIS/8.5
X-AspNetMvc-Version: 4.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
X-CDN-Rule: fetch: 20min JS scripts
Content-Length: 372
Accept-Ranges: bytes
Date: Sat, 14 Jan 2017 07:44:44 GMT
Via: 1.1 varnish
Age: 256
X-Served-By: cache-sjc3621-SJC
X-Cache: HIT
X-Cache-Hits: 48
X-Timer: S1484379884.732511,VS0,VE0
Vary: Accept-Encoding
Proxy-Connection: Keep-alive

/*********************** Environment specific variable ***********************/
var loadAd_UrlLocation = ('https:' == document.location.protocol ? 'https:' : 'http:') + "//aps.hearstnp.com/";

/*********************** Load Main file for serving ads ***********************/
var mainFile = loadAd_UrlLocation + 'Scripts/loadAdsMain.js';
document.write('<scr' + 'ipt id="loadAdConfig" type="text/javascript" src="' + mainFile + '"><\/scr' + 'ipt>');

Request 2 calls the actual headers and ad slot definitions to the page _(carrots)_: http://aps.hearstnp.com/SRO/GetJS?url=www.sfgate.com/news/article/before-after-storm-California-flood-ca-drought-10851105.php%3Fgoogle_editors_picks%3Dtrue

(I'll leave the full response to Request 2 off this, but it's long and includes all the ad logic you'd typically observe called to the page head)

The trick they use is to hide the library call for request 1, and then mask request 2 using the environment specific variable to load the URL location as if it was requesting the actual URL.

Blocking this URL should result in a fix
http://aps.hearstnp.com/Scripts/loadAds.js

[lukemulks]

Tested a fix with a custom filter in about:adblock that blocked the hearst calls for this and any other hearst domains outside of SFGate.com.

Closing the issue here, migrating to here: brave/adblock-lists#15 where I am submitting a PR tonight to fully resolve.

Two other shield-evading requests were discovered, one for sailthru (aggregator) and another that Google is using to sniff for blocking and pass data. Those will be submitted with this update.