Cleanup Google related Rulesets

[fuzzyroddis]

Born from this comment: #1063 (comment)

This issue is to track my progress in simplifying and testing Google related rulesets in HTTPSE.

[fuzzyroddis]

    <!--    Subdomains that work on all in google.*
                            -->
    <rule from="^http://(accounts|adwords|finance|groups|id|m|picasaweb|)\.google\.((?:com?\.)?\w{2,3})/"
        to="https://$1.google.$2/" />

Given that there are a lot of google.tld variants I'm wondering should I just have <test>s with 3 different ccTLDs?

[jsha]

That sounds great.

[fuzzyroddis]

Looking for 3 examples of *.corp.google.com.
I've seen a few over the various google project bugtrackers.

Edit:

login.corp.google.com
http://googlesystem.blogspot.com.au/2006/09/behind-corpgooglecom.html
http://ruscoe.net/google/google-subdomains/

[fuzzyroddis]

Anyone know why;

    <rule from="^http://www\.googletagservices\.com/tag/js/"
        to="https://www.googletagservices.com/tag/js/" />

isn't

    <rule from="^http://www\.googletagservices\.com/"
        to="https://www.googletagservices.com/" />

[jsha]

The corp domains are internal to Google. We shouldn't include them in our rulesets because we can't test them.

It looks like the googletagservices thing maybe only advertises the one URL, and the ruleset author was being conservative? https://support.google.com/dfp_sb/answer/2524536?hl=en. I would be fine with changing it to cover the whole domain.

[fuzzyroddis]

corp domains

Almost all of them are accessible from the outside.

[jsha]

Okay, good point. Even so, I don't think our extension should be in the business of rewriting domains that are logically "internal" to a company's operations, as the corp domains are. Besides, the most important one, login.corp.google.com, has an HSTS header.

[fuzzyroddis]

[\w-]+\.corp existed before I touched the ruleset, perhaps whoever added it knew what they were doing?
Do we know of a *.corp.google.com subdomain that doesn't support HTTPS or any reports of it breaking one of them?

Found it: 0fe9a16

[jsha]

@2d1: Looks like you added *.corp.google.com in 2012. What do you think of removing it now that login.corp has an HSTS header?

[fuzzyroddis]

To throw my 5c in, I'm for keeping it. I feel we should force HTTPS as much as possible.

[jsha]

Alright, since you are the one putting in all the effort to refurbish this rule, I will yield to your judgement. :-)

[fuzzyroddis]

I might end up doing all of the Google related rules.

[fuzzyroddis]

2015-02-16 08:08:49,517 ERROR src/chrome/content/rules/GoogleServices.xml: Not enough tests (0 vs 4) for <Rule from 'regex.Regex('^http://(?:www\\.)?google\\.(?:com?\\.)?\\w{2,3}/custom', flags=regex.V0)' to 'https://www.google.com/cse'> [https-everywhere-checker/src/https_everywhere_checker/check_rules.py:232]

        <test url="http://www.google.ca/custom" />
        <test url="http://www.google.com/custom" />
        <test url="http://www.google.com.au/custom" />

Shouldn't these be sufficient?

[jsha]

Looks like it's not matching those URLs against that regex. Are the double backslashes intentional? Could there be an earlier rule interfering?

[fuzzyroddis]

@jsha I assumed the double backslashes were for escaping slashes in a quoted string.
I had a look there is an early rule, I've switched places and will see the result.