Infra RFC: Move rust-lang.org from github pages to S3/Cloudfront for SSL support

[edunham]

Right now, the site is hosted on GitHub pages. This is great for convenience but does not allow us to serve it over HTTPS, which is expected (rust-lang/rust#16123, rust-lang/rust#13180, https://internals.rust-lang.org/t/broken-cert-on-https-www-rust-lang-org/2258, #144).

We have a couple options that I see as bad, and one that I'd consider good:

Stick with GitHub Pages and add CloudFlare

As described here, one can get SSL between a CDN and the end-user when using a custom domain on GitHub Pages. The fact that HTTPS can't be set up directly through GitHub is a known isue.

Pros:

• Don't have to move the site
• Adding a CDN should make pages load faster

Cons:

• Connection between CDN and GitHub custom domain remains unencrypted, so the secure connection which the user's browser reports is a lie
• Adds yet another service to our infrastructure, with user management etc. (CloudFlare)

Proxy through our existing Nginx

brson metioned that this would be an option on rust-lang/rust#13180, however, I regard introducing a dependency on our Nginx host as a step backwards in terms of reliability and redundancy. I'd rather offload the work onto a CDN with many sysadmins, rather than centralizing it onto ourselves.

Pros:

• Fewer new moving parts

Cons:

• Slower than a CDN
• Introduces a new single point of failure

Move the static site to S3, use CloudFront as CDN and have them handle SSL

Considering the drawbacks of the other methods, this looks to me like the best option available.

Pros:

• We already have S3 and CloudFront set up with SSL for static.rust-lang.org
• S3 provides comparable uptime/redundancy to github pages (either is better than hosting it ourselves)
• Small step toward standardizing our infrastructure

Cons:

• Needs integration to update pages when site is changed, though that can be done with Travis (http://docs.travis-ci.com/user/deployment/s3/)

@brson, @alexcrichton, @aturon Do you see any concerns that I've missed in evaluating these options, or any reason I shouldn't go ahead with migrating the site over to S3?

[alexcrichton]

I talked with @edunham on IRC, and using CloudFront seems fine to me so long as we integrate with Travis for deploying.

[brson]

sgtm. Thanks @edunham .

[samlh]

I see that the master branch is now set up to deploy to s3, but changes are still going into the gh-pages branch, and the http://www.rust-lang.org/ site dns still points to gh. Is the switch-over still planned? Just curious.

[nodakai]

I've just visited HTTPS versions of rust-lang.org by some chance and got errors.

$ nc -zv rust-lang.org 443
DNS fwd/rev mismatch: rust-lang.org != fwd.st2.easydns.com
rust-lang.org [72.8.141.90] 443 (https) : Connection refused

One more:

$ curl -L https://www.rust-lang.org/
curl: (51) SSL: certificate subject name (www.github.com) does not match target host name 'www.rust-lang.org'

Chrome and Iceweasel (Firefox) are also complaining about the domain.

Debian testing x86_64

[edunham]

@samlh @nodakai Sorry about that; I hadn't checked up on this issue in a while. The DNS isn't yet pointed over to cloudfront, because I dropped the ball on #154 and am just now getting it merged.

After the PR is merged, I'll double check that everything's building right on S3, then switch the DNS, verify that it's working, and close this issue.

[edunham]

After learning a downright silly number of ways that Travis does not work, I have https://travis-ci.org/rust-lang/rust-www/builds/73982330 . DNS switchover after verifying that the S3 site works correctly tomorrow.

[samlh]

Awesome, glad to hear https is coming. Thanks for the follow-up!

[edunham]

It's online, directly at http://www-rust-lang-org.s3-website-us-west-1.amazonaws.com/ and the cloudfront version via https://testsite.rust-lang.org/ .

Moving DNS now.

[edunham]

Buttons pushed. DNS provider reports that the switchover may take up to 3hrs to propagate due to TTL settings. Easiest test of whether the page you're seeing was served from AWS or github is that the AWS ones will have a timestamp in the second line of the page source.

[edunham]

https://www.rust-lang.org/ now resolves and serves the version of the site distributed via AWS.

[alexcrichton]

🎊

Awesome!

[jimmycuadra]

Yay! Thank you for this. I think the new infra should set the Strict-Transport-Security header, too.

[edunham]

I agree that setting the strict-transport-security header would be ideal, but right now CloudFront doesn't support it directly (https://forums.aws.amazon.com/thread.jspa?messageID=651244). We could potentially proxy all www.rust-lang.org traffic through the host that's currently proxying for doc.rust-lang.org to add the headers, but gating on that host would negate the redundancy and availability benefits of using a CDN in the first place. I'll create a new ticket for figuring out the best way to get those headers in place.

[edunham]

Postscript: CloudFront invalidations work a lot better when one gives the Travis user CloudFront perms in AWS. I fixed the perms the other day and the latest PR merged into the site appears to have successfully invalidated.

[nodakai]

http://rust-lang.org/ and http://www.rust-lang.org/ are redirected to https://www.rust-lang.org/ but https://rust-lang.org/ is inaccessible.