CORS header/redirect issues cause geoJSON tile layer to fail on profile pages

[JoeGermuska]

When loading a profile page, the expected map layer showing other geographies of the same summary level is not appearing.

In the console, errors like this appear for each tile:

I feel like something like this has happened before, but I can't recall the circumstances.

My first guess was that the CORS headers just weren't being sent for redirects, because we have the trick where it tries S3 and redirects to the API if the cache is empty.

I have verified that https://s3.amazonaws.com/embed.censusreporter.org/1.0/geo/tiger2015/tiles/050/10/275/409.geojson
is an available file. However, when trying to curl it using the CR domain:

https://embed.censusreporter.org/1.0/geo/tiger2015/tiles/050/10/275/409.geojson

I get a 301 redirect (with no CORS header, but that's beginning to look like a corollary to the real issue) The 301 comes from Cloudfront:

HTTP/1.1 301 Moved Permanently
Content-Type: text/html
Content-Length: 193
Connection: keep-alive
Date: Mon, 28 Nov 2016 02:36:33 GMT
Location: https://api.censusreporter.org/1.0/geo/tiger2015/tiles/050/10/275/409.geojson
Server: nginx/1.4.6 (Ubuntu)
Age: 667
X-Cache: Hit from cloudfront
Via: 1.1 34f097a2c19d8752d51d599eaf6a7366.cloudfront.net (CloudFront)
X-Amz-Cf-Id: e-8QNG2oCAwd4dGvzr4uPQcHUWdioqEf1W4MofmRYS7zaJG3ooDaVw==

Here's the config for the embed.censusreporter.org S3 bucket:

<RoutingRules>
    <RoutingRule>
        <Condition>
            <KeyPrefixEquals>1.0/geo/tiger2013/tiles/</KeyPrefixEquals>
            <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
        </Condition>
        <Redirect>
            <HostName>api.censusreporter.org</HostName>
            <ReplaceKeyPrefixWith>1.0/geo/tiger2013/tiles/</ReplaceKeyPrefixWith>
            <HttpRedirectCode>302</HttpRedirectCode>
        </Redirect>
    </RoutingRule>
    <RoutingRule>
        <Condition>
            <KeyPrefixEquals>1.0/geo/tiger2014/tiles/</KeyPrefixEquals>
            <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
        </Condition>
        <Redirect>
            <HostName>api.censusreporter.org</HostName>
            <ReplaceKeyPrefixWith>1.0/geo/tiger2014/tiles/</ReplaceKeyPrefixWith>
            <HttpRedirectCode>302</HttpRedirectCode>
        </Redirect>
    </RoutingRule>
    <RoutingRule>
        <Condition>
            <KeyPrefixEquals>1.0/geo/tiger2015/tiles/</KeyPrefixEquals>
            <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
        </Condition>
        <Redirect>
            <HostName>api.censusreporter.org</HostName>
            <ReplaceKeyPrefixWith>1.0/geo/tiger2015/tiles/</ReplaceKeyPrefixWith>
            <HttpRedirectCode>302</HttpRedirectCode>
        </Redirect>
    </RoutingRule>
</RoutingRules>

[JoeGermuska]

Verified that the CORS header is present on direct requests to the API:

curl -D /tmp/api.txt https://api.censusreporter.org/1.0/geo/tiger2015/tiles/050/10/275/409.geojson

/tmp/api.txt:

Access-Control-Allow-Methods: HEAD, OPTIONS, GET
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 21600
Cache-Control: public,max-age=86400
Content-Type: application/json
Date: Mon, 28 Nov 2016 02:56:01 GMT
Server: nginx/1.4.6 (Ubuntu)
Content-Length: 16052
Connection: keep-alive

[iandees]

This was an unintended side-effect of the changes I made to (1) support SSL on embed.censusreporter.org URLs and (2) redirection to SSL. It's fixed now, but I'll throw some extra details below to hopefully help us if we run into this in the future.

Back when we added SSL support to the site, I implemented it using Amazon's free SSL certificates. The load balancer that sits in front of our instances uses the free SSL certificate to serve api.censusreporter.org and censusreporter.org over HTTPS. This worked well because the ELB would listen on 80 and 443 for HTTP and HTTPS respectively and would forward the requests on to the instances' nginx servers on port 80 with HTTP. There was no configuration or code change required for the api or django apps.

However, when users visited the site on HTTPS, the vector tiles on the profile page wouldn't load because they were sent over HTTP and browsers don't like it when a secure page requests insecure content. At the time, we were serving tiles from the embed.censusreporter.org S3 bucket using S3's "static website hosting" service. The static website hosting service does not support SSL. We were using static website hosting because it allowed us to forward the user's request to the API when the tile didn't exist in S3 already, saving CPU usage on our app servers.

To solve this, I set up a CloudFront distribution using the same SSL certificate as the ELB above. The CloudFront distribution serves the embed.censusreporter.org URL and uses the embed.censusreporter.org S3 bucket as an origin by default. Tile requests are forwarded to the api.censusreporter.org app server, which checks the S3 bucket and returns data from S3 if the tile exists already, otherwise it generates the tile by querying PostGIS and returns the data (while also adding to memcache and S3).

Later, we wanted to redirect all of our visitors to HTTPS. To do this, I changed the nginx configuration so that it would listen to two ports: 80 and 1443. The server listening on port 80 redirects the browser to the HTTPS version of whatever URL was requested. The server listening on port 1443 would serve content without doing any redirection. Since the ELB and CloudFront are actually doing the SSL for us, neither of these servers used SSL, thus the odd 1443 port number. When I set this up, I told the ELB to direct HTTPS requests to port 1443 and regular HTTP requests to port 80. This way, when a user makes an HTTP request to api.censusreporter.org or censusreporter.org, the ELB will make the request to the nginx on port 80 and they will be redirected to the HTTPS version of the page with an HTTP 301 redirect. When they make the same request with HTTPS, it will go to nginx on port 1443 and the data will be returned directly (without a redirect).

This leads us to the problem in this ticket: when the embed.censusreporter.org CloudFront distribution made its tile requests to api.censusreporter.org, it was configured to only use HTTP on port 80. Our nginx on port 80 would respond with the HTTP 301 redirect, which CloudFront would cache and send back to the user. Additionally, CloudFront's default configuration is to not pass along the CORS headers from the API, so the browser would not follow the redirect. To fix this issue, I reconfigured the CloudFront to talk HTTPS to api.censusreporter.org when requesting tiles (which skips the redirect on nginx:80) and added the CORS headers to the whitelist for passthrough from the server.