Setup prod hosting of static media for AWS prod

[pmac]

For dev we'll probably just serve static media from the primary docker container via whitenoise, but that won't scale to prod traffic. We have some options:

• Sync static files to S3 and serve via a CDN
• Deploy a separate Nginx container for serving said files, optionally with a CDN in front
• Put a CDN in front of the entire site (potentially a lot more work)

[glogiotatidis]

Deploy a separate Nginx container for serving said files, optionally with a CDN in front

In this case we can also build a single image that will either run nginx-light or gunicorn depending on the value of a env variable. This would simply the building pipeline, keep the statics with the code, while we keep the flexibility to scale each function (static, app) seperately.

[bookshelfdave]

regarding S3 / Cloudfront: mozmeao/infra#391 (click through to the viewsource issue)

[glogiotatidis]

I suggest that we go fwd with the following plan:

1. Enable WhiteNoise for all dev, stage, prod
2. Setup CDN to serve only static files and not the whole site. See instructions from WhiteNoise docs.

This way:

• Statics remain with the code and we build only one docker image
• We don't have to manage S3 headers and deal with other S3 oddities
• We don't maintain nginx deployments
• Avoid the work and problems related to putting the entire site behind CDN.
• We maintain current SEO as the content will be only accessible through support.m.o and not the CDN url.

To move forward we need:

• 3 CDNs (dev, stage, prod) configured as described in WhiteNoise docs.

• DNS entries for the CDNs above. Names static-media-{dev, stage, prod}-cdn.sumo.mozilla.net CNAME to static-media-{dev,stage,prod}-cdn.sumo.moz.works

• Switch to Whitenoise Storage backend (code) (see Setup prod hosting of static media for AWS prod #2949 (comment))

• Bonus points by enabling Brotli compression (docker images need update to include packages)
  (see Setup prod hosting of static media for AWS prod #2949 (comment))

What do you think @mozilla/sumo?

[bookshelfdave]

How do these CDN setting look? mozmeao/infra#694

[glogiotatidis]

Switch to Whitenoise Storage backend (code)
Bonus points by enabling Brotli compression (docker images need update to include packages)

We're currently using Pipeline which takes care of static files collection and generates bundles with the assets. Pipeline works OK and the bundles are useful and in addition to what whitenoise offers. So we keep the Pipeline based Storage backend as we have it now.

I tested the current Whitenoise implementation and it sets proper headers for static files
Cache-Control: max-age=315360000, public, immutable.

tl;dr no changes needed on the code side to move fwd with this. We can evaluate replacing Pipeline with something different / more modern in the future.

[bookshelfdave]

Setup CDN to serve only static files and not the whole site. See instructions from WhiteNoise docs.

per the WhiteNoise docs, each CDN is limited to static/*.

(See the Behaviors tab for each CDN below)

• dev
• prod
• stage

[glogiotatidis]

I did setup sumo-dev.frankfurt.moz.works to use dev CDN and it works like a charm. nice work as always @metadave

I guess all we need now is the mozilla.net dns entries.

[bookshelfdave]

Sorry, should have dropped the bug # in when I created it last week:
https://bugzilla.mozilla.org/show_bug.cgi?id=1433561

[glogiotatidis]

This is now done. Update from bug

CNAMEs created.

FQDN 		Target 	
static-media-dev-cdn.sumo.mozilla.net 	CNAME 	static-media-dev-cdn.sumo.moz.works


FQDN 		Target 	
static-media-stage-cdn.sumo.mozilla.net 	CNAME 	static-media-stage-cdn.sumo.moz.works


FQDN 		Target 	
static-media-prod-cdn.sumo.mozilla.net 	CNAME 	static-media-prod-cdn.sumo.moz.works```