Implement gzip and brotli compression in production

[CLTPayne]

Introducing gzip and brotli compression for production builds, addressing the file generation portion of #566.

Before this work:

Page Kit was relying on the built-in Fastly on-the-fly compression which favours performance over maximum smallness.

After this work:

• Building for production will generate a gzip and brotli compressed version of all files, as well as the original bundle.js.

Demo:

Tested this new webpack configuration in the kitchen-sink application:

Production build files before:

Production build files after (no longer compressing source maps):

Notes / Considerations:

• Have tried to follow the current pattern for webpack plugin options by providing options objects where
  a) a consumer of Page Kit can extend / amend the compression settings
  b) a developer can see how the compression is configured.
• The following is actually the default values for this compression. I felt including this explicitly would be useful for developer understanding / quick reference but realise it does mean we'd be pinning this behaviour and miss out on changes to the plugin defaults (if that's likely).

  const compressionPluginOptions = { algorithm: 'gzip', compressionOptions: { level: 9 } }
  const brotliPluginOptions = { quality: 11 }

• Currently we generate source maps for production (@adambraimbridge I found this surprising so did some research and found some interesting opinions)
• Keeping the original files after compression per @sjparkinson (Generate compressed assets with compression turned up to 11 #566 (comment))
• Debated whether source maps should also be compressed and decided that anything we ship should be compressed. Perhaps there are alternative opinions and we should reduce the number of files but as the source maps are only downloaded if dev tools are open so perhaps not a concern? UPDATE - Discussed with @i-like-robots and agreed that source maps should not be compressed.
• There is the option to set a minRatio for the compression algorithm or threshold for minimum file size to run through the algorithms but as discussed with @i-like-robots this would lead to more 'surprising' complexity in return for a very slightly quicker build time so have not implemented. UPDATE - Have set both plugins to compress ALL files by setting the minRatio: 1 so that we can guarantee the CDN will find a .br and .gz for all .js and .css files.
• Have not edited the Storybook webpack config as feel these changes do not apply to storybook.

Testing:

• Decided against adding further tests to the kitchen-sink build tests as they currently run against the development build. Is this compression worth adding test step up for testing the production build?
• Currently no tests for the dotcom-page-kit-cli

[sjparkinson]

There is the option to set a minRatio for the compression algorithm or threshold for minimum file size to run through the algorithms but as discussed with @i-like-robots this would lead to more 'surprising' complexity in return for a very slightly quicker build time so have not implemented.

Ahhhhh this makes sense now.

Brotli, there are strong arguments that you should not use it for small files, but this focuses on on-the-fly compression which we are not doing. So I think it's okay to press on with this approach.

If you'd like to pair on the change to the CDN let me know.

[CLTPayne]

If you'd like to pair on the change to the CDN let me know.

@sjparkinson yes!

@i-like-robots changed his mind and would like to get this implemented and see the benefits so 👍to pairing on the CDN update once the manifest is picking up the compressed files (which I'm working on now).

[adambraimbridge]

Re: CDN, Check this out for what it's worth: https://github.com/Financial-Times/ft.com-cdn/pull/573/files

[i-like-robots]

Requires Financial-Times/next-ci-shared-helpers#21