[RFC]: A Conformance system for NextJS

[prateekbh]

tldr; A webpack based conformance system which can look into all source files and generated assets for forbidden anti patterns(in 1P code as well as node modules) and fail the build(or warn) if it does not conform to a preset array of tests.

Feature request

Is your feature request related to a problem? Please describe.

For a website to have a good performance, a developer or a team of developers generally need to have an in-depth understanding of many pieces in the web ecosystem. These include understanding how a framework works, how numerous web APIs works and how to configure the build system.
However, meta frameworks like NextJS make configuring the build system extremely easy. It is still not impossible to mess up the configuration and degrade the website’s performance.
Also, there are way too many gotchas and traps that one can fall into while focussing on the product features and chasing deadlines.
A conformance system will offload the necessity of having this deep understanding of every performance foot gun from every developer to a set of tests that can reside in the build system and fail the build or throw warnings at the end of the build process.

Common performance foot guns

After tracing a bunch of NextJS and other framework websites we have gathered a small set of conformance tests, to begin with. These tests are based on some frequent things that we see delaying FCP/TTI for websites.

• Synchronous scripts
  A synchronous script tag blocks the HTML parsers from parsing the rest of the document till the given resource is downloaded and executed. NextJS take specific steps to avoid this for the scripts it generates, but users can easily inject one for third party synchronous scripts. This test will scan such sync script tags and report them.

Violation of this would result in a total failure of the build.

• Image quality
  Very often the images served from the static/public folder are used by developers as is in their web apps. This system would generate warnings if it finds any resource which is not optimized for web usage and would let the developer know about the same

Violation of this would result in warning around each of those resources not optimized for the web.

• Web Fonts
  Adding a custom font to your web app typically requires a developer to add a simple stylesheet however, browsers don’t go ahead and download these fonts unless used somewhere. Which means a waterfall of other fonts request after some stylesheet uses it, defined in a previous stylesheet.
  Making sure that a font is preloaded along with inclusion would be another conformance check to help keep the first meaningful paint healthy.

• Disabled minification
  Although NextJS recently started throwing an error if the minification is disabled but this proposal shifts the same to the conformance layer.

Violation of this would result in a total failure of the build.

Describe the solution you'd like

A webpack based conformance system that can inspect the code being bundled would be able to throw these errors and warning while webpack parses the code itself. This conformance system could also look at generated assets and the webpack configuration to make sure that the changes made by the user would not regress the web app’s performance.

NextJS can create and add a Webpack Conformance plugin to its configuration "just" before starting the actual compilation. This is essential because such a system would need insight into all the assets generated by all the plugins/child compilers and hence should be the last plugin in the plugins list. Adding this plugin in the static config would make it ignore assets generated by plugins added in next.config.js

e.g.

// next.config.js
module.export = {
  webpack: config => {
    config.plugins.push(new FancyWebPackPlugin())
  }
}

This would make WebpackConformancePlugin ignore assets from FancyWebPackPlugin as it is after the conformance plugin.

Sample output

The plugin will look into all the assets/source code and collect all the errors/warnings from all the tests throughout all the lifecycles and post them all together after webpack’s build is done.

The errors and warnings can be in the following format

> yarn next build
> yarn run v1.15.2
Creating an optimized production build ...

Found experimental config:
Experimental features can change at anytime and aren't officially supported (use at your own risk).
Failed to compile.

[BUILD CONFORMANCE]: Minification for this build is disabled. This can have a very serious impact on load times of the web app.
[BUILD CONFORMANCE]: An inline script was found in a react module. in /pages/home.js

Note: The solution could be extended to similar checks in the ESLint space but would not suffice as it would have no insight on the third party code. Keeping these checks primarily in the build system makes sure that just installing and consuming a rogue node_module would not blow up the performance of a carefully written web app.

Exempting from a specific rule

Similar to all userland configuration, the disabling of a rule should be controlled from next.config.js. Next could read a new key Conformance and get the desired settings of every rule from this config.

e.g.

// next.config.js
module.exports = {
  //…
  conformance: {
    synchronous-scripts-test: {
      disabled: true,
      reason: ‘We rely on synchronous scripts for our analytics to work correctly’
    }
  }
}

Disabling a rule in a directory

There can be cases like an ongoing migration where the whole app is not ready to conform to a rule. Users can add directory level exceptions to a particular rule.
Users can add a .conformance file and add an exception in the same as follows

//.conformance
{
  synchronous-scripts-test: {
      disabled: true,
      reason: ‘We rely on synchronous scripts for our analytics to work correctly’
    }
}