Research: Configuration files (lessons learned from ESLint)

Motivation

Tools like ESLint, TypeScript or Jest have the option of extending config files from other config files. Nx monorepos take advantage of this pattern to create project-specific (or even target-specific) configuration that extends some base configuration (e.g. a project's tsconfig.app.json or tsconfig.spec.json extend a root tsconfig.base.json).

Do we want to support extensible config files for our CLI? If yes, what is the best approach?

There's also a large variety of config file formats used by different tools (JSON, YAML, JavaScript, TypeScript). How many different formats should we support?

ESLint case study

ESLint made big changes to their config files and published a few blog posts detailing the why and how:

• ESLint's new config system, Part 1: Background
• ESLint's new config system, Part 2: Introduction to flat config

ESLint configs gradually became unmaintainable as support for different ways of configuring ESLint were added over the years. So for version 9, they decided to overhaul their config system in favour of a simplified flat config.

The different approaches for composing configs ESLint supported in v8 where:

• configuration cascade
  • ESLint's initial approach to merging configs
  • ESLint would traverse file system from current directory upwards and merge all .eslintrc files
  • also supported "personal config" in home folder
  • later added a root: true option for stopping the cascade
  • 👎 completely dropped in new version (regrets about not dropping earlier), caused unexpected behaviour
• extends key
  • array of relative paths (e.g. ["../../.eslintrc.json"]) or installed ESLint config names (e.g. ["eslint:recommended", "airbnb", "@rx-angular/recommended"])
  • require() used under-the-hood
  • 👎 dropped in new flat config, because of complexity of module resolution
• overrides key
  • array of config objects with added glob-based files and excludedFiles keys
  • turned out to be the best approach
  • added support for extends in an override very confusing
  • 👍 flat configs heavily inspired by this approach

Aside from merging configs, another source of complexity was ESLint supporting many different formats:

• JSON: .eslintrc, .eslintrc.json or "eslintConfig" key in package.json
• YAML: .eslintrc.yml, .eslintrc.yaml
• JavaScript: .eslintrc.js
  • JS configs not fully compatible with other formats

Flat configs simplify this complexity by supporting only the following:

• config file exports an array of glob-based config objects
  • conflicts resolved by array order, last matching config wins
• only JavaScript config files supported (eslint.config.js)
  • config file has to import/require other modules itself (ESLint no longer handles module resolution)

Example eslint.config.js:

import customConfig from "eslint-config-custom";

export default [
    customConfig,
    {
        files: ["**/*.js", "**/*.cjs"],
        rules: {
            "semi": "error",
            "no-unused-vars": "error"
        }  
    },
    {
        files: ["**/*.js"],
        rules: {
            "no-undef": "error",
            "semi": "warn"
        }  
    }
];

Recommendation

There is a lot of complexity in how config files are merged, both for library authors and for their users. We should take inspiration from the lessons learned by the ESLint team.

My main takeaway is we should keep it simple and not add support for configuration options without a clear use case. Specifically, I don't believe we have a compeling reason to support merging our own configs at the moment. Many of the tools we'll be integrating (ESLint, TypeScript, Jest) already have their own config resolutions with support for extending configs and glob-based overrides. That should be enough flexibility for an MVP, so I would stick to a single config file for now, and let each tool resolve their own config.

Also, I think the ESLint team's conclusion to pick JS-only configs is particularly interesting, because it shifts away the burden of module resolution (Common.js vs ES Modules, peer dependencies, ...). This could come in handy for us when it comes to resolving plugins.

An example of our config file might then look something like this:

import eslintPlugin from '@push-up/quality-metrics-plugin-eslint'
import tsPlugin from '@push-up/quality-metrics-plugin-typescript'
import lhciPlugin from '@push-up/quality-metrics-plugin-lighthouse-ci'

/** @type {import('@push-up/quality-metrics-cli').Configuration} */
export default {
  // ...
  plugins: [
    eslintPlugin({ configPath: 'eslint.config.js' }),

    tsPlugin({
      config: {
        ...require('./tsconfig.json'),
        strict: true
      } 
    }),

    lhciPlugin({ 
      config: {
        ci: {
          collect: { /* ... */ }
        }
      } 
    }),

    {
      // some custom plugin object
    }
  ]
}