Standardize our npm scripts

[nzakas]

We have a number of different projects that are using different conventions for various npm scripts. I'd like us to standardize our script names and what they for the following actions:

• Building - in packages where building is necessary, I'd like to standardize on build. I think we are doing this everywhere, but wanted to include here for the record.
• Linting - I'd like to standardize on lint to lint every type of file, and then lint:*, where * is the file extension, for linting just specific types of files (examples: lint:js, lint:md).
• Fixing lint errors - throughout our packages we have fix, fixlint, lint:fix, and some have no lint fix script at all. I'd like to standardize on lint:fix, which should run on all file types, or else lint:fix:* to run just on a subtype.
• Testing - we are all over the place with this. In some packages test runs both linting and testing. In some packages we have unit:* scripts and in some we have test:* scripts. Some packages have coverage and others do not (test:cover, coverage, test-cov, etc.). I think we should standardize:
  1. That test does not include lint
  2. That we do test coverage
  3. That all testing that can have coverage does have coverage
  4. That test always runs with coverage
  5. That any subset of testing be named test:*. Every test:* is run with test

Thoughts?

[bmish]

I agree with pretty much all of that.

I follow the Ember blueprint, which demonstrates a functional setup for running multiple lint/test scripts in parallel:

"build": "ember build --environment=production",
"lint": "npm-run-all --aggregate-output --continue-on-error --parallel \"lint:!(fix)\"",
"lint:fix": "npm-run-all --aggregate-output --continue-on-error --parallel lint:*:fix",
"lint:hbs": "ember-template-lint .",
"lint:hbs:fix": "ember-template-lint . --fix",
"lint:js": "eslint . --cache",
"lint:js:fix": "eslint . --fix",
"start": "ember serve",
"test": "npm-run-all lint test:*",
"test:ember": "ember test"

In summary:

• Using the file extension (lint:js, lint:md)
• If fixing is available, adding :fix after each script
• lint and test should be independent scripts, and CI should call each of them separately

[snitin315]

Makes sense to me 👍🏻

[mdjermanovic]

My thoughts:

• There should be a command that runs all validations we have in a project (linting, unit tests, license check, etc.). npm test seems to me like the expected way to check everything that should be checked, so I think that npm test should run linting.
• lint:js doesn't have to mean literally only *.js, there is no reason to run ESLint separately for *.cjs files.
• ESLint should be run with eslint ., that seems the best way to not miss linting newly added files/directories. Instead of specifying files/directories that should be linted in the command, we can specify files/directories that shouldn't be linted in .eslintignore or ignorePatterns config property.

[mdjermanovic]

"build": "ember build --environment=production",
"lint": "npm-run-all --aggregate-output --continue-on-error --parallel \"lint:!(fix)\"",
"lint:fix": "npm-run-all --aggregate-output --continue-on-error --parallel lint:*:fix",
"lint:hbs": "ember-template-lint .",
"lint:hbs:fix": "ember-template-lint . --fix",
"lint:js": "eslint . --cache",
"lint:js:fix": "eslint . --fix",
"start": "ember serve",
"test": "npm-run-all lint test:*",
"test:ember": "ember test"

In summary:

• Using the file extension (lint:js, lint:md)

• If fixing is available, adding :fix after each script

• lint and test should be independent scripts, and CI should call each of them separately

lint and test are two scripts, but not independent since test runs lint? By this commit message it seems they're of an opinion that npm test should fail if linting fails.

[bmish]

@mdjermanovic Oh, I didn't notice that the Ember blueprint test script was running both lint and test. I guess that's the convention then so I'm fine either way. It's just slightly inconvenient when you want to run tests locally but not linting.

My thoughts:

• There should be a command that runs all validations we have in a project (linting, unit tests, license check, etc.). npm test seems to me like the expected way to check everything that should be checked, so I think that npm test should run linting.
• lint:js doesn't have to mean literally only *.js, there is no reason to run ESLint separately for *.cjs files.
• ESLint should be run with eslint ., that seems the best way to not miss linting newly added files/directories. Instead of specifying files/directories that should be linted in the command, we can specify files/directories that shouldn't be linted in .eslintignore or ignorePatterns config property.

Agreed with all of that.

One more point, I like to ensure unused disable directive comments don't sneak in. Here are different ways to do that:

1. "lint:js": "eslint --report-unused-disable-directives .",
2. Adding reportUnusedDisableDirectives: true to .eslintrc.js
3. Using eslint-plugin-eslint-comments's recommended config (including eslint-plugin-eslint-comments/no-unused-disable)

Any preference? Historically, I've used eslint-plugin-eslint-comments because it has a bunch of other good recommended rules around comment practices.

[mdjermanovic]

One more point, I like to ensure unused disable directive comments don't sneak in. Here are different ways to do that:

1. "lint:js": "eslint --report-unused-disable-directives .",
2. Adding reportUnusedDisableDirectives: true to .eslintrc.js
3. Using eslint-plugin-eslint-comments's recommended config (including eslint-plugin-eslint-comments/no-unused-disable)

Any preference? Historically, I've used eslint-plugin-eslint-comments because it has a bunch of other good recommended rules around comment practices.

We should use the built-in functionality for reporting unused disable directives: --report-unused-disable-directives and/or reportUnusedDisableDirectives: true.

--report-unused-disable-directives is preferable because it produces error severity (reportUnusedDisableDirectives: true produces warnings). However, it makes sense to use both at the same time, since reportUnusedDisableDirectives: true allows reporting unused directives in editors without any additional configuration. We have recently set reportUnusedDisableDirectives: true in eslint-config-eslint (#14699).

[github-actions]

Oops! It looks like we lost track of this issue. @eslint/eslint-tsc what do we want to do here? This issue will auto-close in 7 days without an update.

[pmcelhaney]

I think this is a great idea and I'm willing to champion it. I'll do some research and add a comment here that fleshes out @nzakas's proposal and incorporates the suggestions in this thread. If you don't hear from me by August 1 please send me a gentle reminder. :)

[nzakas]

@pmcelhaney much appreciated! 🙏 this has been in my list to circle back to.

[pmcelhaney]

Here's a list of the current script names and where they're used, generated with this script.

build                              ;eslint.org, eslintrc, espree, playground
build:eleventy                     ;eslint.org, playground
build:sass                         ;eslint.org, playground
build:webpack                      ;eslint.org, playground
coveralls                          ;doctrine
docs:update-links                  ;eslint
eslint:github-action               ;github-action
fetch                              ;eslint.org
fetch:sponsors                     ;eslint.org
fetch:stats                        ;eslint.org
fetch:team                         ;eslint.org
fix                                ;eslint, eslint.org, eslintrc, playground
fix:docsjs                         ;eslint
fixlint                            ;espree
fuzz                               ;eslint
generate-alpharelease              ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-betarelease               ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-rcrelease                 ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-release                   ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-transcript                ;tsc-meetings
gensite                            ;eslint
images                             ;eslint.org, playground
install:playground                 ;eslint.org, playground
integration-tests                  ;typescript-eslint-parser
jest                               ;typescript-eslint-parser
kill-integration-test-containers   ;typescript-eslint-parser
lint                               ;create-config, doctrine, eslint, eslint-canary, eslint-cli, eslint-plugin-markdown, eslint-tester, eslint-transforms, eslint.org, eslintrc, espree, github-action, playground, tsc-meetings, typescript-eslint-parser
lint:docsjs                        ;eslint
lint:fix                           ;tsc-meetings
major                              ;eslint-tester
minor                              ;eslint-tester
new-rule-format                    ;eslint-transforms
patch                              ;eslint-tester
perf                               ;eslint
postinstall                        ;playground
postversion                        ;eslint-cli
prepare                            ;eslint-plugin-markdown, eslintrc
prepublishOnly                     ;espree
pretest                            ;doctrine, espree
publish-release                    ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
start                              ;eslint.org, playground
sync-docs                          ;espree
test                               ;create-config, doctrine, eslint, eslint-canary, eslint-plugin-markdown, eslint-tester, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
test-cov                           ;eslint-plugin-markdown
test:cli                           ;eslint
test:cov                           ;create-config
unit                               ;espree
unit:cjs                           ;espree
unit:esm                           ;espree
update-tag                         ;github-action
update-version                     ;espree
validate-projects                  ;eslint-canary
watch:eleventy                     ;eslint.org, playground
watch:sass                         ;eslint.org, playground
watch:webpack                      ;eslint.org, playground
webpack                            ;eslint

[pmcelhaney]

Here's a proposed standard which fleshes out @nzakas's original request.

ESlint Package Script Naming Conventions

Names

npm script names MUST contain only lower case letters, : to separates parts, - to separate words, and + to separate file extensions. Each part name SHOULD be either a full English word (e.g. coverage not cov) or a well-known initialism in all lowercase (e.g. wasm).

Here is a summary of the proposal in EBNF.

name = life-cycle | main ":fix"? target? option* ":watch"?

life-cycle = prepare | preinstall | install | postinstall | prepublish | preprepare | prepare | postprepare | prepack | postpack | prepublishOnly;

main = "build" | "lint" | "start" | "test";

target = ":" word ("-" word)* | extension ("+" extension)*;

option = ":" word ("-" word)*;

word = [a-z]+;

extension = [a-z0-9]+;

Order

The script names MUST appear in the package.json file in alphabetical order. The other conventions outlined in this document ensure that alphabetical order will coincide with logical groupings.

Main Script Names

With the exception of npm life cycle scripts all script names MUST begin with one of the following names.

Build

Scripts that generate a set of files from source code and / or data MUST have names that begin with build.

If a package contains any build:* scripts, there MAY be a script named build. If so, SHOULD produce the same output as running each of the build scripts individually. It MUST produce a subset of the output from running those scripts.

Lint

Scripts that statically analyze files (mostly, but not limited to running Eslint itself) MUST have names that begin with lint.

If a package contains any lint:* scripts, there SHOULD be a script named lint and it MUST run all of the checks that would have been run if each lint:* script was called individually.

If fixing is available, a linter MUST NOT apply fixes UNLESS the script contains the :fix modifier (see below).

Start

A start script is used to start a server. As of this writing, no ESLint package has more than one start script, so there's no need start to have any modifiers.

Test

Scripts that execute code in order to ensure the actual behavior matches expected behavior MUST have names that begin with test.

If a package contains any test:* scripts, there SHOULD be a script named test and it MUST run of all of the tests that would have been run if each test:* script was called individually.

A test script SHOULD NOT include linting.

A test script SHOULD report test coverage when possible.

Modifiers

One or more of the following modifiers MAY be appended to the standard script names above. If a target has modifiers, they MUST be in the order in which they appear below (e.g. lint:fix:js:watch not lint:watch:js:fix)

Fix

If it's possible for a linter to fix problems that it finds, add a copy of the script with :fix appended to the end that also fixes.

Target

The name of the target of the action being run. In the case of a build script, it SHOULD identify the build artifact(s), e.g. "javascript" or "css" or "website". In the case of a lint or test script, it SHOULD identify the item(s) being linted or tested. In the case of a start script, it SHOULD identify which server is starting.

A target MAY refer to a list of affected file extensions (such as cjs or less) delimited by a +. If there is more than one extension, the list SHOULD be alphabetized. When a file extension has variants (such as cjs for CommonJS and mjs for ESM), the common part of the extension MAY be used instead of explicitly listing out all of the variants (e.g. js instead of cjs+jsx+mjs).

The target SHOULD NOT refer to name of the name of the tool that's performing the action (eleventy, webpack, etc.)

Options

Additional options that don't fit under the other modifiers.

Watch

If a script watches the filesystem and responds to changes, add :watch to the script name.

---

What's next? Does this need to be an RFC? Or can I create a pull request under https://github.com/eslint/eslint/tree/main/docs/src/developer-guide ?

Once the standard is agreed upon, I'm happy to start sending PRs to update each of the package.json files and code that references the script names.

[pmcelhaney]

Related: https://github.blog/2015-06-30-scripts-to-rule-them-all/

[nzakas]

I like the proposal 👍 Thanks for putting this together.

If others agree, I think we can just document it under Developer Guide and then move forward with PRs to other repos.

@eslint/eslint-tsc what do you think?

[btmills]

This is great! Since this isn't an external API, it's easy to change as we learn, so I agree there's no need for an RFC. Thanks @pmcelhaney!

[nzakas]

Marking as accepted. @pmcelhaney please proceed. 👍