Standardize our npm scripts

[nzakas]

We have a number of different projects that are using different conventions for various npm scripts. I'd like us to standardize our script names and what they for the following actions:

• Building - in packages where building is necessary, I'd like to standardize on build. I think we are doing this everywhere, but wanted to include here for the record.
• Linting - I'd like to standardize on lint to lint every type of file, and then lint:*, where * is the file extension, for linting just specific types of files (examples: lint:js, lint:md).
• Fixing lint errors - throughout our packages we have fix, fixlint, lint:fix, and some have no lint fix script at all. I'd like to standardize on lint:fix, which should run on all file types, or else lint:fix:* to run just on a subtype.
• Testing - we are all over the place with this. In some packages test runs both linting and testing. In some packages we have unit:* scripts and in some we have test:* scripts. Some packages have coverage and others do not (test:cover, coverage, test-cov, etc.). I think we should standardize:
  1. That test does not include lint
  2. That we do test coverage
  3. That all testing that can have coverage does have coverage
  4. That test always runs with coverage
  5. That any subset of testing be named test:*. Every test:* is run with test

Thoughts?

[bmish]

I agree with pretty much all of that.

I follow the Ember blueprint, which demonstrates a functional setup for running multiple lint/test scripts in parallel:

"build": "ember build --environment=production",
"lint": "npm-run-all --aggregate-output --continue-on-error --parallel \"lint:!(fix)\"",
"lint:fix": "npm-run-all --aggregate-output --continue-on-error --parallel lint:*:fix",
"lint:hbs": "ember-template-lint .",
"lint:hbs:fix": "ember-template-lint . --fix",
"lint:js": "eslint . --cache",
"lint:js:fix": "eslint . --fix",
"start": "ember serve",
"test": "npm-run-all lint test:*",
"test:ember": "ember test"

In summary:

• Using the file extension (lint:js, lint:md)
• If fixing is available, adding :fix after each script
• lint and test should be independent scripts, and CI should call each of them separately

[snitin315]

Makes sense to me 👍🏻

[mdjermanovic]

My thoughts:

• There should be a command that runs all validations we have in a project (linting, unit tests, license check, etc.). npm test seems to me like the expected way to check everything that should be checked, so I think that npm test should run linting.
• lint:js doesn't have to mean literally only *.js, there is no reason to run ESLint separately for *.cjs files.
• ESLint should be run with eslint ., that seems the best way to not miss linting newly added files/directories. Instead of specifying files/directories that should be linted in the command, we can specify files/directories that shouldn't be linted in .eslintignore or ignorePatterns config property.

[mdjermanovic]

"build": "ember build --environment=production",
"lint": "npm-run-all --aggregate-output --continue-on-error --parallel \"lint:!(fix)\"",
"lint:fix": "npm-run-all --aggregate-output --continue-on-error --parallel lint:*:fix",
"lint:hbs": "ember-template-lint .",
"lint:hbs:fix": "ember-template-lint . --fix",
"lint:js": "eslint . --cache",
"lint:js:fix": "eslint . --fix",
"start": "ember serve",
"test": "npm-run-all lint test:*",
"test:ember": "ember test"

In summary:

• Using the file extension (lint:js, lint:md)

• If fixing is available, adding :fix after each script

• lint and test should be independent scripts, and CI should call each of them separately

lint and test are two scripts, but not independent since test runs lint? By this commit message it seems they're of an opinion that npm test should fail if linting fails.

[bmish]

@mdjermanovic Oh, I didn't notice that the Ember blueprint test script was running both lint and test. I guess that's the convention then so I'm fine either way. It's just slightly inconvenient when you want to run tests locally but not linting.

My thoughts:

• There should be a command that runs all validations we have in a project (linting, unit tests, license check, etc.). npm test seems to me like the expected way to check everything that should be checked, so I think that npm test should run linting.
• lint:js doesn't have to mean literally only *.js, there is no reason to run ESLint separately for *.cjs files.
• ESLint should be run with eslint ., that seems the best way to not miss linting newly added files/directories. Instead of specifying files/directories that should be linted in the command, we can specify files/directories that shouldn't be linted in .eslintignore or ignorePatterns config property.

Agreed with all of that.

One more point, I like to ensure unused disable directive comments don't sneak in. Here are different ways to do that:

1. "lint:js": "eslint --report-unused-disable-directives .",
2. Adding reportUnusedDisableDirectives: true to .eslintrc.js
3. Using eslint-plugin-eslint-comments's recommended config (including eslint-plugin-eslint-comments/no-unused-disable)

Any preference? Historically, I've used eslint-plugin-eslint-comments because it has a bunch of other good recommended rules around comment practices.

[mdjermanovic]

One more point, I like to ensure unused disable directive comments don't sneak in. Here are different ways to do that:

1. "lint:js": "eslint --report-unused-disable-directives .",
2. Adding reportUnusedDisableDirectives: true to .eslintrc.js
3. Using eslint-plugin-eslint-comments's recommended config (including eslint-plugin-eslint-comments/no-unused-disable)

Any preference? Historically, I've used eslint-plugin-eslint-comments because it has a bunch of other good recommended rules around comment practices.

We should use the built-in functionality for reporting unused disable directives: --report-unused-disable-directives and/or reportUnusedDisableDirectives: true.

--report-unused-disable-directives is preferable because it produces error severity (reportUnusedDisableDirectives: true produces warnings). However, it makes sense to use both at the same time, since reportUnusedDisableDirectives: true allows reporting unused directives in editors without any additional configuration. We have recently set reportUnusedDisableDirectives: true in eslint-config-eslint (#14699).

[github-actions]

Oops! It looks like we lost track of this issue. @eslint/eslint-tsc what do we want to do here? This issue will auto-close in 7 days without an update.

[pmcelhaney]

I think this is a great idea and I'm willing to champion it. I'll do some research and add a comment here that fleshes out @nzakas's proposal and incorporates the suggestions in this thread. If you don't hear from me by August 1 please send me a gentle reminder. :)

[nzakas]

@pmcelhaney much appreciated! 🙏 this has been in my list to circle back to.

[pmcelhaney]

Here's a list of the current script names and where they're used, generated with this script.

build                              ;eslint.org, eslintrc, espree, playground
build:eleventy                     ;eslint.org, playground
build:sass                         ;eslint.org, playground
build:webpack                      ;eslint.org, playground
coveralls                          ;doctrine
docs:update-links                  ;eslint
eslint:github-action               ;github-action
fetch                              ;eslint.org
fetch:sponsors                     ;eslint.org
fetch:stats                        ;eslint.org
fetch:team                         ;eslint.org
fix                                ;eslint, eslint.org, eslintrc, playground
fix:docsjs                         ;eslint
fixlint                            ;espree
fuzz                               ;eslint
generate-alpharelease              ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-betarelease               ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-rcrelease                 ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-release                   ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
generate-transcript                ;tsc-meetings
gensite                            ;eslint
images                             ;eslint.org, playground
install:playground                 ;eslint.org, playground
integration-tests                  ;typescript-eslint-parser
jest                               ;typescript-eslint-parser
kill-integration-test-containers   ;typescript-eslint-parser
lint                               ;create-config, doctrine, eslint, eslint-canary, eslint-cli, eslint-plugin-markdown, eslint-tester, eslint-transforms, eslint.org, eslintrc, espree, github-action, playground, tsc-meetings, typescript-eslint-parser
lint:docsjs                        ;eslint
lint:fix                           ;tsc-meetings
major                              ;eslint-tester
minor                              ;eslint-tester
new-rule-format                    ;eslint-transforms
patch                              ;eslint-tester
perf                               ;eslint
postinstall                        ;playground
postversion                        ;eslint-cli
prepare                            ;eslint-plugin-markdown, eslintrc
prepublishOnly                     ;espree
pretest                            ;doctrine, espree
publish-release                    ;create-config, doctrine, eslint, eslint-plugin-markdown, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
start                              ;eslint.org, playground
sync-docs                          ;espree
test                               ;create-config, doctrine, eslint, eslint-canary, eslint-plugin-markdown, eslint-tester, eslint-transforms, eslintrc, espree, github-action, typescript-eslint-parser
test-cov                           ;eslint-plugin-markdown
test:cli                           ;eslint
test:cov                           ;create-config
unit                               ;espree
unit:cjs                           ;espree
unit:esm                           ;espree
update-tag                         ;github-action
update-version                     ;espree
validate-projects                  ;eslint-canary
watch:eleventy                     ;eslint.org, playground
watch:sass                         ;eslint.org, playground
watch:webpack                      ;eslint.org, playground
webpack                            ;eslint