Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: support glob matching coverage files #53553

Merged
merged 1 commit into from
Jul 14, 2024

Conversation

RedYetiDev
Copy link
Member

@RedYetiDev RedYetiDev commented Jun 22, 2024

Fixes #53508
Fixes #51222

This Pull Request introduces two new command-line interface flags to Node.js:

  • --test-coverage-include: Allows users to specify glob patterns to include specific files in the coverage report.
  • --test-coverage-exclude: Allows users to specify glob patterns to exclude specific files from the coverage report.

Notable change text, if any:

Individual file patterns can now be excluded or exclusively included in coverage reports:

- To exclude a specific pattern, use `--test-coverage-exclude`. For example:
  ```bash
  node --experimental-test-coverage --test-coverage-exclude=**/*.test.js .
  ```
  This command excludes all files ending with `.test.js` from the coverage report.

- To include only files that match a specific pattern, use `--test-coverage-include`. For example:
  ```bash
  node --experimental-test-coverage --test-coverage-include=src/**/*.js .
  ```
  This command includes only `.js` files located in the `src` directory and its subdirectories in the coverage report.

Both options can be specified multiple times to match multiple glob patterns. If both options are used in unison, files will need to abide by both guidelines.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Jun 22, 2024
@RedYetiDev
Copy link
Member Author

RedYetiDev commented Jun 22, 2024

Blocked by #52881 - This PR relies on the path.matchesGlob method to match the coverage files.

(Test failures will disappear once the blocking PR lands)


If a PR relies on a semver-minor change, does it become semver-minor?

@RedYetiDev RedYetiDev added blocked PRs that are blocked by other issues or PRs. cli Issues and PRs related to the Node.js command line interface. coverage Issues and PRs related to native coverage support. labels Jun 22, 2024
doc/api/cli.md Outdated Show resolved Hide resolved
doc/api/cli.md Outdated Show resolved Hide resolved
@MoLow
Copy link
Member

MoLow commented Jun 23, 2024

please take a look at the test failures

@MoLow MoLow removed c++ Issues and PRs that require attention from people who are familiar with C++. cli Issues and PRs related to the Node.js command line interface. coverage Issues and PRs related to native coverage support. labels Jun 23, 2024
@RedYetiDev RedYetiDev removed the blocked PRs that are blocked by other issues or PRs. label Jun 23, 2024
@RedYetiDev
Copy link
Member Author

Thanks! I'll made those changes!

I've also unblocked the PR because the blocking PR has landed

@RedYetiDev RedYetiDev force-pushed the coverage-patterns branch 2 times, most recently from f5f51e9 to ad28311 Compare June 23, 2024 17:54
@RedYetiDev RedYetiDev requested a review from MoLow June 23, 2024 18:20
Copy link
Member

@MoLow MoLow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@rozzilla rozzilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one! ❤️
About the run option, will those new options be supported?
Ideally, as we have files option, it'd be nice to be able to pass the coverage exclusion files too.

@RedYetiDev
Copy link
Member Author

Nice one! ❤️

About the run option, will those new options be supported?

Ideally, as we have files option, it'd be nice to be able to pass the coverage exclusion files too.

I'm not planning on that, this is specific to coverage, not all of testing.

Copy link
Contributor

@cjihrig cjihrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the run option, will those new options be supported?

We should probably make all coverage options available via run() prior to moving this out of experimental status.

src/node_options.cc Outdated Show resolved Hide resolved
doc/api/cli.md Show resolved Hide resolved
doc/api/cli.md Outdated Show resolved Hide resolved
lib/internal/test_runner/coverage.js Outdated Show resolved Hide resolved
lib/internal/test_runner/coverage.js Outdated Show resolved Hide resolved
for (let i = 0; i < excludeFileGlobs.length; ++i) {
const absolutePath = fileURLToPath(url);
const relativePath = relative(workingDirectory, absolutePath);
if (matchesGlob(relativePath, excludeFileGlobs[i]) || matchesGlob(absolutePath, excludeFileGlobs[i])) return true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems pretty inefficient to check the same path as both an absolute path and a relative path. Same comment below for included globs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you recommend doing instead?

I assume users would want relative checks, but if the path is somewhere completely different, they may also want absolute?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So one issue is that you seem to be computing absolutePath and relativePath for each iteration of the loop, even though I believe they only need to be computed once per function call.

The other thing is, since you already have absolute paths, shouldn't it be enough to glob against those? I guess I don't see anything in the tests added by this PR that would require converting url to both an absolute and relative path. This would have a nice side effect of not needing to pass workingDirectory around as well. Do you have an example where that wouldn't be the case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can fix the position of the generation, but here is my reasoning for both relative and absolute testing:

var { minimatch } = require("minimatch")
var path = require("path");

const cwd = "/home/user"
const absTestFile = "/home/user/path/to/file.js"
const relTestFile = path.relative(cwd, absTestFile);

console.log("relTestFile: " + relTestFile + " | Minimatch: " + minimatch(relTestFile, "path/to/*.js")) // true
console.log("absTestFile: " + absTestFile + " | Minimatch: " + minimatch(absTestFile, "path/to/*.js")) // false

Users probably want to match the relative test file when excluding coverage files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, fair enough.

@RedYetiDev RedYetiDev force-pushed the coverage-patterns branch 2 times, most recently from bc609d0 to 9efbc49 Compare June 24, 2024 14:36
@RedYetiDev
Copy link
Member Author

RedYetiDev commented Jun 24, 2024

I need to review the new test failures, I'll address them when I get a chance

@nodejs-github-bot
Copy link
Collaborator

@RedYetiDev
Copy link
Member Author

@MoLow did you ever get a chance to prepare for manually land?

@nodejs-github-bot
Copy link
Collaborator

@RedYetiDev
Copy link
Member Author

CI passed 🎉

@benjamingr benjamingr added commit-queue Add this label to land a pull request using GitHub Actions. notable-change PRs with changes that should be highlighted in changelogs. labels Jul 14, 2024
Copy link
Contributor

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @benjamingr.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 14, 2024
@nodejs-github-bot nodejs-github-bot merged commit 4174b73 into nodejs:main Jul 14, 2024
40 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 4174b73

aduh95 pushed a commit that referenced this pull request Jul 16, 2024
PR-URL: #53553
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@aduh95
Copy link
Contributor

aduh95 commented Jul 16, 2024

This was merged with the wrong subsystem (it should be test_runner: IIUC)

aduh95 pushed a commit that referenced this pull request Jul 16, 2024
PR-URL: #53553
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
aduh95 added a commit that referenced this pull request Jul 16, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) #53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) #53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) #52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) #52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) #53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) #53462
test_runner:
  * support glob matching coverage files (Aviv Keller) #53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) #53682

PR-URL: #53826
@aduh95 aduh95 mentioned this pull request Jul 16, 2024
@RedYetiDev RedYetiDev changed the title test: support glob matching coverage files test_runner: support glob matching coverage files Jul 16, 2024
aduh95 added a commit that referenced this pull request Jul 16, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) #53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) #53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) #52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) #52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) #53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) #53462
test_runner:
  * support glob matching coverage files (Aviv Keller) #53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) #53682

PR-URL: #53826
aduh95 added a commit that referenced this pull request Jul 16, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) #53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) #53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) #52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) #52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) #53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) #53462
test_runner:
  * support glob matching coverage files (Aviv Keller) #53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) #53682

PR-URL: #53826
aduh95 added a commit that referenced this pull request Jul 17, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) #53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) #53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) #52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) #52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) #53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) #53462
test_runner:
  * support glob matching coverage files (Aviv Keller) #53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) #53682

PR-URL: #53826
RafaelGSS pushed a commit that referenced this pull request Jul 17, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) #53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) #53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) #52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) #52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) #53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) #53462
test_runner:
  * support glob matching coverage files (Aviv Keller) #53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) #53682

PR-URL: #53826
ehsankhfr pushed a commit to ehsankhfr/node that referenced this pull request Jul 18, 2024
PR-URL: nodejs#53553
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
ehsankhfr pushed a commit to ehsankhfr/node that referenced this pull request Jul 18, 2024
Notable changes:

http:
  * (SEMVER-MINOR) expose websockets (Natalia Venditto) nodejs#53721
lib:
  * (SEMVER-MINOR) add `node:sqlite` module (Colin Ihrig) nodejs#53752
module:
  * add `__esModule` to `require()`'d ESM (Joyee Cheung) nodejs#52166
path:
  * (SEMVER-MINOR) add `matchesGlob` method (Aviv Keller) nodejs#52881
process:
  * (SEMVER-MINOR) port on-exit-leak-free to core (Vinicius Lourenço) nodejs#53239
stream:
  * (SEMVER-MINOR) pipeline wait for close before calling the callback (jakecastelli) nodejs#53462
test_runner:
  * support glob matching coverage files (Aviv Keller) nodejs#53553
worker:
  * (SEMVER-MINOR) add `postMessageToThread` (Paolo Insogna) nodejs#53682

PR-URL: nodejs#53826
@targos targos added the backported-to-v22.x PRs backported to the v22.x-staging branch. label Jul 28, 2024
@targos
Copy link
Member

targos commented Jul 28, 2024

It looks like the commit message was edited when it landed on v22.x: 3a0fcbb

I'm marking this as backported so it doesn't show up in branch-diff's output.

@targos targos added the dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. label Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backported-to-v22.x PRs backported to the v22.x-staging branch. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exclude test files from coverage report add coverage-path and coverage-ignore-path in test_runner coverage.