-
Notifications
You must be signed in to change notification settings - Fork 28.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: add coverage GitHub action #35653
Conversation
It's almost working, but the below three errors on CI seem to be a cause of the workflow failure: file: internal/error-serdes.js error: Error: ENOENT: no such file or directory, open '/home/runner/work/node/node/lib/internal/error-serdes.js'
file: internal/process/main_thread_only.js error: Error: ENOENT: no such file or directory, open '/home/runner/work/node/node/lib/internal/process/main_thread_only.js'
file: internal/process/stdio.js error: Error: ENOENT: no such file or directory, open '/home/runner/work/node/node/lib/internal/process/stdio.js' |
Codecov Report
@@ Coverage Diff @@
## master #35653 +/- ##
==========================================
- Coverage 96.87% 96.40% -0.48%
==========================================
Files 212 220 +8
Lines 69641 73672 +4031
==========================================
+ Hits 67466 71024 +3558
- Misses 2175 2648 +473
Continue to review full report at Codecov.
|
I don't know if it's an issue. Codecov found many extra "reports":
|
Non-blocking question: IIRC, the current nightly job combines Windows and POSIX coverage at codecov. Is the long-term plan to try to get that happening with this too? If so, does it make sense to add a "TODO" or something? |
@thomasrockhu before we land this, I wanted to see if you have any feedback. My main questions being:
|
Hi @bcoe
|
# TODO(bcoe): fix the couple tests that fail with the inspector enabled. | ||
# The cause is most likely coverage's use of the inspector. | ||
- name: Test | ||
run: NODE_V8_COVERAGE=coverage/tmp make run-ci -j2 V=1 TEST_CI_ARGS="-p dots" || exit 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a little flummoxed as to why having NODE_V8_COVERAGE
inline works great, whereas having it in the global env:
stanza resulted in bad counts ... but there you have it, it's worked twice in a row with the environment variable here.
"text", | ||
"lcov" | ||
], | ||
"lines": 95, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can ratchet this number up as we improve the linux coverage #s, and it will result in a failure to the action if it drops below the threshold.
@thomasrockhu thank you 👏 , Sounds like we'll just turn this setting on if the delayed merging becomes an annoyance. I'm really excited to get coverage as part of our immediate feedback loop, and appreciate the help you've given.
@targos I've fixed it so we should only be uploading one report from the |
@watilde I believe those errors you mentioned are false It would be good to dig into these warnings, and into the three tests that fail under coverage. If we land this, and make coverage one of the signals on PRs, perhaps it will help motivate us to dig into these problems. |
This comment has been minimized.
This comment has been minimized.
PR-URL: nodejs#35653 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
48de3fc
to
bb2945e
Compare
Landed in bb2945e |
Collect Windows and C++ coverage. Configure codecov so that comments are more concise and are only left when coverage varies. PR-URL: #35670 Fixes: #35696 Refs: #35653 Refs: #35646 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>
Collect Windows and C++ coverage. Configure codecov so that comments are more concise and are only left when coverage varies. PR-URL: #35670 Fixes: #35696 Refs: #35653 Refs: #35646 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>
PR-URL: #35653 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Collect Windows and C++ coverage. Configure codecov so that comments are more concise and are only left when coverage varies. PR-URL: #35670 Fixes: #35696 Refs: #35653 Refs: #35646 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>
Collect Windows and C++ coverage. Configure codecov so that comments are more concise and are only left when coverage varies. PR-URL: #35670 Fixes: #35696 Refs: #35653 Refs: #35646 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>
@bcoe is this PR expected to land on the v14.x release line? I cherry-picked it into the v14.15.2-proposal, but the action is failing on the release proposal PR (refs: #36476 (comment)). Just trying to figure out if there's real failure we need to address, or the PR should not land on v14.x. |
@BethGriggs I think it's okay to keep this action on the main branch, and not back-port. My hope is that it gives us insights into our test coverage going forward. |
This is a first attempt at adding a GitHub Action that pushes coverage results to codecov.
Fixes #35646
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passesCC: @nodejs/testing