-
Notifications
You must be signed in to change notification settings - Fork 481
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Show negative results in details #95
Comments
Love the idea, can you submit a PR for all checks! |
Negative results logged with a "!!" prefix. Updates ossf#95 $ go run . --repo=github.com/ossf/scorecard --show-details --checks=Pull-Requests Starting [Pull-Requests] Finished [Pull-Requests] RESULTS ------- Pull-Requests: Pass 9 !! found commit without PR: 71dace5, committer: dlorenc found PRs for 29 out of 30 commits
Negative results logged with a "!!" prefix. Updates ossf#95 $ go run . --repo=github.com/cilium/cilium --show-details --checks=Signed-Tags Starting [Signed-Tags] Finished [Signed-Tags] RESULTS ------- Signed-Tags: Fail 4 verified tag found: v1.9.0-rc1, commit: a46b5c308779b00676bcbffe6847701984fb7ec7 !! unverified tag found: v1.9.0-rc2, commit: 2ee8e4659ad4050154eb83008ba6434bddad44eb, reason: unsigned verified tag found: v1.9.0-rc3, commit: ee77e846a9b85e318d6d077c801e2615d5e7dbe3 !! unverified tag found: v1.9.0, commit: 1cdd547dce26adb046d117494d559c64007365fd, reason: unsigned verified tag found: v1.9.1, commit: bb4abe1720cb56c6a5f74d0567665555ad8434f1 found 3 of 5 verified tags
Negative results logged with a "!!" prefix. Updates ossf#95 $ go run . --repo=github.com/gohugoio/hugo --show-details --checks=Signed-Releases Starting [Signed-Releases] Finished [Signed-Releases] RESULTS ------- Signed-Releases: Fail 10 release found: v0.79.1 !! release v0.79.1 has no signed artifacts release found: v0.79.0 !! release v0.79.0 has no signed artifacts release found: v0.78.2 !! release v0.78.2 has no signed artifacts release found: v0.78.1 !! release v0.78.1 has no signed artifacts release found: v0.78.0 !! release v0.78.0 has no signed artifacts release found: v0.77.0 !! release v0.77.0 has no signed artifacts found signed artifacts for 0 of 6 releases
Add negative check results to the CI-Tests output. Assuming that a repo will only support one CI system, GithubStatuses and GithubCheckRuns are merged into a single CITests function. Since both GithubStatuses and GithubCheckRuns were essentially validating the same PRs, it makes more sense to keep all of that state together in a single check. Additionaly, a single check can reduce the number of API queries once we detect the CI system in use. Fixes ossf#96 Updates ossf#95
@inferno-chromium, |
Can you try running the current version of scorecard? We've tried to generally address this. |
Is this something that still needs to be discussed considering the user submitting the issue did not provide feedback on the updated version? |
This issue is already completed. All score deductions will have a WARN associated them in the detail logger. |
Howdy! New here. 👋 🤠
While using scorecard to evaluate projects I contribute to, I found myself having to read (and patch) the checks code to understand why I was seeing failing or marginal results. It would be very helpful to see negative results in the
--show-details
output instead of only positive results. It makes it easier for newcomers like me to learn what is considered bad practice, and I assume it would be help everyone to investigate any negative findings.For example, I've patched scorecard locally to give negative results (starting with
!!
) for some checks:I can submit a PR for the above if this idea is acceptable.
The text was updated successfully, but these errors were encountered: