✨ Feature [experimental]: The Scorecard Dependencydiff CLI (Version 0 Part 1)

[aidenwang9867]

What kind of change does this PR introduce?

This PR introduces a v0 experimental feature - the Dependencydiff CLI powered by the Scorecard API GetDependencydiffResults (PR #2046).

Update: This PR (v0-p1) only introduces the option changes, cmd changes are made in PR #

Minor fixes: some potential misleading var names in the GetDependencydiffResults API, such as changing baseSHA/headSHA to base/head, since we can use branch names as the inputs.

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

In PR #2046, the v0 of the GetDependencydiffResults function API was introduced to the Scorecard repo, but Scorecard hasn't gotten use of it.

What is the new behavior (if this is a feature change)?**

Now with the SCORECARD_EXPERIMENTAL env var set in system or CLI, users can use the dependency-diff sub-command to specify two code commits either by commitSHA or branch name, to surface Scorecard check result of those dependencies to better understand the security posture of their repo. This works as a separate sub-command so it won't affect the original root usage mode of Scorecard (running checks on a repo).

A possible CLI usage might be:

SCORECARD_EXPERIMENTAL=1 \
./scorecard dependency-diff \
--repo=aidenwang9867/go_manifest_test \
--base main \
--head 112997623f24d5f677ff9155f3868964d4b81df3 \
--checks Branch-Protection,Fuzzing,License

where:

• SCORECARD_EXPERIMENTAL=1 enables the dependency-diff sub-command feature
• dependencydiff is the sub-command and shifts the Scorecard from the original check mode to dependencydiff
• --repo specifies the repo to be checked (same usage as the original Scorecard CLI)
• --base specifies main as the BASE, and --head specifies main (the latest commit to the 112997623f24d5f677ff9155f3868964d4b81df3 branch) as the HEAD.
• --checks specifies the checks to run on the dependencies (not the repo itself) (same usage format as the original Scorecard CLI)

Which issue(s) this PR fixes

Issue #2008

Special notes for your reviewer

Note the v0 of the CLI runs pretty slow since we are running scorecard checks on every dependency with a valid srcRepo URI. I am considering adding Parallel() support for the scorecard running on dependencies, or an option (YES/NO) to ask if the user would like to continue when there are more than, for example, 20+ dependencies to check and tell the user it might be slow.
@naveensrinivasan @azeemshaikh38 @laurentsimon wdut?

Does this PR introduce a user-facing change?

Yes.
For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

Now with the SCORECARD_EXPERIMENTAL env var set in system or CLI, users can use the dependency-diff sub-command to specify two code commits either by commitSHAs or branch names, to surface Scorecard check result of those dependencies to better understand the security posture of their repo.

[codecov]

Codecov Report

Merging #2077 (5c9fed1) into main (6fc08e7) will increase coverage by 2.27%.
The diff coverage is 0.00%.

❗ Current head 5c9fed1 differs from pull request most recent head cb142ce. Consider uploading reports for the commit cb142ce to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2077      +/-   ##
==========================================
+ Coverage   41.87%   44.14%   +2.27%     
==========================================
  Files          95       97       +2     
  Lines        7945     8014      +69     
==========================================
+ Hits         3327     3538     +211     
+ Misses       4358     4207     -151     
- Partials      260      269       +9     

[github-actions]

Integration tests success for
[aee106e]
(https://github.com/ossf/scorecard/actions/runs/2699794335)

[laurentsimon]

LGTM overall!

Added a few comment

[github-actions]

Integration tests success for
[9d3668a]
(https://github.com/ossf/scorecard/actions/runs/2702451278)

[github-actions]

Integration tests success for
[403b17c]
(https://github.com/ossf/scorecard/actions/runs/2702547842)

[github-actions]

Integration tests success for
[4fe1faa]
(https://github.com/ossf/scorecard/actions/runs/2707990009)

[github-actions]

Integration tests success for
[6739263]
(https://github.com/ossf/scorecard/actions/runs/2792775649)

[github-actions]

Integration tests success for
[72cc949]
(https://github.com/ossf/scorecard/actions/runs/2792776037)

[github-actions]

Integration tests success for
[2a7e09e]
(https://github.com/ossf/scorecard/actions/runs/2792997558)

[github-actions]

Integration tests success for
[c3f2503]
(https://github.com/ossf/scorecard/actions/runs/2835496211)

[github-actions]

Integration tests success for
[c9d82c0]
(https://github.com/ossf/scorecard/actions/runs/2870265267)

[github-actions]

Stale pull request message

[naveensrinivasan]

@laurentsimon @azeemshaikh38 What is pending in this PR?

[github-actions]

Integration tests success for
[5c9fed1]
(https://github.com/ossf/scorecard/actions/runs/2941003053)

[github-actions]

Stale pull request message

[naveensrinivasan]

@aidenwang9867 Are you still planning to work on this?

[naveensrinivasan]

Closing this PR as it has been stale for a while. Please reopen.