-
Notifications
You must be signed in to change notification settings - Fork 480
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 fix Docker remediations for unpinned GHA dependencies #4131
🐛 fix Docker remediations for unpinned GHA dependencies #4131
Conversation
Previously, as both the check for unpinned dependencies in GitHub Actions and the check for unpinned Docker dependencies contribute to d.Dependencies, the loop that created remediations for Docker dependencies would also create try to create Docker remediations for the unpinned GitHub Actions dependencies. This could get really slow, especially when scanning a repo with many GitHub Actions such as https://github.com/apache/beam. Signed-off-by: Arnout Engelen <arnout@bzzt.net>
789b16e
to
fcdfdb7
Compare
fdd03b2
to
fa6b0df
Compare
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
fa6b0df
to
0a2dcf0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the context of how the code works now, I think this change seems reasonable.
How did the speed-up look on your end? Is there still need for improvement, we can try to get rid of the quadratic behavior currently exhibited as the code iterates over dependencies multiple times, which for large projects like the ASF may take a lot of time still.
These functions are all called from
scorecard/checks/raw/pinned_dependencies.go
Lines 36 to 63 in 5447253
func PinningDependencies(c *checker.CheckRequest) (checker.PinningDependenciesData, error) { | |
var results checker.PinningDependenciesData | |
// GitHub actions. | |
if err := collectGitHubActionsWorkflowPinning(c, &results); err != nil { | |
return checker.PinningDependenciesData{}, err | |
} | |
// // Docker files. | |
if err := collectDockerfilePinning(c, &results); err != nil { | |
return checker.PinningDependenciesData{}, err | |
} | |
// Docker downloads. | |
if err := collectDockerfileInsecureDownloads(c, &results); err != nil { | |
return checker.PinningDependenciesData{}, err | |
} | |
// Script downloads. | |
if err := collectShellScriptInsecureDownloads(c, &results); err != nil { | |
return checker.PinningDependenciesData{}, err | |
} | |
// Action script downloads. | |
if err := collectGitHubWorkflowScriptInsecureDownloads(c, &results); err != nil { | |
return checker.PinningDependenciesData{}, err | |
} | |
So collectGitHubActionsWorkflowPinning
identifies workflow deps W1, W2, and W3 via validateGitHubActionWorkflow
and then cycles through them.
Then collectDockerfilePinning
identifies docker images D1, D2, and D3, and via validateDockerfilesPinning
, and then cycles through W1, W2, W3, AND D1, D2, and D3.
This continues with collectDockerfileInsecureDownloads
, collectShellScriptInsecureDownloads
, and collectGitHubWorkflowScriptInsecureDownloads
.
This could be fixed by changing where we append to the the final dep slice, and iterating over temporary slices, but if its not worth the effort we don't need to do it yet.
/scdiff generate Pinned-Dependencies |
Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.
These don't iterate over the |
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
4a1c34f
to
1d01c01
Compare
Fantastic. Thanks for catching this!
I did a quick and dirty test to avoid going over the GHA deps again in |
/scdiff generate Pinned-Dependencies |
What kind of change does this PR introduce?
Bug fix
What is the current behavior?
As both the check for unpinned dependencies in GitHub Actions and the check for unpinned Docker dependencies contribute to d.Dependencies, the loop that created remediations for Docker dependencies would also create try to create Docker remediations for the unpinned GitHub Actions dependencies.
This could get really slow, especially when scanning a repo with many GitHub Actions such as https://github.com/apache/beam.
What is the new behavior (if this is a feature change)?**
Only create Docker remediations for Docker dependencies (and similar for GitHub workflow remediations)
Which issue(s) this PR fixes
NONE
Special notes for your reviewer
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the
release-note
(In particular, describe what changes users might need to make in their
application as a result of this pull request.)