-
Notifications
You must be signed in to change notification settings - Fork 422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add bandit
to CI for security linting
#1775
Add bandit
to CI for security linting
#1775
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally, looks good, one question here: you made all those exclusions here and there to suppress known security vulnerabilities? Why I'm asking: some of them are not hard to fix, like switching from md5 to else and so on.
@avishniakov good question. The problem with changing the hashing algorithm relates to backwards compatibility. i.e. anyone upgrading to a newer version would have the hashes stored in the database using the old algo but this update would mean we essentially wipe / invalidate their cache (in a way that perhaps does unexpected things). To do it in a graceful way would be fairly involved, I felt, where we'd have to maybe store the hashing algorithm in the database along with the hash itself. Since none of the places where we're currently using old hashing algorithms involve security, I felt like it was an ok tradeoff. It will prevent us from adding any new places where we're using broken hashing algorithms and we can potentially consider switching out the places where it's ignored going forward as part of a bigger ticket. |
You're right though that the fix is basically replacing md5 with sha256, but that would break things in unpredictable ways, I thought. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the reason for the various different timeout adjustments (5/7/31/#nosec) for the different requests?
@schustmi see the PR description. I didn't want to make it generically long, but that said I can also see that it might be confusing. Not sure what the right way to handle this is. Raw |
* Add `bandit` to CI for security linting (#1775) * ignore bandit false positives * add timeout for requests call * ignore bandit false positive * ignore bandit false positives * ignore bandit false positives * update as per bandit suggestions * add bandit as a dependency * add security checks to lint-unit-test workflow * add safety to dev dependencies + update version * loosen safety dependency pin * remove safety dependency * Add `mlstacks` compatibility check to CI (#1767) * add compatibility check to CI * add compatibility check to earlier in CI * remove dependency on integration tests from CI * add compatibility check to earlier in CI * remove compatibility check from integration tests --------- Co-authored-by: Safoine El Khabich <34200873+safoinme@users.noreply.github.com> * extend `StepContext` visibility to materializers (#1769) * extend StepContext visibility to materializers * skip output processing on failed step * PR comments * Revert GH changes to fix colima bug in macos gh (#1779) * revert the changes done on gh in macos due to failing bug in colima * revert the changes done on gh in macos due to failing bug in colima * reinstall qemu * Reduce CI runner count (#1777) * combine setup with unit tests * combine integration with template tests * further reduce runner count * remove dynamic uses --------- Co-authored-by: Safoine El Khabich <34200873+safoinme@users.noreply.github.com> * Add auto-population of E2E example from template (#1766) * add checked-out template * add setup to action * auto-update e2e * add OPT_IN_OUT_EMAIL * pass email * let e2e auto populate * debug * fix mkdir * debug * debug * update commit command * use ref_name * update ref * add ref to checkout * Auto-update of template * update template * Auto-update of template * restore full CI * fix branch changes check * debug * restore full CI --------- Co-authored-by: GitHub Actions <actions@github.com> * fix step name in ci --------- Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com> Co-authored-by: Safoine El Khabich <34200873+safoinme@users.noreply.github.com> Co-authored-by: GitHub Actions <actions@github.com>
Added bandit to the CI to detect medium and high severity warnings about code we write.
(Made some fixes to handle the errors it found -- some were ignored (like hashing algorithms where security was not the intended purpose) and other small fixes were just made.)
For the
timeout
changes, I read that it's best to choose something that is 1 larger than a multiple of 3, and then beyond that I just chose values that I thought made sense for that particular piece of code.Pre-requisites
Please ensure you have done the following:
Types of changes