Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improved adhoc string scanning #386

Merged
merged 6 commits into from
Jan 13, 2021

Conversation

domanchi
Copy link
Contributor

Summary

Adhoc string scanning is hella useful. e.g.

$ detect-secrets scan --string 'butactuallynotasecret'
...
Base64HighEntropyString: False

However, sometimes (especially for the HighEntropyString plugins), we want to see why it failed. This is existing behavior in 0.14.3; this PR carries this functionality forward. With this change, it will be:

$ detect-secrets scan --string 'butactuallynotasecret'
...
Base64HighEntropyString: False (3.404)

Reviewer Notes

It was somewhat difficult to surface the non-secret to the scan layer in a generic manner (i.e. without using if plugin in {Base64HighEntropyString}), so I had to be creative with the analyze_line and analyze_string functionality. As a result, for people using the analyze_string function directly (like the test case), they would need to be responsible for filtering out the secret themselves.

Not ideal, but it's the best way I could find.

This PR also introduces another gotcha -- for our DI system, we now leverage the pythonic fashion of duck-typing to determine whether there are enough required parameters to inject into the function. This needed to be done to support cases where the underlying function had default values for their parameters (e.g. HighEntropyStringsPlugin.analyze_line), and as such, we don't need to supply all the parameters that a function needs. However, this also means that if the underlying function raises an uncaught TypeError, it will be silenced by the DI system.

I guess future improvements could be done to distinguish default parameters from non-default parameters in a function declaration, but I didn't want to dive too deep into Python magic internals for this PR.

Copy link
Member

@calvinli calvinli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine overall, just have a few comments

detect_secrets/core/scan.py Outdated Show resolved Hide resolved
detect_secrets/plugins/high_entropy_strings.py Outdated Show resolved Hide resolved
detect_secrets/core/scan.py Outdated Show resolved Hide resolved
detect_secrets/core/scan.py Outdated Show resolved Hide resolved
detect_secrets/plugins/base.py Show resolved Hide resolved
detect_secrets/util/inject.py Outdated Show resolved Hide resolved
Copy link
Member

@calvinli calvinli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@domanchi domanchi merged commit 481137a into pre-v1-launch Jan 13, 2021
@domanchi domanchi deleted the pre-v1-better-adhoc-string-scanning branch January 13, 2021 23:03
jimmyhlee94 pushed a commit to jimmyhlee94/detect-secrets that referenced this pull request Aug 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants