Add filtering of english words from entropy (and keyword) plugins #241
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In get_raw_secret_value()
`os.path.splitext(filename)[1]` includes the '.'
The current heuristic will never return True
Start capitalized and after 2 spaces when on the same line as code
- Add `pyahocorasick` as an optional dependency See issue #240 for more information.
|
Forgot to make it case-insensitive, will do tomorrow. |
By .lower()ing when creating and retrieving
|
I used Something random that made me think it was missing things was that |
- 🐍 Refactor hadouken code - 🐍 Remove high_entropy_strings.py from the uncovered files list in tox
KevinHock
force-pushed
the
add_filter_for_words
branch
from
cee53a8
to
de7fbd2
Compare
- Change `is_secret` string names to `(true|false)-positives` (Negative meaning false-positive was confusing.)
- Replace list of secrets with {filename: {plaintext: line:}}
- Replace top-level `results` key with `plugins` since we have a results key already
And normalize main_test.py comments
KevinHock
force-pushed
the
add_filter_for_words
branch
from
739f3bf
to
20d1921
Compare
KevinHock
force-pushed
the
add_filter_for_words
branch
from
098e956
to
1bbdcce
Compare
KevinHock
force-pushed
the
add_filter_for_words
branch
from
1bbdcce
to
b529d8e
Compare
OiCMudkips
reviewed
Sep 24, 2019
| @@ -65,7 +79,7 @@ def initialize( | |||
| files_to_scan, | |||
| ) | |||
|
|
|||
| for file in files_to_scan: | |||
| for file in sorted(files_to_scan): | |||
There was a problem hiding this comment.
No real reason, just looked nicer when I was running it on things.
| @@ -570,15 +570,18 @@ def test_determine_audit_results_plugin_config( | |||
|
|
|||
| results = audit.determine_audit_results(baseline, '.secrets.baseline') | |||
|
|
|||
| assert results['results']['HexHighEntropyString']['config'].items() \ | |||
| >= plugins_used[0].items() | |||
| assert ( | |||
| coverage report --show-missing --include=tests/* --fail-under 100 | ||
| coverage report --show-missing --include=detect_secrets/* --fail-under 98 | ||
| # This is so that we do not regress unintentionally |
When following a symlink, we just subtracted the `cwd` from the path. This caused us to scan non-existant files.
KevinHock
force-pushed
the
add_filter_for_words
branch
from
b529d8e
to
b440623
Compare
OiCMudkips
approved these changes
Sep 24, 2019
killuazhu
pushed a commit
to IBM/detect-secrets
that referenced
this pull request
May 28, 2020
* use more python env * keep pypy out
killuazhu
pushed a commit
to IBM/detect-secrets
that referenced
this pull request
Jul 9, 2020
* use more python env * keep pypy out
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a solution for #240.
I will add that I am personally not against hardcoding a list of ~2,298 English words in the future, like from the
How bad can it gitpaper, as that can be immediately beneficial to most users with no effort, but for now this is okay.I tried to keep the commit history pretty clean 😁