Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix JWT pattern #179

Closed
wants to merge 8 commits into from
Closed

Fix JWT pattern #179

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
84be086
Sample added
babenek Aug 8, 2022
14a88ab
Regex fix. ml_threshold n tests set to 0.5 due ML decision = 0.546
babenek Aug 8, 2022
3af8731
fix
babenek Aug 8, 2022
eb58ad7
Fix example
babenek Aug 9, 2022
2169ebc
Update tests/samples/false_positive
babenek Aug 9, 2022
b33e409
Merge branch 'main' into bugfix
babenek Aug 9, 2022
f1836cf
Update credsweeper/rules/config.yaml
babenek Aug 9, 2022
aad4cc0
Merge branch 'main' into bugfix
babenek Aug 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: Prepare commit status
run: |
ITEMS_CNT=$(jq '. | length' output.json)
if [ 18 -eq ${ITEMS_CNT} ]; then
if [ 20 -eq ${ITEMS_CNT} ]; then
echo "Samples were successfully scanned"
else
echo "Unexpected items: ${ITEMS_CNT}"
Expand Down
2 changes: 1 addition & 1 deletion credsweeper/rules/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@
severity: medium
type: pattern
values:
- (?P<value>eyJ[A-Za-z0-9-_=]+(\.[A-Za-z0-9-_.+\/=]+)?)
- (?<![0-9a-zA-Z_-])(?P<value>eyJ[A-Za-z0-9-_=]+\.eyJ[A-Za-z0-9-_=]+(\.[A-Za-z0-9-_=]+)?)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JWT with a single eyJ entry might still be a valid credential

#104

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?<![0-9a-zA-Z_-]) probably should not be in JWT regular expression itself
We can concatenate it to ALL pattern based expressions, so it would fix all other short expressions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'''(?<![0-9a-zA-Z_-])''' removes case when eyJ is only a substring in long base64 case. even (?<![0-9a-zA-Z]) might reduce suspicious strings.

babenek marked this conversation as resolved.
Show resolved Hide resolved
filter_type: GeneralPattern
use_ml: true
validations: []
Expand Down
8 changes: 4 additions & 4 deletions tests/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# total number of files in test samples, included .gitignore
SAMPLES_FILES_COUNT: int = 45
SAMPLES_FILES_COUNT: int = 47

# credentials count after scan
SAMPLES_CRED_COUNT: int = 50
SAMPLES_CRED_LINE_COUNT: int = 53
SAMPLES_CRED_COUNT: int = 51
SAMPLES_CRED_LINE_COUNT: int = 54

# credentials count after post-processing
SAMPLES_POST_CRED_COUNT: int = 18
SAMPLES_POST_CRED_COUNT: int = 19

# well known string with all latin letters
AZ_DATA = b"The quick brown fox jumps over the lazy dog"
Expand Down
4 changes: 4 additions & 0 deletions tests/samples/example_jwt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@

eyJhbGciOiJSUzI1NiIsIng1dCI6IjdkRC1nZWNOZ1gxWmY3R0xrT3ZwT0IyZGNWQSIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJodHRwczovL2NvbnRvc28uY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvZTQ4MTc0N2YtNWRhNy00NTM4LWNiYmUtNjdlNTdmN2QyMTRlLyIsIm5iZiI6MTM5MTIxMDg\
1MCwiZXhwIjoxMzkxMjE0NDUwLCJzdWIiOiIyMTc0OWRhYWUyYTkxMTM3YzI1OTE5MTYyMmZhMSJ9.C4Ny4LeVjEEEybcA1SVaFYFS6nH-Ezae_RrTXUYInjXGt-vBOkAa2ryb-kpOlzU_R4Ydce9tKDNp1qZTomXgHjl-cKybAz0Ut90-dlWgXGvJYFkWRXJ4J0JyS893EDwTEHYaAZH_lCBvoYPhXexD2yt1b-73xS\
P6oxVlc_sMvz3DY__1Y_OyvbYrThHnHglxvjh88x_lX7RN-Bq82ztumxy97rTWaa_1WJgYuy7h7okD24FtsD9PPLYAply0ygl31ReI0FZOdX12Hl4THJm4uI_4_bPXL6YR2oZhYWp-4POWIPHzG9c_GL8asBjoDY9F5q1ykQiotUBESoMML7_N1g
6 changes: 6 additions & 0 deletions tests/samples/false_positive
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# https://github.com/tosca/web/blob/f2d42d2b3311994edd68133547fcde8973392464/packages/EntityFramework.5.0.0/tools/EntityFramework.PS3.psd1#L172
# http://calebb.net/ says: JWT is required to have three segments
babenek marked this conversation as resolved.
Show resolved Hide resolved
# https://www.rfc-editor.org/rfc/rfc7519.html
# UK3O3RhOJA/u0afRTK10MCAR6wfVVJUVSZQbQpKumFwwJtoAa+h7veyJBw/3DgSY
# -eyJhbGciOiJSUzI1NiIsIng1dCI6IjdkRC1nZWNOZ1gxWmY3R0xrT3ZwT0IyZGNWQSIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJodHRwczovL2NvbnRvc28uY29tIiwiaXNzIjo