Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve AWS access key detection #796

Merged
merged 2 commits into from
Apr 12, 2024

Conversation

mikedidomizio
Copy link
Contributor

  • Please check if the PR fulfills these requirements
  • Tests for the changes have been added
  • Docs have been added / updated
  • All CI checks are green
  • What kind of change does this PR introduce?

Updates the pattern for matching different types of AWS access keys

  • What is the current behavior?

It currently only flags access keys that are prefixed with AKIA. AWS access keys can come in a few different formats, and without this change they would go undetected.

Here is a list from AWS of different prefixes and what they are.

For instance ASIA is for temporary service token.

This:

export const KEY="ASIAZZZZZZZZZZZZZZZZ"

would be flagged by

git-secrets
git-secrets.js:1:export const KEY="ASIAZZZZZZZZZZZZZZZZ"

[ERROR] Matched one or more prohibited patterns

Possible mitigations:
- Mark false positives as allowed using: git config --add secrets.allowed ...
- Mark false positives as allowed by adding regular expressions to .gitallowed at repository's root directory
- List your configured patterns: git config --get-all secrets.patterns
- List your configured allowed patterns: git config --get-all secrets.allowed
- List your configured allowed patterns in .gitallowed at repository's root directory
- Use --no-verify if this is a one-time false positive

and by

gitleaks
  ○
  │╲
  │ ○
  ○ ░
  ░    gitleaks

Finding:     export const KEY="ASIAZZZZZZZZZZZZZZZZ
Secret:      ASIAZZZZZZZZZZZZZZZZ
RuleID:      aws-access-token
Entropy:     3.621928
File:        git-secrets.js
Line:        1
Fingerprint: git-secrets.js:aws-access-token:1

but not

detect-secrets
{
  "version": "1.4.0",
  "plugins_used": [
    {
      "name": "ArtifactoryDetector"
    },
    {
      "name": "AWSKeyDetector"
    },
    {
      "name": "AzureStorageKeyDetector"
    },
    {
      "name": "Base64HighEntropyString",
      "limit": 4.5
    },
    {
      "name": "BasicAuthDetector"
    },
    {
      "name": "CloudantDetector"
    },
    {
      "name": "DiscordBotTokenDetector"
    },
    {
      "name": "GitHubTokenDetector"
    },
    {
      "name": "HexHighEntropyString",
      "limit": 3.0
    },
    {
      "name": "IbmCloudIamDetector"
    },
    {
      "name": "IbmCosHmacDetector"
    },
    {
      "name": "JwtTokenDetector"
    },
    {
      "name": "KeywordDetector",
      "keyword_exclude": ""
    },
    {
      "name": "MailchimpDetector"
    },
    {
      "name": "NpmDetector"
    },
    {
      "name": "PrivateKeyDetector"
    },
    {
      "name": "SendGridDetector"
    },
    {
      "name": "SlackDetector"
    },
    {
      "name": "SoftlayerDetector"
    },
    {
      "name": "SquareOAuthDetector"
    },
    {
      "name": "StripeDetector"
    },
    {
      "name": "TwilioKeyDetector"
    }
  ],
  "filters_used": [
    {
      "path": "detect_secrets.filters.allowlist.is_line_allowlisted"
    },
    {
      "path": "detect_secrets.filters.common.is_ignored_due_to_verification_policies",
      "min_level": 2
    },
    {
      "path": "detect_secrets.filters.gibberish.should_exclude_secret",
      "limit": 3.7
    },
    {
      "path": "detect_secrets.filters.heuristic.is_indirect_reference"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_likely_id_string"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_lock_file"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_not_alphanumeric_string"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_potential_uuid"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_prefixed_with_dollar_sign"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_sequential_string"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_swagger_file"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_templated_secret"
    }
  ],
  "results": {},
  "generated_at": "2024-03-02T16:23:36Z"
}
  • What is the new behavior (if this is a feature change)?

It will flag other related AWS access keys one would probably want to be notified about.

  • Does this PR introduce a breaking change?

No

  • Other information:

This is the same pattern used in gitleaks (although pattern changed to be in alphabetical order), but not as detailed as git-secrets. The gitleaks pattern makes the most sense to me. It doesn't include every variation as explained here, just the ones that make sense.

@lorenzodb1
Copy link
Member

@mikedidomizio I'll go ahead and merge this. Thank you for your contribution!

@lorenzodb1 lorenzodb1 merged commit cd77447 into Yelp:master Apr 12, 2024
12 checks passed
@mikedidomizio mikedidomizio deleted the improve-aws-key-id-detection branch April 13, 2024 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants