(Y)et (A)nother (R)obber: Sail ye seas of git for booty is to be found

Sail ho! Yar is a tool for plunderin' organizations, users and/or repositories...

In all seriousness though, yar is an OSINT tool for reconnaissance of repositories/users/organizations on Github. Yar clones repositories of users/organizations given to it and goes through the whole commit history in order of commit time, in search for secrets/tokens/passwords, essentially anything that shouldn't be there. Whenever yar finds a secret, it will print it out for you to further assess.

Yar searches for secrets either by regex, entropy or both, the choice is yours! Inspired by other git secret grabbers.

Installation

Make sure you have the GOPATH environment variable set in your preferred shell rc and that the $GOPATH/bin directory is in your PATH. More info here.
You can install this by running go get github.com/nielsing/yar
Or you can download the latest release of Yar for your operating system here. Just make sure you have the yarconfig.json file as well in $GOPATH/src/github.com/nielsing/yar/config.

Usage

Want to search for secrets within an organization?

yar -o orgname

You can also include the members of the organization with:

yar -o orgname --include-members

Want to search for secrets within a users repositories?

yar -u username

Want to search for secrets within a single repository?

yar -r https://github.com/User/Repo

or if you have already cloned the repository

yar -r /path/to/.git/folder

Want to search for secrets within an organization, a user and a repository?

yar -o orgname -u username -r https://github.com/User/Repo

Getting too much/not enough noise?

All rules are marked with a noise level from 0 to 9. Noise levels from 0 to 4 are considered secrets while noise levels from 5 to 9 are considered reconnaissance info (emails, IPs, etc...). You can decide which noise levels yar searches for, the default is to (and including 3).

Search for all secrets with noise level 4 or less

yar -r https://github.com/User/Repo -n -4

Search for all secrets with noise level 6 or more

yar -r https://github.com/User/Repo -n 6-

Search for all secrets from 1 to (and including) 3.

yar -r https://github.com/User/Repo -n 1-3

Search for all secrets with noise level exactly 7

yar -r https://github.com/User/Repo -n 7

Search for all secrets with any noise level

yar -r https://github.com/User/Repo -n -

Have your own predefined rules?

Rules are stored in a JSON file with the following format:

{
    "Rules": [
        {
            "Reason": "The reason for the match",
            "Rule": "The regex rule",
            "Noise": 3
        },
        {
            "Reason": "Super secret token",
            "Rule": "^Token: .*$",
            "Noise": 2
        }
    ]
    "FileBlacklist": [
        "Regex rule here"
        "^.*\\.lock"
    ]
}

You can then load your own rule set with the following command:

yar -u username --rules PATH_TO_JSON_FILE

If you already have a truffleHog config and want to port it over to a yar config there is a script in the config folder that does it for you. Simply run python3 trufflestoconfig.py PATH_TO_TRUFFLEHOG_CONFIG and the script will give you a file named yarconfig.json.

Don't like regex?

yar -u username --entropy

Want the best of both worlds?

yar -u username --both

Want to search as an authenticated user?

Add your github token to your environment variables.

export YAR_GITHUB_TOKEN=YOUR_TOKEN_HERE

Want to save your findings to a JSON file for later analysis?

yar -o orgname --save

Don't like the default colors and want to add your own color settings?

It is possible to customize the colors of the output for Yar through environment variables. The possible colors to choose from are the following:

black
blue
cyan
green
magenta
red
white
yellow
hiBlack
hiBlue
hiCyan
hiGreen
hiMagenta
hiRed
hiWhite
hiYellow

Each color can then be suffixed with bold, i.e. blue bold to make the letters bold.

This is done through the following env variables:

YAR_COLOR_VERBOSE -> Color of verbose lines.
YAR_COLOR_SECRET  -> Color of the highlighted secret.
YAR_COLOR_INFO    -> Color of info, that is, simple strings that tell you something.
YAR_COLOR_DATA    -> Color of data, i.e. commit message, reason, etc.
YAR_COLOR_SUCC    -> Color of succesful messages.
YAR_COLOR_WARN    -> Color of warnings.
YAR_COLOR_FAIL    -> Color of fatal warnings.

Like so export YAR_COLOR_SECRET="hiRed bold".

Extra Knowledge

There are some design decisions which might be good to know about. Yar saves all cloned github repos in a folder named yar within the temp directory. Yar then tries to load github repos from this cache by default, if you don't want to load from cache then you can add the --no-cache flag.

Yar also clones bare repos by default, if you want to get all files within a repo and not just the metadata then you can add the --no-bare flag.

If you want to remove repos from cache then you can use the --cleanup flag. This flag either removes the whole cache if no folder was specified or just removes the specified folder. The folder structure within the cache folder is like so:

/yar
|--- /User1
|  |--- /Repo1
|  |--- /Repo2
|
|--- /User2
|  |--- /Repo1
|  |--- /Repo2

So you can run --cleanup User1 to remove the cache of User1 or --cleanup User1/Repo1 to clean up Repo1 of User1. You can think of the flag as a wrapper around rm -r /tmp/yar/{USER_INPUT}.

Finally yar goes 10000 commits deep by default and goes through them in order of time (oldest to newest). This depth is configurable so if you ever want to cover more or fewer commits simply add the --depth flag with the depth you want.

Help

usage: yar [-h|--help] [-o|--org "<value>"] [-u|--user "<value>"] [-r|--repo
           "<value>"] [-c|--context <integer>] [-e|--entropy] [-b|--both]
           [-f|--forks] [-n|--noise "<value>"] [-d|--depth <integer>]
           [-C|--config <file>] [--no-bare] [--no-cache] [--no-context]
           [--include-members] [--skip-duplicates] [--cleanup "<value>"]
           [-s|--save "<value>"]

           Sail ye seas of git for booty is to be found

Arguments:

  -h  --help             Print help information
  -o  --org              Organization to plunder
  -u  --user             User to plunder
  -r  --repo             Repository to plunder
  -c  --context          Show N number of lines for context. Default: 2
  -e  --entropy          Search for secrets using entropy analysis. Default:
                         false
  -b  --both             Search by using both regex and entropy analysis.
                         Overrides entropy flag. Default: false
  -f  --forks            Specifies whether forked repos are included or not.
                         Default: false
  -n  --noise            Specify the range of the noise for rules. Can be
                         specified as up to (and including) a certain value
                         (-4), from a certain value (5-), between two values
                         (3-5), just a single value (4) or the whole range (-).
                         Default: -3
  -d  --depth            Specify the depth limit of commits fetched when
                         cloning. Default: 10000
  -C  --config           JSON file containing yar config.
      --no-bare          Clone the whole repository. Default: false
      --no-cache         Don't load from cache. Default: false
      --no-context       Only show the secret itself, similar to trufflehog's
                         regex output. Overrides context flag. Default: false
      --include-members  Include an organization's members for plunderin'.
                         Default: false
      --skip-duplicates  Skip duplicate secrets within repositories. Default:
                         false
      --cleanup          Remove specified cloned directory within yar cache
                         folder. Leave blank to remove the cache folder
                         completely.
  -s  --save             Yar will save all findings to a specified file.
                         Default: findings.json

Acknowledgements

It is important to point out that this idea is inspired by the infamous truffleHog tool and the code used for entropy searching is in fact borrowed from the truffleHog repository which in turn is borrowed from this blog post.

This project wouldn't have been possible without the following libraries:

License

AriESQ/yar

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages