gitleaks ignores default config and doesn't merge configs

[adamdecaf]

Describe the bug
If I have a config like the following zero secrets are detected. I'm unable to specify a few allow directives.

[allowlist]
paths = ["testdata/malformed_key.pem"]

gitleaks reports no secrets, but there are some. This is likely due to gitleaks thinking --config should replace the default config, but that's not what I want.

gitleaks version: 8.8.5

    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks 

9:39AM INF scan completed in 4.701718ms
9:39AM INF no leaks found
finished gitleaks check

Expected behavior
It would be nice to have a way to allow a few entries as many projects require dummy keys.

Basic Info (please complete the following information):

• OS: macOS 12.x
• Gitleaks Version: 8.8.5

cc @zricethezav

[zricethezav]

Hi @adamdecaf, this isn't a bug as --config replaces the default config by design. I am considering ways to extend the default config in the future.

[adamdecaf]

That would be great. I've got ~100 repositories to manage and while we do template codebases I was hoping to rely on the default config. Having overrides would be a big help to us.

[electriquo]

relates to #741, #596

[Dingjie-Daniel-Yang]

@zricethezav, I am echoing with @adamdecaf. It will be great if there is a separate config to allow users to customize the global allowlists without overwriting the default config.

[zricethezav]

Thanks, I hear you guys. I'm thinking of ways to introduce this. In the meantime if you want this feature expedited consider reaching out to me for a maintenance and support agreement https://gitleaks.io/products.html

Some design restrictions I want to impose:

• no cli arguments, everything should be defined within the config
• resource authentication should be kept to a minimum if none at all. I,e. if the base config is pointing to a url, that url should be publicly available without authentication or maybe a bearer token... still undecided on whether or not I want to support authentication
• no merging, only extending the config

So a config might look something like

title = "repo foo gitleaks config"

[extends]
  url = "https://github.com/zricethezav/gitleaks/config/baseconfig.toml"
  # path = "path/to/base/config.toml" 
  # useDefault = true

Only one url, path, or useDefault should be specified. useDefault would extend the repo config with the default gitleaks config

cc @adamdecaf

[adamdecaf]

That would work for us! I can help out on a PR if you're interested.

[zricethezav]

@adamdecaf certainly, if you're up for it. I probably won't be able to get to this for at least another week or two

[wolfch-elsevier]

Can you tell us how to access/list the default config so we can create a custom config based on the default?
I briefly combed through the source code and didn't find anything obvious.

[electriquo]

@wolfch-elsevier https://github.com/zricethezav/gitleaks/blob/master/config/gitleaks.toml

[very-doge-wow]

What's the current status on this? I'm undecided wether or not to implement a workaround as long as extending configs is not yet available.

[mazlum]

Hello,
Are there any updates?

[zricethezav]

I have a branch I'm working on locally and still deciding on the design of this. Next month or two probably, or maybe next week. idk

[zricethezav]

https://github.com/zricethezav/gitleaks/tree/extend-config wip

[zricethezav]

@adamdecaf @very-doge-wow @foolioo @wolfch-elsevier I just merged #926. Check out the updated https://github.com/zricethezav/gitleaks#configuration section for instructions on how to use this new feature. Feel free to pull down master and try it out before I release it sometime this week with 8.9.0

[very-doge-wow]

@zricethezav Awesome, thanks! Will test it out.

[very-doge-wow]

@zricethezav Is there a way to extend from a default config which is hosted somewhere else than GitHub? The reason being that I'm running gitleaks inside an isolated CI/CD environment which doesn't have access to GitHub directly, but only to a generic GitHub mirror inside an Artifactory server. Meaning: I would need to be able to use something like this:

[extend]
path = "https://some.artifactory.internal/artifactory/some-repo/gitleaks-config.toml"

[zricethezav]

@very-doge-wow not at the moment. What I recommend is pulling in https://some.artifactory.internal/artifactory/some-repo/gitleaks-config.toml prior to running gitleaks in your CI/CD and adding includeDefault=true to that config

[very-doge-wow]

@very-doge-wow not at the moment. What I recommend is pulling in https://some.artifactory.internal/artifactory/some-repo/gitleaks-config.toml prior to running gitleaks in your CI/CD and adding includeDefault=true to that config

But won't includeDefault=true try and download the default config from GitHub? Because that's exactly the problem, there's no access to public GitHub repositories. The Artifactory repository is a mirror of GitHub. That's why I wanted to use path instead and simply supply a reference to the remote default config inside Artifactory. Sorry if I didn't explain that clearly 😄

[zricethezav]

But won't includeDefault=true try and download the default config from GitHub?

@very-doge-wow, nope. includeDefault will use what is already baked into the binary

https://github.com/zricethezav/gitleaks/blob/master/config/config.go#L186-L206

https://github.com/zricethezav/gitleaks/blob/master/config/config.go#L13-L14

[very-doge-wow]

Okay perfect. Then that solves my problem! Thanks. 🏅

[maltemorgenstern]

@zricethezav awesome!
We just tested this feature and it is working just as expected/hoped 🥇

Looking forward to upgrading to a new version with this feature included 😄

[zricethezav]

With the release of https://github.com/zricethezav/gitleaks/releases/tag/v8.9.0 I'm gonna close out this issue. Thanks for being patient and hope you get some value out of the new feature. Feel free to re-open or continue the discussion 👍🏻