Enable Generic scanner in trufflehog v3

[vaibhavjpr]

Description

Enable Generic scanner in trufflehog as it's unable to scan any generic password or token committed in the code.

Problem to be Addressed

The generic.Scanner{} is commented out in defaults.go file under pkg/engine location. We need to enable this as it will help many orgs or communities to scan 16-64 character length passwords

Description of the Preferred Solution

The below changes are recommended
1. uncomment line number 1351 in defaults.go file generic.Scanner{}
2. Add below line after line number 249
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/generic"
3. Chnage the password or token length from 16 to 8 and it should be upto 64
Original value in the file https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/generic/generic.go#L56-L61
var keywords = []string{"pass", "token", "cred", "secret", "key"}

var (
// \x21-\x7e == ASCII 33 (0x21) and 126 (0x7e)
keyPat = regexp.MustCompile(detectors.PrefixRegex(keywords) + (\b[\x21-\x7e]{16,64}\b))
)

It should be changed to 8 character as many devs who are still not aware of password policy
and they may still put password of less than 16 characters and it will not be detected by trufflehog.

var keywords = []string{"pass", "token", "cred", "secret", "key"}

var (
// \x21-\x7e == ASCII 33 (0x21) and 126 (0x7e)
keyPat = regexp.MustCompile(detectors.PrefixRegex(keywords) + (\b[\x21-\x7e]{8,64}\b))
)

Additional Context

The screenshot is attached with the mail. It's able to detect 16 character length password in the file.

References

• #0000

[haf-tech]

without generic detector simple password fields in prop files will be not detected.

I would recommend that we can enable this detector too. Probably with an additional flag, which enable this detector, otherwise by default, the generic detector will be not integrated.

[SecTheBit]

I think the problem is not with the integration of "generic" detector, but with the quality of results. Suppose you have used the generic detector on a random GitHub repository. In that case, you will find out that there are too many false positives, which is possibly why the trufflesecurity team has not enabled it.

[haf-tech]

@SecTheBit yes correct, but without using it, you miss also all the clear problematic fields. This is the reason, why requesting this as optional detector and the caller/user has the option to decide, if one can live with possible false positives

[hoernrob]

would love to be able to find generic secrets as well, knowing that there will be false positives

[tbenthompson]

Perhaps this could at least be an optional feature?

I ran a test repository with five known secrets against both trufflehog and git-leaks. git-leaks found all five (along with three false positives). Trufflehog found only three of the five secrets. The two missing secrets would've been caught by this generic detector.

[khanhnh95]

Hi guys,

Any updates on this?

[itoffshore]

I think this should be an --option that can be optionally enabled