Bug auditing test_data/config.yaml with binary values

[pablosnt]

We found the following bug scanning detect-secrets project when the file test_data/config.yaml is audited:

We tried to solve this bug in #414, and we get it in the most cases, but this file includes the following line: low_entropy_binary_secret: !!binary MjNjcnh1IDJieXJpdXYyeXJpaTJidnl1MnI4OXkyb3UwMg==. We try to parse this file as YAML from python and we get the following behaviour:

As can you see, the secret value changes when the file is parsed as YAML, so I think that this bug can be introduced by the YAML transformer. I'm not sure because I don't know how this transformer works exactly.

I also think that this bug is due to the specific value included in this file and it isn't a generic problem. I don't know how to fix it, so I open this issue to report it.

[domanchi]

Hmm. I'm unable to reproduce this. Can you provide a minimal YAML file that causes this issue on master?

Cases I've tried:

• low_entropy_binary_string in YAML file by itself
• low_entropy_binary_string in YAML file, preceded by a new line
• low_entropy_binary_string in YAML file, preceded by an arbitrary key pair
• low_entropy_binary_string in YAML file, preceded by a secret
• low_entropy_binary_string in YAML file, preceded by an arbitrary key pair + secret and a new line

They all have varying results (raising other issues), but none that raises this traceback that you provided.

[pablosnt]

Yes of course, the YAML file that causes this issue is this. Can you reproduce this?

[domanchi]

Oh goodie. This means we have good test cases. But the fact we didn't catch it in our test cases means we're missing something lol.

I was able to reproduce with this file. I'll check it out.

[pablosnt]

Thank you very much @domanchi !!