#74: Testing of seekrets rulesets using bats #87
Conversation
rogeruiz
commented
Nov 9, 2016
rogeruiz
commented
- basic functional tests for git-seekrets
alain-hoang
changed the title
alain-hoang
changed the title
|
@alain-hoang I would expect to see a couple more tests here, since the newrelic rule has explicit exceptions ( |
|
More tests updated to take into account unmatch rules for newrelic. Also a new feature has been submitted upstream for being able to query for active hooks in a git repository #27 |
|
Waiting for feedback from upstream |
LinuxBozo
force-pushed
the
seekrets-install
branch
from
970f301
to
786415c
Compare
* basic functional tests for git-seekrets
* Add more tests to cover new relic false matches and aws ids
LinuxBozo
force-pushed
the
seekrets-bats
branch
from
0b99561
to
f951d75
Compare
pulled out version into variable gitseekretversion
|
|
||
| @test "git-seekrets does not find newrelic false positives in test repo" { | ||
| run addFileWithFalseNewrelicSecrets | ||
| [ $(expr "${lines[1]}" : "Found Secrets: 0") -eq 0 ] |
There was a problem hiding this comment.
This test fails. Going by previous tests, shouldn't this be -ne 0?
There was a problem hiding this comment.
If a confirmed secret is found it would be expected that the return value would satisfy the condition -ne 0 as mentioned. This test is specifically trying to test a false positive which should not show up as a secret. If it does show up as a secret then this would be an invalid test.
There was a problem hiding this comment.
Given that logic, and the following:
#!/bin/bash
RIGHT="right"
if [ $(expr "${RIGHT}" : "right") -eq 0 ]; then
echo "yep"
else
echo "nope"
fi
You would then expect to see "yep" echo back on the terminal then, yes?
$ ./test-eq-0.sh
nope
There was a problem hiding this comment.
I've put in an update that addresses this. I think my original thought was to capture the exit value of the expr command however the intention of the above code will test against the output generated from the expr command. I've updated the test appropriately. Thank you for pointing this out @LinuxBozo!
|
|
||
| @test "git-seekrets only matches newrelic secrets in test repo" { | ||
| run addFileWithSomeNewrelicSecrets | ||
| [ $(expr "${lines[1]}" : "Found Secrets: 1") -eq 0 ] |
There was a problem hiding this comment.
This test fails. Going by previous tests, shouldn't this be -ne 0?
| [ $status -gt 0 ] | ||
| } | ||
|
|
||
| @test "git-seekrets does not find secrets in test repo" { |
There was a problem hiding this comment.
This test is currently failing. Needs investigation into why and addressed.
There was a problem hiding this comment.
False positive from outdated binary.
|
After looking into the test failures, it appears that the draft binary is being built with upstream which does not include the 18F changes necessary to work. I have put in an updated binary that is built against 18F's version of the git-seekret repo. |
update seekrets-install to do curl -O from 18F repo
