Add log-level flag to policy-tester, update output

[malancas]

Summary

When using policy-tester, I noticed the tool always output debug level logging. I added an optional log-level flag that will default to Info level logging. I also updated the Info level output to include logs indicating to the user what part of the verification process the tool was on.

Old successful verification output:

$ ./policy-tester --policy=test/testdata/policy-controller/tester/cip-public-keyless.yaml --image=ghcr.io/sigstore/cosign/cosign:v1.12.1
2024-05-09T09:08:21.768-0600	DEBUG	webhook/validator.go:498	Checking Authority: authority-0
2024-05-09T09:08:24.479-0600	DEBUG	webhook/validator.go:806	validated signature for ghcr.io/sigstore/cosign/cosign:v1.12.1, got 1 signatures
2024-05-09T09:08:24.479-0600	DEBUG	webhook/validator.go:644	Converting signature &{b:[123 34 99 114 105 116 105 99 97 108 34 58 123 34 105 100 101 110 116 105 116 121 34 58 123 34 100 111 99 107 101 114 45 114 101 102 101 114 101 110 99 101 34 58 34 103 99 114 46 105 111 47 112 114 111 106 101 99 116 115 105 103 115 116 111 114 101 47 99 111 115 105 103 110 34 125 44 34 105 109 97 103 101 34 58 123 34 100 111 99 107 101 114 45 109 97 110 105 102 101 115 116 45 100 105 103 101 115 116 34 58 34 115 104 97 50 53 54 58 97 99 56 101 48 56 97 50 49 52 49 101 48 57 51 102 52 102 100 55 100 49 100 48 98 48 53 52 52 56 56 48 52 101 98 51 55 55 49 98 54 54 53 55 52 98 49 51 97 100 55 51 101 51 49 98 52 54 48 97 102 54 52 100 34 125 44 34 116 121 112 101 34 58 34 99 111 115 105 103 110 32 99 111 110 116 97 105 110 101 114 32 105 109 97 103 101 32 115 105 103 110 97 116 117 114 101 34 125 44 34 111 112 116 105 111 110 97 108 34 58 123 34 71 73 84 95 72 65 83 72 34 58 34 48 98 97 97 48 52 52 98 101 97 54 49 101 55 99 49 54 100 53 54 48 50 51 98 101 50 48 101 97 100 51 100 57 50 48 52 98 50 52 97 34 44 34 71 73 84 95 86 69 82 83 73 79 78 34 58 34 118 49 46 49 50 46 49 34 125 125] b64sig:MEYCIQCOFGCC+Sj1GIigPu6MSaCdJ8tcwoN5PFXgr0n7AKLGhwIhAJtx4mbmg1ZA4NrXxR3SIYwijoj00X/aWfxA5ohzHNqO opts:0x14000020380}
{}

New successful verification output using the default Info level logging:

$ ./policy-tester --policy=test/testdata/policy-controller/tester/cip-public-keyless.yaml --image=ghcr.io/sigstore/cosign/cosign:v1.12.1 --log-level=info
2024-05-09T08:04:46.667-0600	INFO	tester/main.go:130	Validating policy

2024-05-09T08:04:46.693-0600	INFO	tester/main.go:147	Policy was successfully validated

2024-05-09T08:04:46.694-0600	INFO	tester/main.go:228	Verifying the provided image against the policy

2024-05-09T08:04:49.819-0600	INFO	tester/main.go:250	Verification was successful!

Old failing verification output:

{"errors":["ghcr.io/sigstore/cosign/cosign:v1.4.0 is uncovered by policy"]}

New failing verification output using the default Info level logging:

$ ./policy-tester --policy=test/testdata/policy-controller/tester/cip-public-keyless.yaml --image=ghcr.io/sigstore/cosign/cosign:v1.4.0 --log-level=info
2024-05-09T09:06:43.940-0600	INFO	tester/main.go:130	Validating policy

2024-05-09T09:06:43.970-0600	INFO	tester/main.go:147	Policy was successfully validated

2024-05-09T09:06:43.971-0600	INFO	tester/main.go:228	Verifying the provided image against the policy

2024-05-09T09:06:43.971-0600	INFO	tester/main.go:236	Errors encountered during verification

{"errors":["ghcr.io/sigstore/cosign/cosign:v1.4.0 is uncovered by policy"]}

We can definitely keep iterating on the output as needed but I think this is a good start.

Release Note

Documentation

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.92%. Comparing base (50ef092) to head (030f311).
Report is 54 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1414   +/-   ##
=======================================
  Coverage   52.92%   52.92%           
=======================================
  Files          44       44           
  Lines        3979     3979           
=======================================
  Hits         2106     2106           
  Misses       1651     1651           
  Partials      222      222           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cpanato]

lgtm

[hectorj2f]

@malancas Thanks for the PR!

[malancas]

@hectorj2f thanks for reviewing. Looks like there are few flaky test failures, would you mind rerunning them?

[hectorj2f]

@malancas Let me do it!