ci: add retry to cosign keyless test #1109
Conversation
Codecov ReportAll modified lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1109 +/- ##
=======================================
Coverage 52.30% 52.30%
=======================================
Files 101 101
Lines 6305 6305
=======================================
Hits 3298 3298
Misses 2688 2688
Partials 319 319 ☔ View full report in Codecov by Sentry. |
akashsinghal
requested review from
binbin-li,
jimmyraywv,
luisdlp,
susanshi and
toddysm
as code owners
| @@ -81,8 +81,7 @@ SLEEP_TIME=1 | |||
| run kubectl replace -f ./config/samples/config_v1beta1_store_oras.yaml | |||
| sleep 5 | |||
|
|
|||
| run kubectl run cosign-demo-keyless --namespace default --image=wabbitnetworks.azurecr.io/test/cosign-image:signed-keyless | |||
| assert_success | |||
| wait_for_process 20 10 'kubectl run cosign-demo-keyless --namespace default --image=wabbitnetworks.azurecr.io/test/cosign-image:signed-keyless' | |||
There was a problem hiding this comment.
consider adding a retry message with outputs of the command
There was a problem hiding this comment.
consider updating the cosign verifier docs to mention potential timeout
There was a problem hiding this comment.
added note to the cosign doc. This doc should probably be migrated to the website.
added a retry statement with the command that gets outputed to a separate IO stream that is always printed. see docs: https://bats-core.readthedocs.io/en/stable/writing-tests.html#printing-to-the-terminal
| @@ -68,6 +68,7 @@ wait_for_process() { | |||
| return 0 | |||
| else | |||
| sleep "$sleep_time" | |||
| echo "# retrying $cmd" >&3 | |||
There was a problem hiding this comment.
This the FD for the TAP stream produced by BATS. The TAP stream is always outputted to the console regardless of success or failure. I think it might be helpful to see if retries were for a success and not just a failure. https://bats-core.readthedocs.io/en/stable/writing-tests.html#printing-to-the-terminal
Description
What this PR does / why we need it:
Adds 3 retries to the cosign keyless test due to intermittent rekor server timeouts.
Adds 10 second backoff
Which issue(s) this PR fixes (optional, using
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when the PR gets merged):Fixes #1057
Type of change
Please delete options that are not relevant.
mainbranch)How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration
Checklist:
Post Merge Requirements
Helm Chart Change