chore: expire webhook cert generation Jobs #346
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Set a finished TTL on webhook cert Jobs, to clean them after they are finished.
Which issue this PR fixes
Related to https://github.com/Kong/gateway-operator-archive/issues/429
Special notes for your reviewer:
Jobs may continue to produce failures and associated alerts, but should at least clean up their failures after.
There's ongoing discussion with the cloud gateways team re the TTL value. Ideally this doesn't need to be configurable, but we're unsure what a reasonable value is to allow Jobs of interest (essentially Jobs that are finished because they've failed and could warrant human review--successful Jobs should always be okay to clean up).
I initially chose 60s with the rationale that actively failing Jobs won't reach finished for a while, and will spend a while in a fail->backoff->retry loop before reaching their final finished state after they give up retrying. Upped it to 600s after initial discussion; 3600s/1h was the other value I thought would be reasonable.
PR Readiness Checklist:
Complete these before marking the PR as
ready to review
:CHANGELOG.md
release notes have been updated to reflect significant changes