csr: add resync to csr approver

[mikedanese]

fixes #49787

Fix an issue where if a CSR is not approved initially by the SAR approver is not retried.

[liggitt]

I definitely wouldn't expect resyncs this frequently

[mikedanese]

Picked 15 seconds for initial testing but I was hoping to discuss what would be reasonable in this review. What do you think?

[jcbsmpsn]

Our use case is sometimes nodes post a CSR before the RBAC role bindings are initialize during cluster start up. In that particular case, 15s would be great. In the more general case, a few minutes seems appropriate.

If there are a lot of CSRs in the system, they will all be reprocessed with each resync, right? Approved/Denied ones will be quickly processed, but they will all go through this queue each cycle, right?

[jcbsmpsn]

More like outside the state of this controller. Adding a new RBAC role binding is not exactly outside the Kubernetes universe.

[mikedanese]

Not just rbac. An IAM change could require resyncing the controller.

[mikedanese]

@liggitt @jcbsmpsn I pushed an alternative approach that relies on the controller default backoff which is:

// DefaultControllerRateLimiter is a no-arg constructor for a default rate limiter for a workqueue.  It has
// both overall and per-item rate limitting.  The overall is a token bucket and the per-item is exponential
func DefaultControllerRateLimiter() RateLimiter {
	return NewMaxOfRateLimiter(
		NewItemExponentialFailureRateLimiter(5*time.Millisecond, 1000*time.Second),
		// 10 qps, 100 bucket size.  This is only for retry speed and its only the overall factor (not per item)
		&BucketRateLimiter{Bucket: ratelimit.NewBucketWithRate(float64(10), int64(100))},
	)
}

Basically recognized certificates will be continuously with a backoff to ~15 minutes.

[jcbsmpsn]

/lgtm

[k8s-github-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jcbsmpsn, mikedanese

Associated issue: 49787

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

• pkg/controller/OWNERS [mikedanese]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 49615, 49321, 49982, 49788, 50355)

[k8s-cherrypick-bot]

Removing label cherrypick-candidate because no release milestone was set. This is an invalid state and thus this PR is not being considered for cherry-pick to any release branch. Please add an appropriate release milestone and then re-add the label.

[k8s-cherrypick-bot]

Commit found in the "release-1.7" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this is an error find help to get your PR picked.