Conflicts on update don't count as retry

[maruina]

What type of PR is this?

/kind bug

What this PR does / why we need it:

When updating the resource quota, e.checkAttributes calls e.checkQuotas up to 3 times. This is to prevent transient failures when updating the resource quota object via e.quotaAccessor.UpdateQuotaStatus. In case of an error, we call e.checkQuotas again and we decrement the remaining retries.

On busy clusters, we might exhaust the 3 retries before making a successful update, bubbling up the error to the client.

This PR is adding a check to avoid decrementing the retries when the error is because of a conflict. While this might increase the processing times because of the additional retries, there are two benefits:

• doesn't bubble up the error to the user
• we're still respecting the 10s timeout

If upstream doesn't like this approach, we might consider switching to a workqueue.TypedRateLimitingInterface so we can use a rate limiter before re-adding the object on the queue. This will required more code changes the requeue would happen at a higher level, so we need to be able to inspect the waiter to understand why it got denied.

Which issue(s) this PR fixes:

Fixes kubernetes#67761

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[maruina]

Artifacts: https://github.com/DataDog/kubernetes/releases/tag/v1.31.6-dd.2-rq
Image: https://github.com/DataDog/images/pull/6824

[boymucheng]

Has this method been tried? Is it feasible?