Increase or Decrease polling rate for instance in a final state ready or failed. #53
Conversation
…about the service binding controller.
Co-authored-by: RalfHammer <119853077+RalfHammer@users.noreply.github.com>
|
|
||
| func TestReconcilerHelpers(t *testing.T) { | ||
| RegisterFailHandler(Fail) | ||
| // RunSpecs(t, "Reconciler Helpers Suite") |
There was a problem hiding this comment.
Do you want to activate this test?
santiago-ventura
requested review from
TheBigElmo
and removed request for
bKiralyWdf
|
Hi @TheBigElmo, please review the PR. |
|
Why would you set this on a CRO level, but not in general on startup of the controller, so you can override the default values? Do you have numbers for end users, when to adjust it (e.g. number of resources)? Are the CF rate limits maybe too low? |
|
Hi @TheBigElmo I understand your question about why not set these time durations for general use on the controller's startup. In our use case, we might want different timeout values depending on which instance we're creating and its purpose. So, by making it configurable per instance, we can always set it when we create the resource. We can differentiate it, so we took this approach because of the ability to decouple it per resource. |
|
That's completely fine to do this on a CRO level, but it should have a possibility on startup too. |
|
|
@TheBigElmo, you are right; it is possible to have both options. |
There was a problem hiding this comment.
Maybe mentioning the default values/behavior is helpful.
There was a problem hiding this comment.
I made some changes to the document.
This includes information about the default values and behavior under Default Requeue After Interval.
|
ok, let's do it with another PR next.
|
Hi @TheBigElmo Here is the number we have managed so far without reconciling the limit. It reconciles every 10 min at least; we would have with caching six requests per instance per hour. Keep in mind that Spaces are much more wasteful, taking 240 requests per hour. So, in theory, we would have problems at around 10k instances per user, assuming no other workload. If you add keys, it becomes more like 5k. In practice, we have problems around 200 spaces, which makes sense given that it's 200*240 > 40k requests. We currently have a backlog item for testing how many requests the operator will execute with 10.000 CRs. |
TheBigElmo
deleted the
decrease-polling-rate-for-instance-in-a-final-state
branch
|
Hi @TheBigElmo Yes, our product owner will create a backlog item for the controller flags. |
In order to prevent the Operator hits the CF rates limits, we have introduced two new annotations,
service-operator.cf.cs.sap.com/polling-interval-readyandservice-operator.cf.cs.sap.com/polling-interval-failto facilitate the increase or decrease of the re-queue time duration when a Custom Resources is in a State final Ready or Failed.