Change dataflow and nificluster controllers to avoid changing status on every reconciliation loop #142
Conversation
…on every reconciliation loop
| } | ||
|
|
||
| return Reconciled() | ||
| return RequeueAfter(intervalRunning) |
There was a problem hiding this comment.
@cannonpalms asked:
Can you speak to the strategy for watching child resources (namely pods) now that the NifiCluster controller is not flapping?
Prior to this change, the NifiCluster controller perpetually triggered reconciliation requests by updating its own status (Reconciling vs Running). The controller still watches all of the resources it was watching before, including all of the pods. The difference is that now the controller schedules reconciliation requests respecting its reconciliation interval of 15s -- that's the change on this line of code. Reconciliation requests will also be triggered if the NifiCluster CRD is modified by a user.
The same is true for the NifiDataflow controller.
Definitely! Let me add an event there instead |
|
@erdrix I've made sure an event notification and log message are generated when the controllers begin & end reconciliation 👍 |
|
@cannonpalms, can you confirm that this solves some of your problems? |
Since the dataflow controller will now respect its interval, should we either revert the division or reset the default interval the 5 ( Line 86 in b3a6369 |
|
Just FYI i've deployed this branch w/ argo to a remote k8s cluster and verified that the
A screenshot doesn't really prove this, but the resource |
What's in this PR?
Tweaks
NifiClusterandNifiDataflowcontrollers to avoid changing their respective statuses on every reconciliation loop. Instead, the controllers will only change the status to ready/done and log any actions taken thereafter and they will respect their reconciliation interval settings (15s by default). Previously to this, the controllers would reconcile many times per second, which drastically increases the load on the control plane and on NiFi itself.Additionally, i changed the logger timestamp encoder to user ISO8601 instead of epoch time so it is human readable.
Why?
Issue #119 does an excellent job of explaining why this is an issue.
Checklist