Reduce RCR Throttling #1545
Reduce RCR Throttling #1545
Conversation
Signed-off-by: Shuting Zhao <shutting06@gmail.com>
realshuting
force-pushed
the
reduce_rcr_throttling
branch
from
fd67c3a
to
7b4129a
Compare
| @@ -86,6 +82,7 @@ func NewReportChangeRequestGenerator(client *policyreportclient.Clientset, | |||
| polListerSynced: polInformer.Informer().HasSynced, | |||
| queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), workQueueName), | |||
| dataStore: newDataStore(), | |||
| requestCreator: newChangeRequestCreator(dclient, 3*time.Second, log.WithName("requestCreator")), | |||
There was a problem hiding this comment.
I set the default ticker interval to 3 seconds (the changeReqeustCreator will aggregate buffered requests every 3s). Should we allow this to be tuned dynamically across various loads of clusters?
There was a problem hiding this comment.
So, if there are no buffered requests will there be an overhead?
Why not use a fixed size pool of workers that block on the queue instead?
There was a problem hiding this comment.
if there are no buffered requests will there be an overhead
No.
We can use a fixed size pool, but also need to set a "timeout" interval to flush the queue. So I buffer the requests and aggrate them every fixed interval.
Signed-off-by: Shuting Zhao <shutting06@gmail.com>
Signed-off-by: Shuting Zhao <shutting06@gmail.com>
Signed-off-by: Shuting Zhao <shutting06@gmail.com>
| @@ -86,6 +82,7 @@ func NewReportChangeRequestGenerator(client *policyreportclient.Clientset, | |||
| polListerSynced: polInformer.Informer().HasSynced, | |||
| queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), workQueueName), | |||
| dataStore: newDataStore(), | |||
| requestCreator: newChangeRequestCreator(dclient, 3*time.Second, log.WithName("requestCreator")), | |||
There was a problem hiding this comment.
So, if there are no buffered requests will there be an overhead?
Why not use a fixed size pool of workers that block on the queue instead?
* buffer report change requests Signed-off-by: Shuting Zhao <shutting06@gmail.com> * fix clusterReportChangeRequest Signed-off-by: Shuting Zhao <shutting06@gmail.com> * further reduce RCRs in background scan Signed-off-by: Shuting Zhao <shutting06@gmail.com> Signed-off-by: vyankatesh_neualto <vyankatesh@neualto.com>
Related issue
Fixes #1497.
This PR adds a buffer to slow down report change request creation.