Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Use workqueue model in expand_controller #71760
What would you like to be added:
This model might cause some extra overhead for controller to periodically sync the desired and actual state and store them. We are considering to use workqueue model which has some build-in features such as setting rate limit (e.g., exponentialFailureRateLimiter) and exposing metrics.
Just for background - the reason we chose not to use workqueue is because, resize is level based. So resize of a PVC from 2GB to 4GB and then to 6GB and finally to 8GB - need not visit all the nodes in transaction. If while 2GB to 4GB resize was in progress and other resizes arrive, then for that PVC only final resize will be done ignoring intermediate ones. So, we will have to internally manage some state anyways, so as we don't process all resize operations on same PVC.
The rate limiting feature supported by workqueue is on per item basis, which I doubt will be useful for volume expansion where if a user is submitting multiple resizes for same PVC, intermediate resize requests are simply ignored.
I think depending on how fast user sends out their request and how fast controller processes, both desired/actual state and workqueue model might end up processing every request or just one final request. They should not have much differences.
For example, if user changes from 2GB to 4GB, and then change from 4GB to 6GB in the next second. In desired/actual state model, sync_volume_resize controller resync very 400 Millisecond. So it will process first request and then second request.
For workqueue model, we are not going to put every request into the queue. Instead, we put PVC unique name into the queue and worker get an item from the queue. If user put multiple PVC change requests very fast, the worker gets the PVC name and then get the lastest PVC object from the cache, it might resize directly from 2GB to 4GB.