Join GitHub today
Cannot autoscale StatefulSet via kubectl #48591
Is this a BUG REPORT or FEATURE REQUEST?:
I can create HPA policy for StatefulSet but cannot scale the StatefulSet via kubectl.
What you expected to happen:
I should be able to scale the StatefulSet via kubectl.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
It's weird, the document already claim support https://kubernetes.io/docs/tasks/run-application/scale-stateful-set/ , let me check more
@gyliu513 This isn't a bug. Autoscaling isn't implemented for StatefulSets or DaemonSets. Auto-scaling a stateless application based on cpu utilization generally makes sense. Autoscaling a stateful workload based on that criteria can often do more harm (at least in the short term) than good. If you are asking for an implementation of horizontal Pod auto-scaling for StatefulSet you should open this as a feature request.
HPA based on CPU makes two assumptions.
Stateful workloads may violate .1. For example, log structured merge trees and append only btrees may cause CPU spikes during compaction that are only loosely correlated with QPS and that do not vary with direct proportion to the current QPS.
More importantly, for distributed storage applications, when you increase the number of processes in the system by increasing the number of Pods, you may trigger replication or replica re-balancing (depending on the nature of the stateful application). This is likely to increase network and CPU utilization in the short term. Only after the re-balance or replication is complete would you be able to make use of the additional capacity offered by the new Pods.
For synchronously and asynchronously replicated ACID systems, reducing the number or replicas based on CPU utilization might be safe, but, for BASE systems, scaling down will also trigger a re-balance (so the systems data are not under-replicated), and potentially require global anti-entropy repair.
That isn't to say that Autoscaling is not possible, but it is not likely to have a trivial, general solution for StatefulSet. That is why I suggest opening this as a feature request.
@kow3ns I think Statefulset can't meet the requirements of applications like database、distributed storage service or even spark. Those applications is too complicate to fit in statefulset. They should be manipulated by customized controller through CRD, like what so many Operators do.