Add step to edit pod annotation controller.kubernetes.io/pod-deletion-cost before scaling.

[LQBing]

Is your feature request related to a problem? Please describe.

The different pods of an application could have different utilization levels. On scale down, the application may prefer to remove the pods with lower utilization. This works if the application itself controls the down scaling; for example, the driver pod of a Spark deployment.

Kubernetes v1.22 added a new annotation controller.kubernetes.io/pod-deletion-cost` for pod. But I don't know how to add a step to reorder pod-deletion-cost in CPA.

Describe the solution you'd like

Adding a step allows me to customize the sorting pod-deletion-cost action before scaling.

Additional context

https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#pod-deletion-cost

https://kubernetes.io/docs/reference/labels-annotations-taints/#pod-deletion-cost

[jthomperoo]

I think this is a great idea - I had previously looked at something similar to this but couldn't find any way to do it with Kubernetes, but as your links show it looks like that's changed with Kubernetes v1.22. I've assigned this to myself to investigate what the work would be to implement this. Thanks for raising this!

[lyudmilalala]

Want to quickly ask that, if I want to take advantage of the pod-deletion-cost, is it workable to set the pod's pod-deletion-cost by myself in my own application?
In my understanding, CPA just help me to customize my logic of determining the number of replicas to scale, so when k8s really take the deletion action, it will still consider the pod-deletion-cost. I have not finish reading the code of the CPA operator yet, so please correct me if I am not on the right way.

[jthomperoo]

Hi @lyudmilalala that's correct, you should be able to set the pod-deletion-cost yourself, the CPA will just interface with Kubernetes scaling APIs for adjusting replica counts, so this shouldn't overwrite any modifications you make, and the scaling API should respect these pod-deletion-costs.