Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
HPA scales up deployment to max replicas if using custom metrics that are exposed by the application and the pods fail to deploy with rolling update #87199
When I then look at the ReplicaSets, it seems to scale the NEW ReplicaSet (where all the containers are crashing), and leaving the old one alone. If I directly create a deployment that fails, without any previously running containers, then it will just fail with one pod, and not scale up.
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
After a couple of minutes the count should still be the same. And as you see, the metric is correctly retrieved.
The ONLY difference is that it provides a command line parameter to the container that makes it crash before the metrics page is up.
After a few more seconds, yet another one is spawned:
Note that NONE of these ever reached "Ready" state. And if I check the details on the HPA:
It does not say that it couldn't get metrics - it says that the metrics are above target. Which they CAN'T be because they're hard-coded. And as you see, the reported metrics on the HPA is 10/80, so WELL below the threshold.
After about 5 minutes:
If I delete the whole namespace and deploy the failing deployment directly (i.e., there's no running pods), the HPA seems to report that there's no metrics available, and it won't scale (as I would assume is the correct behavior):
Then it does NOT scale up, which is what I would expect.
Anything else we need to know?: