Fix HPA feedback from writing status.replicas to spec.replicas.

[josephburnett]

/kind bug

What this PR does / why we need it:

There are various reasons that the HPA will decide not the change the current scale. Two important ones are when missing metrics might change the direction of scaling, and when the recommended scale is within tolerance of the current scale.

The way that ReplicaCalculator signals it's desire to not change the current scale is by returning the current scale. However the current scale is from scale.Status.Replicas and can be larger than scale.Spec.Replicas (e.g. during Deployment rollout with configured surge). This causes a positive feedback loop because scale.Status.Replicas is written back into scale.Spec.Replicas, further increasing the current scale.

This PR fixes the feedback loop by plumbing the replica count from spec through horizontal.go and replica_calculator.go so the calculator can punt with the right value.

It also introduces separate types for replica counts derived from scale.Spec and scale.Status (specReplicas and statusReplicas respectively) to guard against this kind of cross-talk. When returning a desired scale to be written into spec, the calculator must either return the given, current scale, or explicitly call newSpecReplicas.

With separate types, other sources of cross-talk became compiler errors. E.g. recording Status.Replicas as an initial recommendation. This would manifest if a deployment was in the process of rolling out when the HPA reboots.

Which issue(s) this PR fixes:

Fixes #78712
Fixes #72775

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Hi @josephburnett. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mwielgus]

/ok-to-test

[josephburnett]

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

New comment.

[josephburnett]

If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.

Done.

[josephburnett]

/test pull-kubernetes-integration

[josephburnett]

/test pull-kubernetes-verify

[krzysztof-jastrzebski]

/lgtm

[juanpmarin]

Hi, is this currently released ?

[austinpray]

Hi, is this currently released ?

@juanpmarin it looks like this is gonna be in 1.16+

λ  kubernetes master ✓ git tag --contains 39c4875321991f305d51e30481a66701b6b76f5f
v1.16.0-alpha.1
v1.16.0-alpha.2
v1.16.0-alpha.3
v1.16.0-beta.0
v1.16.0-beta.1
v1.17.0-alpha.0

[kimxogus]

@juanpmarin This is also cherry picked to 1.13(#79707), 1.14(#79708), 1.15(#79709, #79727). Latest version of them may have this fix too.

[paalkr]

I would just like to draw some attention to #72775. Many are still facing issues with rolling updates.