Create HPA v2 Stable API

[wangyysde]

Signed-off-by: wangyysde net_use@bzhy.com

What type of PR is this?

/kind feature
/kind design

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #102362

Special notes for your reviewer:

I found that "list_type_missing,k8s.io/api/autoscaling/v2beta2" listed in the api/api-rules/violation_exceptions.list. So I generated code by run build/run.sh make UPDATE_API_KNOWN_VIOLATIONS=true.

Does this PR introduce a user-facing change?

Create HPA v2 from v2beta2 with some fields changed. 

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-autoscaling/2702-graduate-hpa-api-to-GA

[fejta-bot]

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

[wangyysde]

/label api-review

[fedebongio]

/sig autoscaling

[fedebongio]

/remove-sig api-machinery

[josephburnett]

/assign

[elmiko]

this generally makes sense to me, but as i am newer to this code base i have a question.

[josephburnett]

This is great! Thanks.

One request: can you please split this into two commits.

1. just the API types (types.go) so we can review the API itself
2. the rest of the conversion and registration mechanics

I think our API reviewer will only want to see the first commit.

[wangyysde]

@josephburnett Thanks! I will split this into two commits and modify the files following your reviews.

[josephburnett]

I'm unable to get a v2 HPA object when I set this up with kubetest --build --up.

josephburnett@josephburnett:~/go/src/k8s.io/kubernetes$ kubectl get hpa.v2beta2.autoscaling
Warning: autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.22+, unavailable in v1.25+
NAME         REFERENCE               TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   <unknown>/50%   1         10        1          35s

josephburnett@josephburnett:~/go/src/k8s.io/kubernetes$ kubectl get hpa.v2.autoscaling
error: the server doesn't have a resource type "hpa"

[josephburnett]

PR to fix the build rule violation list: https://github.com/wangyysde/kubernetes/pull/1

However I would prefer we fix the violation in the new API so this isn't necessary.

[wangyysde]

/test pull-kubernetes-unit

[wangyysde]

/label api-review

[wangyysde]

rebase on latest master and rerun, and you'll get a diff

@liggitt @josephburnett @fedebongio Thanks. The all tests have passed after I rebased on master. Coult you review this PR?

[josephburnett]

/lgtm
/approve

[liggitt]

/approve

for API changes

/hold are the follow-up PRs switching the controller to use v2, and adding e2e CRUD coverage of the v2 API ready?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: josephburnett, liggitt, wangyysde

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• api/OWNERS [liggitt]
• cmd/kube-apiserver/OWNERS [liggitt]
• hack/OWNERS [liggitt]
• pkg/api/OWNERS [liggitt]
• pkg/apis/OWNERS [liggitt]
• pkg/controller/podautoscaler/OWNERS [josephburnett,liggitt]
• pkg/controlplane/OWNERS [liggitt]
• pkg/registry/OWNERS [liggitt]
• staging/src/k8s.io/api/OWNERS [liggitt]
• staging/src/k8s.io/client-go/OWNERS [liggitt]
• test/cmd/OWNERS [liggitt]
• test/integration/apiserver/OWNERS [liggitt]
• test/integration/etcd/OWNERS [liggitt]
• vendor/OWNERS [liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[josephburnett]

are the follow-up PRs switching the controller to use v2, and adding e2e CRUD coverage of the v2 API ready?

I sent the PR switching the controller to use v2 to @wangyysde directly (https://github.com/wangyysde/kubernetes/pull/2) which he merged into this PR. So this one has the horizontal.go changes to use the v2 API.

Regarding CRUD coverage, we already have E2E tests in horizontal_pod_autoscaling.go that exercise the v1 and v2beta2 APIs (see CreateCPUHorizontalPodAutoscaler and CreateContainerResourceCPUHorizontalPodAutoscaler respectively). And custom_metrics_stackdriver_autoscaling.go also covers v2beta2. So I would like to just change those tests to use the new v2 stable API. I can prepare the PR.

[liggitt]

I sent the PR switching the controller to use v2 to @wangyysde directly (wangyysde#2) which he merged into this PR. So this one has the horizontal.go changes to use the v2 API.

It looks like the controller is still making v1 API requests, then internally converting to v2. I'd expect the controller to start using the v2 API directly and stop doing v1 <-> v2 conversions. That's fine to do in a follow-up PR.

[liggitt]

/hold cancel

will look for the follow-up PR to switch the controller API calls to v2 and switch the e2e tests from v2beta2 to v2

[josephburnett]

controller to start using the v2 API directly and stop doing v1 <-> v2 conversions

Why do we want to do this?

The data is stored on disk in v1 format so we would have to carefully migrate it. That would be costly. We aren't planning to deprecate the v1 API since it's GA and the v1 <--> v2 conversion is lossless.

We will deprecate the v2beta1 and v2beta2 APIs so we have to stop using those of course.

[josephburnett]

switch the e2e tests from v2beta2 to v2

I'm working on this one right now and running the tests. So I should have this out in the next day or so.

[liggitt]

controller to start using the v2 API directly and stop doing v1 <-> v2 conversions

Why do we want to do this?

Only kube-apiserver should be doing API conversions.

The data is stored on disk in v1 format so we would have to carefully migrate it. That would be costly.

Switching kube-controller-manager to use the v2 API is ~free, because kube-apiserver is already maintaining a cache of HPA objects in v2 format for the watch cache, so the v1<->v2 conversions will be done regardless of whether the controller uses v2 or not.

(separately, in 1.24, the storage version should switch to v2, which is more expressive and avoids needing to constantly do annotation serialization/deserialization in conversion like v1 does)

[josephburnett]

Switching kube-controller-manager to use the v2 API is ~free

Got it. So I'll send a PR to remove the conversion too.

in 1.24, the storage version should switch to v2

And I'll prepare this follow PR too after 1.23 code freeze lifts.

Thanks!

[sftim]

Did we ever document which fields?

The release note says:

Create HPA v2 from v2beta2 with some fields changed. 

but I can't find confirmation of which those were.

[liggitt]

no serialized fields changed, the name of a couple constants in the go files changed... the release note should have said "Promoted HPA v2beta2 to v2 with no schema changes."

[sftim]

Thanks @liggitt