migrate kubelet custom metrics to stability framework part 2

[RainbowMango]

What type of PR is this?
/kind feature

What this PR does / why we need it:
Currently, custom metrics emitted from kubelet do not offer any stability guarantees.

And #83062 tries to make it possible to do so.

About metrics stability please refer to KEP .

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:
IMO, we can use the stability framework to offer stability guarantee and drop the traditional versioned endpoint like xxx/v1alpha.

More details please refer to why not traditional versioned endpoint.

Please let me know if we need a follow-up issue or KEP.

Does this PR introduce a user-facing change?:

Following metrics from kubelet are now marked as with the ALPHA stability level:
node_cpu_usage_seconds_total
node_memory_working_set_bytes
container_cpu_usage_seconds_total
container_memory_working_set_bytes
scrape_error

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-instrumentation/20190404-kubernetes-control-plane-metrics-stability.md
- [Usage]: <link>
- [Other doc]: <link>

/priority important-soon
/milestone v1.17

[RainbowMango]

/cc @dashpole @brancz @logicalhan

[dashpole]

/lgtm
/approve

I agree that we should migrate away from the versioned endpoint after this PR. I think adding this prometheus handler at /metrics/resource in addition to /metrics/resource/v1alpha1 should be non-controversial. I'm not sure how long we should keep the /metrics/resource/v1alpha1 around, but 3 releases seems reasonable so.

[logicalhan]

/lgtm
/approve

[RainbowMango]

/assign @vishh @yujuhong
@kubernetes/sig-node-pr-reviews

[josiahbjorgaard]

A quick note that this PR will need to be merged before code freeze on 11/14 at 5PM pacific time for v1.17.

[derekwaynecarr]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dashpole, derekwaynecarr, logicalhan, RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/kubelet/OWNERS [derekwaynecarr]
• staging/src/k8s.io/component-base/metrics/OWNERS [logicalhan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[bethforsyth]

@RainbowMango Given the reversion of the container_cpu_usage_seconds_total -> container_cpu_usage_seconds change (from #84987) in #89540 , is it still correct that container_cpu_usage_seconds_total is listed as deprecated? e.g.

kubernetes/staging/src/k8s.io/component-base/metrics/testutil/promlint.go

Line 63 in 8f6ffb2

"container_cpu_usage_seconds_total", // non-counter metrics should not have "_total" suffix

[RainbowMango]

@bethforsyth Thanks. Would like to file a PR for this?

[bethforsyth]

@RainbowMango Great, I will do - just wanted to confirm I'd not misinterpreted first.