prefix GVK labels in CustomResourceMonitoring #1942

bavarianbidi · 2023-01-03T12:35:00Z

Signed-off-by: Mario Constanti mario@constanti.de

What this PR does / why we need it:

This PR will prefix the auto-generated GVK labels for CustomResources with cr_ to make it a bit more clear that these labels got generated.
It also make it more easier to reflect e.g. a version of a CR in a metric by just label it version instead of some specific app_version (or similar).

How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality)

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

k8s-ci-robot · 2023-01-03T12:35:07Z

@bavarianbidi: This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

CatherineF-dev · 2023-01-03T14:32:44Z

I think "group" is fine and it aligns with k8s documentations, cc @rexagod to have another review.

dgrisonnet · 2023-01-03T18:12:03Z

I am a bit on the fence here as I like the current naming because it aligns with the labels we have in the kube-apiserver HTTP metrics. But at the same time I can see how one could want to use a version label for other purposes. Maybe we could prefix them with crd similarly to how we prefix the metrics name kube_crd?

Do you have an opinion on that @logicalhan?

bavarianbidi · 2023-01-04T08:00:45Z

Don't get me wrong, because it's really nice to get the group, version and kind data as label values in the metric without additional work.
But the main purpose from a user point of view is to reflect information from CRs.
As many third-party-components have a version field which i'm more interested in than the version of the CRD itself (as i already have to define the GVK for a CRD in the configuration) i'm very happy if we find a solution for that.

As an alternative we could also think about an additional configuration value/flag where a user can opt-in/opt-out the auto-generated GVK labels.

Beside the origin issue with the version field, i've already stumbled 2 times this morning about the version field in a CR where i have to decided how to continue.

from cluster-api-azure, where an AzureMachinePool can reflect the version of an image in the status field:

k explain azuremachinepool.status.image.marketplace.version
KIND:     AzureMachinePool
VERSION:  infrastructure.cluster.x-k8s.io/v1beta1

FIELD:    version <string>

DESCRIPTION:
     Version specifies the version of an image sku. The allowed formats are
     Major.Minor.Build or 'latest'. Major, Minor, and Build are decimal numbers.
     Specify 'latest' to use the latest version of an image available at deploy
     time. Even if you use 'latest', the VM image will not automatically update
     after deploy time even if a new version becomes available.

from flux, where you can define a HelmRelease and point to a specific version:

$ k explain helmreleases.spec.chart.spec.version
KIND:     HelmRelease
VERSION:  helm.toolkit.fluxcd.io/v2beta1

FIELD:    version <string>

DESCRIPTION:
     Version semver expression, ignored for charts from v1beta2.GitRepository
     and v1beta2.Bucket sources. Defaults to latest when omitted.

dgrisonnet · 2023-01-04T11:12:38Z

As many third-party-components have a version field which i'm more interested in than the version of the CRD itself (as i already have to define the GVK for a CRD in the configuration) i'm very happy if we find a solution for that.

Yeah this scenario totally makes sense.

As an alternative we could also think about an additional configuration value/flag where a user can opt-in/opt-out the auto-generated GVK labels.

I wouldn't be in favor of that since I don't think that there are any advantages to removing the GVK labels since they provide key information for barely any overhead.

dgrisonnet · 2023-01-04T11:20:09Z

docs/customresourcestate-metrics.md

@@ -49,7 +49,7 @@ spec:
          - --resources=certificatesigningrequests,configmaps,cronjobs,daemonsets,deployments,endpoints,foos,horizontalpodautoscalers,ingresses,jobs,limitranges,mutatingwebhookconfigurations,namespaces,networkpolicies,nodes,persistentvolumeclaims,persistentvolumes,poddisruptionbudgets,pods,replicasets,replicationcontrollers,resourcequotas,secrets,services,statefulsets,storageclasses,validatingwebhookconfigurations,volumeattachments,verticalpodautoscalers
 ```

-NOTE: The `group`, `version`, and `kind` common labels are reserved, and will be overwritten by the values from the `groupVersionKind` field.
+NOTE: The `cr_group`, `cr_version`, and `cr_kind` common labels are reserved, and will be overwritten by the values from the `groupVersionKind` field.


I would prefer if we had some consistency between this prefix and the one in the metric name. Currently the metric name is using the crd prefix which might not be well suited. Might be better to go with kube_cr or kube_custom_resources.

do you mean we should use the future prefix in the metric name and in the label itself?
I like kube_custom_resource more than kube_cr.

Yes, I'd like to avoid having crd somewhere and cr elsewhere, it's better be consistent.

I agree, would be nice to have one thing. I believe right now we mix Custom Resource, Custom Resource Definition and Custom Resource State. kube_customresource would align with kube_horizontalpodautoscaler (no additional underscores).

Should we rename everything custom resource state into simply custom resource and abbreviate it with "cr" where necessary?

~~i've pushed a new commit (ed85920) - this should reflect the way we want to go further~~

rebased and squashed

FYI: Based on the previous commit i started to improve the unit-tests because they didn't cover the default generated group,version,kind labels atm. will continue on monday morning.

CatherineF-dev · 2023-01-04T13:07:32Z

Another way might be kube-state-metrics to support renaming labels for CRs.
So that kube-state-metrics collects HelmRelease.version and renames it into helm_version.

$ k explain helmreleases.spec.chart.spec.version
KIND:     HelmRelease
VERSION:  helm.toolkit.fluxcd.io/v2beta1

FIELD:    version <string>

DESCRIPTION:
     Version semver expression, ignored for charts from v1beta2.GitRepository
     and v1beta2.Bucket sources. Defaults to latest when omitted.

This will prefix the auto-generated GVK labels for CustomResources with customresource_ to make it a bit more clear that these labels got generated. Signed-off-by: Mario Constanti <mario@constanti.de>

CatherineF-dev · 2023-01-09T14:10:04Z

pkg/customresourcestate/custom_resource_metrics_test.go

+						Help: "metrics for testing",
+						Each: &compiledInfo{},
+						Labels: map[string]string{
+							"customresource_group":   "apps",


customresource_group to cr_group?

comming from this comment

I agree, would be nice to have one thing. I believe right now we mix Custom Resource, Custom Resource Definition and Custom Resource State. kube_customresource would align with kube_horizontalpodautoscaler (no additional underscores).

Should we rename everything custom resource state into simply custom resource and abbreviate it with "cr" where necessary?

i've started with customresource (because i like it a bit more) but as i'm not very opinionated i'm also fine to switch to cr

cc @mrueg which label name do you prefer? cr_group or customresource_group

My personal preference is the second one, two letter acronyms can become ambiguous

pkg/customresourcestate/config.go

docs/verticalpodautoscaler-metrics.md

docs/customresourcestate-metrics.md

Signed-off-by: Mario Constanti <mario@constanti.de>

mrueg · 2023-01-10T12:57:29Z

/approve

Thanks for your contribution!

rexagod · 2023-01-10T20:36:15Z

/approve

bavarianbidi · 2023-01-11T18:43:41Z

I've got 2 approvals... Who could give an LGTM to get this merged?

mrueg · 2023-01-11T23:40:53Z

I'll make sure to merge it by the end of this week, want to provide other maintainers the opportunity to give further feedback

dgrisonnet

Thank you for bearing with us @bavarianbidi :)

/lgtm
/approve

k8s-ci-robot · 2023-01-12T08:57:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bavarianbidi, dgrisonnet, mrueg, rexagod

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dgrisonnet,mrueg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 3, 2023

k8s-ci-robot requested review from CatherineF-dev and fpetkovski January 3, 2023 12:35

k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Jan 3, 2023

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 3, 2023

This was referenced Jan 3, 2023

KSM in v2.7.0 lead to some conflicts with the existing CustomResourceMetricConfiguration kubernetes-sigs/cluster-api#7832

Closed

reserved labels of CustomResourceStateMetrics are getting overwritten #1943

Closed

bavarianbidi mentioned this pull request Jan 4, 2023

machinepool capi giantswarm/cluster-api-monitoring-app#28

Merged

3 tasks

dgrisonnet reviewed Jan 4, 2023

View reviewed changes

prefix GVK labels in CustomResourceMonitoring

7130bd0

This will prefix the auto-generated GVK labels for CustomResources with customresource_ to make it a bit more clear that these labels got generated. Signed-off-by: Mario Constanti <mario@constanti.de>

bavarianbidi force-pushed the prefix_gvk_labels_in_customresourcemonitoring branch from ed85920 to 7130bd0 Compare January 9, 2023 14:07

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 9, 2023

bavarianbidi requested review from mrueg and dgrisonnet and removed request for fpetkovski, CatherineF-dev and mrueg January 9, 2023 14:08

CatherineF-dev reviewed Jan 9, 2023

View reviewed changes

bavarianbidi requested review from mrueg and dgrisonnet and removed request for dgrisonnet and mrueg January 9, 2023 15:11

mrueg reviewed Jan 10, 2023

View reviewed changes

pkg/customresourcestate/config.go Show resolved Hide resolved

bavarianbidi requested review from mrueg and removed request for dgrisonnet January 10, 2023 12:20

mrueg reviewed Jan 10, 2023

View reviewed changes

docs/verticalpodautoscaler-metrics.md Outdated Show resolved Hide resolved

mrueg reviewed Jan 10, 2023

View reviewed changes

docs/customresourcestate-metrics.md Outdated Show resolved Hide resolved

replace kube_crd with kube_customresource in docs

b410166

Signed-off-by: Mario Constanti <mario@constanti.de>

bavarianbidi force-pushed the prefix_gvk_labels_in_customresourcemonitoring branch from cfb26ce to b410166 Compare January 10, 2023 12:31

bavarianbidi added 2 commits January 10, 2023 13:50

fix vpa crd metric names

771f85c

update crd monitoring and mention new flags

488c0bc

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 10, 2023

bavarianbidi requested review from dgrisonnet and CatherineF-dev and removed request for dgrisonnet and CatherineF-dev January 10, 2023 14:01

dgrisonnet approved these changes Jan 12, 2023

View reviewed changes

k8s-ci-robot assigned dgrisonnet Jan 12, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 12, 2023

k8s-ci-robot merged commit b7a7070 into kubernetes:main Jan 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

prefix GVK labels in CustomResourceMonitoring #1942

prefix GVK labels in CustomResourceMonitoring #1942

bavarianbidi commented Jan 3, 2023

k8s-ci-robot commented Jan 3, 2023

CatherineF-dev commented Jan 3, 2023

dgrisonnet commented Jan 3, 2023

bavarianbidi commented Jan 4, 2023 •

edited

dgrisonnet commented Jan 4, 2023

dgrisonnet Jan 4, 2023

bavarianbidi Jan 4, 2023

dgrisonnet Jan 4, 2023

mrueg Jan 4, 2023

bavarianbidi Jan 5, 2023 •

edited

bavarianbidi Jan 5, 2023 •

edited

CatherineF-dev commented Jan 4, 2023

CatherineF-dev Jan 9, 2023

bavarianbidi Jan 9, 2023

CatherineF-dev Jan 10, 2023

mrueg Jan 10, 2023

mrueg commented Jan 10, 2023 •

edited

rexagod commented Jan 10, 2023

bavarianbidi commented Jan 11, 2023

mrueg commented Jan 11, 2023

dgrisonnet left a comment

k8s-ci-robot commented Jan 12, 2023

prefix GVK labels in CustomResourceMonitoring #1942

prefix GVK labels in CustomResourceMonitoring #1942

Conversation

bavarianbidi commented Jan 3, 2023

k8s-ci-robot commented Jan 3, 2023

CatherineF-dev commented Jan 3, 2023

dgrisonnet commented Jan 3, 2023

bavarianbidi commented Jan 4, 2023 • edited

dgrisonnet commented Jan 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bavarianbidi Jan 5, 2023 • edited

Choose a reason for hiding this comment

bavarianbidi Jan 5, 2023 • edited

Choose a reason for hiding this comment

CatherineF-dev commented Jan 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrueg commented Jan 10, 2023 • edited

rexagod commented Jan 10, 2023

bavarianbidi commented Jan 11, 2023

mrueg commented Jan 11, 2023

dgrisonnet left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jan 12, 2023

bavarianbidi commented Jan 4, 2023 •

edited

bavarianbidi Jan 5, 2023 •

edited

bavarianbidi Jan 5, 2023 •

edited

mrueg commented Jan 10, 2023 •

edited