Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define semantic conventions for k8s metrics #1032

Open
ChrsMark opened this issue May 13, 2024 · 3 comments
Open

Define semantic conventions for k8s metrics #1032

ChrsMark opened this issue May 13, 2024 · 3 comments
Assignees
Labels
area:k8s enhancement New feature or request experts needed This issue or pull request is outside an area where general approvers feel they can approve

Comments

@ChrsMark
Copy link
Member

ChrsMark commented May 13, 2024

Area(s)

area:k8s

Is your change request related to a problem? Please describe.

At the moment there are not Semantic Conventions for k8s metrics.

Describe the solution you'd like

Even if we cannot consider the k8s metrics as stable we can start considering adding metrics that are not controversial to get some progress here. This issue aims to collect the existing k8s metrics that exist in the Collector and keep track of any related work.
Bellow I'm providing an initial list with metrics coming from the kubeletstats and k8scluster receivers. Note that these are matter to change with time being so we should get back to the Collector to verify the current state.

cc: @open-telemetry/semconv-k8s-approvers

Describe alternatives you've considered

No response

Additional context

Below there are some metrics from namespaces other than k8s.* as well. I leave them in there intentionally in order to take them into account accordingly.

kubeletstats metrics

cpu metrics

k8s.node.cpu.usage
k8s.node.cpu.utilization
k8s.node.cpu.time
k8s.pod.cpu.usage
k8s.pod.cpu.utilization: Deprecated
k8s.pod.cpu.time
k8s.pod.cpu.node.utilization
k8s.pod.cpu_limit_utilization
k8s.pod.cpu_request_utilization
container.cpu.usage: #1128
container.cpu.utilization: Deprecated
container.cpu.time#282
k8s.container.cpu_limit_utilization
k8s.container.cpu_request_utilization
k8s.container.cpu.node.utilization
k8s.container.memory.node.utilization

memory metrics

k8s.node.memory.available
k8s.node.memory.usage
k8s.node.memory.rss
k8s.node.memory.working_set
k8s.node.memory.page_faults
k8s.node.memory.major_page_faults
k8s.pod.memory.available
k8s.pod.memory.usage
k8s.pod.memory.node.utilization
k8s.pod.memory_limit_utilization
k8s.pod.memory_request_utilization
k8s.pod.memory.rss
k8s.pod.memory.working_set
k8s.pod.memory.page_faults
k8s.pod.memory.major_page_faults
container.memory.available
container.memory.usage: ✅ #282
k8s.container.memory_limit_utilization
k8s.container.memory_request_utilization
container.memory.rss
container.memory.working_set
container.memory.page_faults
container.memory.major_page_faults

filesystem metrics

k8s.node.filesystem.available
k8s.node.filesystem.capacity
k8s.node.filesystem.usage
k8s.pod.filesystem.available
k8s.pod.filesystem.capacity
k8s.pod.filesystem.usage
container.filesystem.available
container.filesystem.capacity
container.filesystem.usage

network metrics

k8s.node.network.io
k8s.node.network.errors
k8s.pod.network.io
k8s.pod.network.errors

uptime metrics

k8s.node.uptime
k8s.pod.uptime
container.uptime

volume metrics

k8s.volume.available
k8s.volume.capacity
k8s.volume.inodes
k8s.volume.inodes.free
k8s.volume.inodes.used

k8scluster metrics

container metrics

k8s.container.cpu_request
k8s.container.cpu_limit
k8s.container.memory_request
k8s.container.memory_limit
k8s.container.storage_request
k8s.container.storage_limit
k8s.container.ephemeralstorage_request
k8s.container.ephemeralstorage_limit
k8s.container.restarts
k8s.container.ready

pod metrics

k8s.pod.phase
k8s.pod.status_reason

deployment metrics

k8s.deployment.desired
k8s.deployment.available

cronjob metrics

k8s.cronjob.active_jobs

daemonset metrics

k8s.daemonset.current_scheduled_nodes
k8s.daemonset.desired_scheduled_nodes
k8s.daemonset.misscheduled_nodes
k8s.daemonset.ready_nodes

hpa metrics

k8s.hpa.max_replicas
k8s.hpa.min_replicas
k8s.hpa.current_replicas
k8s.hpa.desired_replicas

job metrics

k8s.job.active_pods
k8s.job.desired_successful_pods
k8s.job.failed_pods
k8s.job.max_parallel_pods
k8s.job.successful_pods

namespace metrics

k8s.namespace.phase

replicaset metrics

k8s.replicaset.desired
k8s.replicaset.available

replication_controller metrics

k8s.replication_controller.desired
k8s.replication_controller.available

resource_quota metrics

k8s.resource_quota.hard_limit
k8s.resource_quota.used

statefulset metrics

k8s.statefulset.desired_pods
k8s.statefulset.ready_pods
k8s.statefulset.current_pods
k8s.statefulset.updated_pods

Openshift metrics

openshift.clusterquota.limit
openshift.clusterquota.used
openshift.appliedclusterquota.limit
openshift.appliedclusterquota.used

node metrics

k8s.node.condition

Related issues

TBA

@ChrsMark ChrsMark added enhancement New feature or request experts needed This issue or pull request is outside an area where general approvers feel they can approve triage:needs-triage labels May 13, 2024
@TylerHelmuth
Copy link
Member

I love the idea of moving forward with this work. According to the collector end-user survey k8s and the collector are a big part of our end-user's stack, so moving the related semconvs forwards is a great idea.

@sirianni
Copy link

In general, my team has been happy with the metrics collected by kubeletstatsreceiver and how they are modeled. They are struggling significantly with the "state" metrics that come from k8sclusterreceiver. We are coming from a Datadog background.

codeboten pushed a commit to open-telemetry/opentelemetry-collector-contrib that referenced this issue Jun 18, 2024
#33598)

Having recently been working with the `kubeletstats` receiver (using it
and contributing to it), I would like to volunteer to help with its
maintainance by intending to dedicate time to contribute to the
component as well as help with the existing and future issue queue.

Also being a member of the
[semconv-k8s-approvers](https://github.com/orgs/open-telemetry/teams/semconv-k8s-approvers)
and
[semconv-container-approvers](https://github.com/orgs/open-telemetry/teams/semconv-container-approvers)
will help to bring more alignment between the [Semantic
Conventions](open-telemetry/semantic-conventions#1032)
and the Collector's implementation within this specific scope.

-  ✅ Being a member of Opentelemetry organization 
- PRs authored:
https://github.com/open-telemetry/opentelemetry-collector-contrib/pulls?q=is%3Apr+author%3AChrsMark++label%3Areceiver%2Fkubeletstats%2Cinternal%2Fkubeletstats%2Cinternal%2Fkubelet
- Issues have been involved:
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aissue+commenter%3AChrsMark+label%3Areceiver%2Fkubeletstats%2Cinternal%2Fkubeletstats+

/cc @dmitryax @TylerHelmuth with whom I have already discussed about it

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
cparkins pushed a commit to AmadeusITGroup/opentelemetry-collector-contrib that referenced this issue Jul 11, 2024
open-telemetry#33598)

Having recently been working with the `kubeletstats` receiver (using it
and contributing to it), I would like to volunteer to help with its
maintainance by intending to dedicate time to contribute to the
component as well as help with the existing and future issue queue.

Also being a member of the
[semconv-k8s-approvers](https://github.com/orgs/open-telemetry/teams/semconv-k8s-approvers)
and
[semconv-container-approvers](https://github.com/orgs/open-telemetry/teams/semconv-container-approvers)
will help to bring more alignment between the [Semantic
Conventions](open-telemetry/semantic-conventions#1032)
and the Collector's implementation within this specific scope.

-  ✅ Being a member of Opentelemetry organization 
- PRs authored:
https://github.com/open-telemetry/opentelemetry-collector-contrib/pulls?q=is%3Apr+author%3AChrsMark++label%3Areceiver%2Fkubeletstats%2Cinternal%2Fkubeletstats%2Cinternal%2Fkubelet
- Issues have been involved:
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aissue+commenter%3AChrsMark+label%3Areceiver%2Fkubeletstats%2Cinternal%2Fkubeletstats+

/cc @dmitryax @TylerHelmuth with whom I have already discussed about it

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark
Copy link
Member Author

I have updated the description to group metrics together in a meaningful way.
For kubeletstats the grouping is per resource type (cpu, memory, network etc).
For k8sclusterrecevier metrics the grouping is per K8s Resource type (pod, deployment etc).

I hope this makes the list less overwhelming and people willing to help on this could pick up a group all together and work on it. Maybe we could create standalone issues per group if that helps, link them here to simplify the list in this issue's description and use this issue as a meta issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:k8s enhancement New feature or request experts needed This issue or pull request is outside an area where general approvers feel they can approve
Development

No branches or pull requests

5 participants