Export Kubernetes Labels with Pods

[alex-g-tejada]

Hey!

We're using the helm chart within our kubernetes cluster. We're able to collect exported pod/namespace but labels belonging to the pods are not being collected. Is there a way to collect these as well?

[nikkon-dev]

@alex-g-tejada,

Currently, dcgm-exporter does not provide a way to output all pod labels to Prometheus.
This is not hard to add, but printing out all labels would overwhelm the output.
There should be some way to specify which pod labels are needed.
This may be a reasonable feature request.

[syandroo]

@nikkon-dev is there any way to specify adding replicaset / deployment to the output too? or would that be in line with the pod annotations request

[nikkon-dev]

@syandroo,
That would be a bit different as the ReplicaSet or Deployment names are not labels. But what information would you want to see there? Just names of a replicaset/deployment?

[syandroo]

@nikkon-dev either the names of the replicaset or deployment are good, but being able to export pod labels would be sufficient, as we have them labeled anyway

[thekuffs]

Hi, you don't want to do this as part of the dcgm exporter. You should make sure that these pods are collected with a kubernetes service discovery style job in prometheus. (https://prometheus.io/docs/prometheus/latest/configuration/configuration/#pod) You can then use a labelmap to assign pod labels as labels in prometheus. There's a perfectly good example at https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus-kubernetes.yml#L288.