Support for only selecting certain fields (without using operator) in queries

[a-roberts]

Proposal

If I am retrieving cadvisor metrics for containers, I currently get back lots of fields such as

 {
        "metric": {
          "__name__": "kube_pod_container_resource_limits",
          "container": "runtime",
          "endpoint": "https-main",
          "job": "kube-state-metrics",
          "namespace": "blah",
          "node": "ip-blah.baz.compute.internal",
          "pod": "my-example-64c7fcbd7b-94gzv",
          "prometheus": "openshift-monitoring/k8s",
          "resource": "cpu",
          "service": "kube-state-metrics",
          "uid": "0d892907-2c02-4050-878f-dc3cd76a8baa",
          "unit": "core"
        },

Instead of sending that data over the wire, I propose a way to ask for these fields only.

I know with a query like this:

 query: `sum by (a,b)(rate(c{namespace="${namespace}"}[${timePeriod}]))`,

the result includes a and b, which is great.

But if we are doing a simpler query such as:

kube_pod_container_resource_limits{namespace="${namespace}",resource=~"cpu|memory"}`

we get back all of the fields I mentioned above. In our particular use case, we only care for fields like pod, unit, and __name__.

My proposal is that we can ask for only certain fields through a syntax such as the existing by field.

If I want only pod, unit and __name__, my query would be:

kube_pod_container_resource_limits{namespace="${namespace}",resource=~"cpu|memory"} by pod, unit, __name__

Questions

• Is this already possible?
• Would we want "by" to be the way to do it?
• Would PromQL struggle with __ in name?

[roidelapluie]

What should happen when there are duplicates?

[a-roberts]

What should happen when there are duplicates?

That is a good point (so when the metrics exporter is writing back pod and pod for one particular result) - thinking "out loud" we could combine this with, say, some other identifier (like if pod is a child of some other object).

If there is always a metric field name called name, we could do something like:

kube_pod_container_resource_limits{namespace="${namespace}",resource=~"cpu|memory"} by pod[metricName], unit, __name__.

That could be a confusing syntax though because it looks like pod has an attribute called metricName (as would pod.metricName.). I think this is one of those that requires a little more thought.

The alternative (less work, less awesome) would be to throw an error and then it would be up to the exporter and the caller to query the right things. And as it would be a new function, we wouldn't need to deal with breaking folks just yet as nobody could be using this syntax...some food for thought.