incorrect query for "usage per core" on nodes dashboard

[woodsaj]

kubernetes-mixin/dashboards/node.libsonnet

Line 33 in d0209e8

.addTarget(prometheus.target('avg by (cpu) (irate(node_cpu_seconds_total{%(nodeExporterSelector)s, mode!="idle", instance="$instance"}[5m])) * 100' % $._config, legendFormat='{{cpu}}'));

the query is currently showing the average of iowait, irq, nice, softirq, steal, system and user instead of the sum.

Also the y-axis is set to "percent 0.0-1.0", but the datapoints are "0-100"

[metalmatze]

I'm not entirely sure what the best aggregate with this is. Comparing the graphs with sum and avg I can't see it making a big difference. Only the scale seems to vary quite a bit, with the same spike going to 0.42 with sum and 0.058 with avg.

@tomwilkie do you have any insights into this one? You originally worked on this Node dashboard.

[woodsaj]

A cpu core is being "used" if it is not in an idle state. The only way to calculate the percentage of time the core is being used is to sum the amount of time spent in all other CPU states.

If a single process thread is using 100% of a CPU
You would have the following datapoints

CPU State	rate
user	1.0
system	0
iowait	0
nice	0
irq	0
softirq	0
steal	0

If you used 'avg', the CPU utilization would be incorrectly reported as 14.28%. (100 + 0 + 0 + 0 + 0 + 0 + 0) / 7

[metalmatze]

That makes a lot of sense. I will wait for some more comments and then go ahead with merging #132

[cbeneke]

Hey, just saying, that exact PR is open since about a month.... #118

[metalmatze]

Whoops. Sorry about that. We'll merge #118 then.
Thanks everybody for raising the issue!

[tomwilkie]

Closing as fixed by #118