[feature] Introduce metrics aggregation by labelNames #825
Comments
|
@tjiuming For the group by label name / Example JUnit test / package io.prometheus.client;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.util.ArrayList;
import java.util.List;
public class CounterGroupByCollectorTest {
CollectorRegistry registry;
Counter counter;
CounterGroupByCollector counterGroupByCollector;
@Before
public void setUp() {
registry = new CollectorRegistry();
counter = Counter.build("metrics_name", "metrics_name help").labelNames("cluster", "namespace", "topic").create();
counterGroupByCollector = new CounterGroupByCollector(counter);
}
@Test
public void test() {
counter.labels("a1", "b1", "c1").inc();
counter.labels("a1", "b1", "c2").inc();
counter.labels("a1", "b2", "c3").inc();
counter.labels("a1", "b2", "c4").inc();
System.out.println("Group by \"cluster\"...");
counterGroupByCollector.groupBy("cluster");
List<Collector.MetricFamilySamples> mfs = counterGroupByCollector.collect();
for (Collector.MetricFamilySamples samples : mfs) {
for (Collector.MetricFamilySamples.Sample sample : samples.samples) {
System.out.println(String.format("sample [%s]", sample));
}
}
System.out.println("---");
System.out.println("No group by...");
counterGroupByCollector.groupBy(null);
mfs = counterGroupByCollector.collect();
for (Collector.MetricFamilySamples samples : mfs) {
for (Collector.MetricFamilySamples.Sample sample : samples.samples) {
System.out.println(String.format("sample [%s]", sample));
}
}
System.out.println("---");
System.out.println("Group by \"cluster\", \"namespace\"...");
counterGroupByCollector.groupBy("cluster", "namespace");
mfs = counterGroupByCollector.collect();
for (Collector.MetricFamilySamples samples : mfs) {
for (Collector.MetricFamilySamples.Sample sample : samples.samples) {
System.out.println(String.format("sample [%s]", sample));
}
}
System.out.println("---");
System.out.println("Group by \"cluster\", \"namespace\", \"topic\"...");
counterGroupByCollector.groupBy("cluster", "namespace", "topic");
mfs = counterGroupByCollector.collect();
for (Collector.MetricFamilySamples samples : mfs) {
for (Collector.MetricFamilySamples.Sample sample : samples.samples) {
System.out.println(String.format("sample [%s]", sample));
}
}
}
public static class CounterGroupByCollector extends Collector {
private Counter counter;
private String[] groupByLabelNames;
public CounterGroupByCollector(Counter counter) {
this.counter = counter;
}
public void groupBy(String ... labelNames) {
if ((labelNames == null) || (labelNames.length == 0)) {
synchronized (this) {
groupByLabelNames = null;
}
return;
}
if (labelNames.length > counter.labelNames.size()) {
throw new IllegalArgumentException("Group by labels name contains more labels than Counter");
}
List<String> labelNameList = toList(labelNames);
List<String> counterLabelNameList = counter.labelNames;
for (int i = 0; i < labelNameList.size(); i++) {
if (!labelNameList.get(i).equals(counterLabelNameList.get(i))) {
throw new IllegalArgumentException("Group by labels names are not a subset of Counter label names");
}
}
synchronized (this) {
this.groupByLabelNames = labelNames;
}
}
@Override
public List<MetricFamilySamples> collect() {
String[] localGroupByLabelNames;
synchronized (this) {
localGroupByLabelNames = groupByLabelNames;
}
if (localGroupByLabelNames == null) {
return counter.collect();
}
Counter localCounter =
Counter
.build("metrics_name", "metrics_name help")
.labelNames(localGroupByLabelNames).create();
List<Collector.MetricFamilySamples> mfs = counter.collect();
for (Collector.MetricFamilySamples samples : mfs) {
for (Collector.MetricFamilySamples.Sample sample : samples.samples) {
if (sample.name.endsWith("_total")) {
String[] labelValues = sample.labelValues.subList(0, localGroupByLabelNames.length).toArray(new String[localGroupByLabelNames.length]);
localCounter.labels(labelValues).inc(sample.value);
}
}
}
return localCounter.collect();
}
}
private static List<String> toList(String ... values) {
List<String> list = new ArrayList<String>(values.length);
for (String value : values) {
list.add(value);
}
return list;
}
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, in Prometheus client, we don't have metrics aggregations, the exposed metrics data is the origin data.
For a example:
the exposed metrics as below:
But in some conditions, we want to expose the metrics in custom levels. Say, expose metrics data in
clusterlevel as below:or in [
cluster,namespace] level as below:If this request is feasible, it will greatly reduce the pressure on the Prometheus Server side. And also benefits to the Client side, because it can reduce the size of the response body.
Implementation
Aggregator
We can introduce 2 aggregators,
SUMandAVG.For COUNTER/GAUGE/HISTOGRAM, we can apply the
SUMaggregator to them, for SUMMARY, we can applyAVGandSUMaggregators to it.Gauge:
The origin data:
Aggregate in
clusterlevel:Aggregate in [
cluster,namespace] level:Counter:
The origin data:
Aggregate in
clusterlevel:Aggregate in [
cluster,namespace] level:Histogram:
The origin data:
Aggregate in
clusterlevel:Aggregate in [
cluster,namespace] level:Summary
Unlike the above meters, SUMMARY is special.
For
metrics_name_countandmetrics_name_sum, we have to use theSUMaggregator.But for the timeseries with
quantilelabel, I thinkAVGaggregator is the best choice.The text was updated successfully, but these errors were encountered: