Why java client not support metrics with custom timestamp?

[xySandra]

On some batch job scenes, we want to use histogram track data distribution and set the timestamp with hour level for monitoring, but there no function available. Golang provide the NewMetricWithTimestamp function , I want to know is there have a plan support java and when the feature can be released ? Thanks a lot !

[fstab]

I'm not sure why you need a custom timestamp. What's missing if you just observe the data whenever the batch job runs?

[erks]

One use case would be to mirror metrics with given timestamps from other metric sources, which is the exact use case the Golang client's NewMetricWithTimestamp is for.

This is only useful in rare cases as the timestamp of a Prometheus metric should usually be set by the Prometheus server during scraping. Exceptions include mirroring metrics with given timestamps from other metric sources.

[fstab]

If your goal is to forward a metric from another metric source, you'll need to implement a Collector that collects the metrics from that metric source and returns it as a List<MetricFamilySamples>. You can add custom timestamps there. Something like this:

public class MetricWithTimestamp extends Collector {

    private double valueFromOtherMetricSource = 0; // get this from the other metric source
    private long timestamp = 0; // get this from the other metric source

    @Override
    public List<MetricFamilySamples> collect() {
        List<MetricFamilySamples> mfs = new ArrayList<MetricFamilySamples>();
        List<String> labelNames = Arrays.asList("label1", "label2");
        List<String> labelValues = Arrays.asList("value1", "value2");
        CounterMetricFamily counters = new CounterMetricFamily("my_counter", "Help text", labelNames);
        counters.samples.add(new MetricFamilySamples.Sample("my_counter_total", labelNames, labelValues, valueFromOtherMetricSource, timestamp));
        mfs.add(counters);
        return mfs;
    }
}

Does that work for you? (I didn't run the code, but I hope you get the idea).

[helloworlde]

If your goal is to forward a metric from another metric source, you'll need to implement a Collector that collects the metrics from that metric source and returns it as a List<MetricFamilySamples>. You can add custom timestamps there. Something like this:

public class MetricWithTimestamp extends Collector {

    private double valueFromOtherMetricSource = 0; // get this from the other metric source
    private long timestamp = 0; // get this from the other metric source

    @Override
    public List<MetricFamilySamples> collect() {
        List<MetricFamilySamples> mfs = new ArrayList<MetricFamilySamples>();
        List<String> labelNames = Arrays.asList("label1", "label2");
        List<String> labelValues = Arrays.asList("value1", "value2");
        CounterMetricFamily counters = new CounterMetricFamily("my_counter", "Help text", labelNames);
        counters.samples.add(new MetricFamilySamples.Sample("my_counter_total", labelNames, labelValues, valueFromOtherMetricSource, timestamp));
        mfs.add(counters);
        return mfs;
    }
}

Does that work for you? (I didn't run the code, but I hope you get the idea).

It could work after consummate.

And if it could be support in Go, why not keep consistent in Java? @fstab

[mateusduboli]

I'm interested in that as well.

I have a system which reports metrics every X seconds for my monitoring code, it is a proprietary database so I don't have access to the code, but I can assign a callback to collect the report.

The way in which we export the metrics is using a text-file, where use the TextFormat to create the metrics and then write them to a file, which will be later read by a node-exporter.

This database performs regular tasks, like reindexing, cleaning, compact information, and we need accurate readings about the instances of those jobs, to be able to correlate them with possible misbehaviors that we might face.

Since we use text-file exporters, we have to overwrite the text file after every report, meaning if we lose a scrap collection that information is lost, and we will have gaps between readings.

I have another use case for the same database when performing other batch operations like, backups and restores, in my scenario those tools use the same codebase and can leverage the fact that monitoring is done in a similar way.

For me is also confusing why the golang interface allows for timestamps and the java doesn't, I understand that it was targeted towards online systems and not batch processing, however monitoring internal tasks in online systems is a valid use case IMHO and can leverage a lot from this feature.

[maksumant]

Absolutely, not sure why java client doesn't have support to add custom timestamps? It is useful in mirroring metrics with given timestamps from other metric sources. Do we have plan to add this support?

[fstab]

Do we have plan to add this support?

Yes.

The upcoming 1.0.0 release (pre-releases are available, see Github releases) ships with a completely new implementation of the underlying data model. It's implemented in the prometheus-metrics-model module (see 1.0.x feature branch, it will be merged to main in the next couple of days).

The data model implements read-only snapshots that are produced when the Prometheus server scrapes. All data point snapshots builders inherit the scrapeTimestampMillis() method that you can use to set the scrape timestamp.

So what you can do is implement your own Collector that produces snapshots with the scrape timestamp set.

Implementing custom collectors is not a common case, but when your use case is to bridge existing metrics to a PrometheusRegistry it might be a good option. Please let me know if that helps in your case.

Btw: The 1.0.0 release is only a couple of days away. There are already some docs here: https://prometheus.github.io/client_java/

[fstab]

Btw if you are looking for examples of how to create custom snapshots, the ExpositionFormatsTest is a great source. Here's an example of creating a counter snapshot with an explicit scrapeTimestampMillis.

        CounterSnapshot counter = CounterSnapshot.builder()
                .name("service_time_seconds")
                .help("total time spent serving")
                .unit(Unit.SECONDS)
                .dataPoint(CounterDataPointSnapshot.builder()
                        .value(0.8)
                        .labels(Labels.builder()
                                .label("path", "/hello")
                                .label("status", "200")
                                .build())
                        .exemplar(exemplar1)
                        .createdTimestampMillis(createdTimestamp1)
                        .scrapeTimestampMillis(scrapeTimestamp1)
                        .build())
                .dataPoint(CounterDataPointSnapshot.builder()
                        .value(0.9)
                        .labels(Labels.builder()
                                .label("path", "/hello")
                                .label("status", "500")
                                .build())
                        .exemplar(exemplar2)
                        .createdTimestampMillis(createdTimestamp2)
                        .scrapeTimestampMillis(scrapeTimestamp2)
                        .build())
                .build();

[maksumant]

So what you can do is implement your own Collector that produces snapshots with the scrape timestamp set.

Thanks, this really helps. I was hoping we will be able to use existing Collectors (like Counter) with ability to attach custom timestamps instead of implementing our own Collector.