Add functions for extracting count and sum metrics from Histograms

[swiatekm]

Component(s)

processor/transform

Is your feature request related to a problem? Please describe.

I have a histogram, but I'm only interested in the sum and count values. I'd like to extract them as individual metrics and drop the rest of the histogram.

Describe the solution you'd like

I propose that we add convert_histogram_count_val_to_sum and convert_histogram_count_val_to_count functions and deprecate the existing convert_summry_count_val_to_sum and convert_summary_count_val_to_count functions. The new functions would work for Summaries, Histograms and ExponentialHistograms.

My reasoning is that this operation - extracting the count or sum value - is logically identical for all of these metric types, and the implementation is very similar as well. I used histogram in the function name because this obviously covers Histograms and ExponentialHistograms, while Summaries are effectively a legacy metric type, so they should be given less weight here.

In terms of implementation, the new functions would be very similar to the ones for summary, with some additional logic for copying between the different DataPoint types.

Describe alternatives you've considered

I also considered deleting all the buckets from the histogram, but this is both incompatible with the spec and currently impossible using OTTL. Extracting the metrics via an OTTL function feels like the most idiomatic solution.

Additional context

Prometheus handles this use case correctly if we tell it to only scrape the sum and count parts of a histogram, so this would improve our interoperability.

[github-actions]

Pinging code owners:

• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

I'm not the best at metrics, but Histograms are essentially Summaries with extra meaning right? The logic for converting a Summary to a Sum/Count would be the same for a Histogram?

[swiatekm]

Yeah, I believe the code would be almost exactly the same, and it should be easy to have a generic function covering all of these, if that's what we want. Fundamentally what we're doing is just extracting a part of the histogram/summary datapoint into a new metric.

I think this might even be doable with just the metrics transform processor and OTTL as-is, but there's no way to iterate over the datapoints.

[TylerHelmuth]

I would definitely support merging functions if we can do it in a meaningful way, even if it'd be a breaking change. Based on the existing transformprocessor functions, can you propose how we could handle the use cases for Summary/Histogram/ExpoHistogram (if applicable) in a *to_sum and *to_count functions?

[swiatekm]

Sure, I'll verify that a generic function will work the way I expect and then submit a concrete proposal for the change.

[swiatekm]

I've updated the description with the proposal for the change, and also linked a draft PR adding the sum conversion function. @TylerHelmuth can you have a look and let me know if that PR looks about right?

[swiatekm]

See also the count implementation: swiatekm#2449. It uses some machinery from the sum PR, so I'd like to get that merged first. I can also submit both at once, but that doesn't look easier to review.

[swiatekm]

In the course of discussing #24368, we ended up with the following:

• The function is named extract_sum_metric
• It doesn't take an aggregation temporality argument, but instead copies it from the source metric. For Summaries, it assumes cumulative, but we plan to add an optional argument to change this.
• When passed a metric of the wrong type, it throws an error instead of ignoring it

[swiatekm]

Resolved in #24368 and #24897