Adapt otel exponential histogram data

[mufiye]

I find the OpenTelemetryMetricRequestProcessor#adaptMetrics in otel-receiver-plugin can not process exponential histogram metric. But some service will use this exponentialHistogram instead of histogram metric, such as statsd receiver in opentelemetry collector. So I do this pull request to adapt this case.
The related discussion about this pull request is in issue #10341.

• If this is non-trivial feature, paste the links/URLs to the design doc.

• Update the documentation to include this new feature.

• Tests(including UT, IT, E2E) are added to verify the new feature.

• If it's UI related, attach the screenshots below.

• If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

• Update the CHANGES log.

[wu-sheng]

Please build a test case for the convert, as otel exponential histogram is pretty new for many people.

[mufiye]

Please build a test case for the convert, as otel exponential histogram is pretty new for many people.

Ok, I get it. I will add the test case into the test dir. I think I need some time to learn how to build my test case because it is different from the plugin-test which I done before.

[wu-sheng]

We just need a unit test. What is in the test folder(root) is e2e testing.
Eventually, we will need that for airflow monitoring, but it doesn't check data correction, mostly just prove data flow.
But this one is very detailed data, so a JUnit based UT is enough.

[wu-sheng]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double? Because when we calculate percentile from the histogram (usually we do today), the values are bucket boundaries and they should be integer/long only.

[mufiye]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double?

This histogram always builds bucket boundaries as double. I don't fully understand what you mean.

[wu-sheng]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double?

This histogram always builds bucket boundaries as double. I don't fully understand what you mean.

Check hisgram percentile function, which calculate the value. The percentile value is the rank of the bucket.
Read the calculation will help you to understand the process.

[wu-sheng]

Check how SampleFamily#histogram_percentile works(Ref Analyzer#init, createMetric).

[mufiye]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double? Because when we calculate percentile from the histogram (usually we do today), the values are bucket boundaries and they should be integer/long only.

I have understood this question. I think the small double case maybe happen. But this case also maybe happen for otel histogram data.

[wu-sheng]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double? Because when we calculate percentile from the histogram (usually we do today), the values are bucket boundaries and they should be integer/long only.

I have understood this question. I think the small double case maybe happen. But this case also maybe happen for otel histogram data.

If current, the buckets of airflow definition are good, we should be fine. Otherwise, we may need some closure to process the buckets, such as maybe, we need multiple 1000 from microsecond to millisecond precision.

[mufiye]

@mufiye One question, does this histogram always build bucket boundaries as integers? Could it be a very small double? Because when we calculate percentile from the histogram (usually we do today), the values are bucket boundaries and they should be integer/long only.

I have understood this question. I think the small double case maybe happen. But this case also maybe happen for otel histogram data.

If current, the buckets of airflow definition are good, we should be fine. Otherwise, we may need some closure to process the buckets, such as maybe, we need multiple 1000 from microsecond to millisecond precision.

I review the data collected from airflow and find there are some small timer data, such as 1.182709 and 1.348083 in one exponentialHistogram data. Where should I add the closure?

[wu-sheng]

I review the data collected from airflow and find there are some small timer data, such as 1.182709 and 1.348083 in one exponentialHistogram data. Where should I add the closure?

When you write MAL.

[mufiye]

I review the data collected from airflow and find there are some small timer data, such as 1.182709 and 1.348083 in one exponentialHistogram data. Where should I add the closure?

When you write MAL.

I read other services' otel rule cases and meter analysis language but have no idea how to do this. Could you explain it more?

[wu-sheng]

I review the data collected from airflow and find there are some small timer data, such as 1.182709 and 1.348083 in one exponentialHistogram data. Where should I add the closure?

When you write MAL.

I read other services' otel rule cases and meter analysis language but have no idea how to do this. Could you explain it more?

There is no other for these. Because they don't use small doubles as buckets.

If you want to learn this for now, then MAL is groovy based script. SampleFamily is the root class running in MAL engine, and this is the type referred by the metric name of the expression.
You can see SampleFamily#filter(Closure<Boolean> filter), this is what I mean about closure. It means a code block as an inner class for the metric process.
We don't have a similar mechanism for this, but from what you described these buckets, we seem to need one for the exponential histogram. For example, we may need buildBucketsFromExponentialHistogram(..., Closure<Boolean> processor) as another new method. Then we could write some codes in the MAL expression to do mutiple 1000(ns -> ms) mentioned above.

[mufiye]

If you want to learn this for now, then MAL is groovy based script. SampleFamily is the root class running in MAL engine, and this is the type referred by the metric name of the expression. You can see SampleFamily#filter(Closure<Boolean> filter), this is what I mean about closure. It means a code block as an inner class for the metric process. We don't have a similar mechanism for this, but from what you described these buckets, we seem to need one for the exponential histogram. For example, we may need buildBucketsFromExponentialHistogram(..., Closure<Boolean> processor) as another new method. Then we could write some codes in the MAL expression to do mutiple 1000(ns -> ms) mentioned above.

I think I need to fully understand the meter analyzer code first if I want to add this function. In my current opinion, the function can not be put into the OpenTelemetryMetricRequestProcessor#adaptMetrics part and should be put in the analyzer part. And should we also consider small double data case for "otel histogram data"? Anyway, the difference between histogram and ExponentialHistogram is only that the latter does the data compression.

[wu-sheng]

the function can not be put into the OpenTelemetryMetricRequestProcessor#adaptMetrics part and should be put in the analyzer part. And should we also consider small double data case for "otel histogram data"?

Yes, it should be on the MAL(SampleFamily) part.

[wu-sheng]

LGTM @kezhenxu94 @wankai123 Please take another look.

[kezhenxu94]

looks good