Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional Zero Threshold for Exponential Histograms to the metrics data model #2665

Merged
merged 3 commits into from Aug 4, 2022

Conversation

gouthamve
Copy link
Member

@gouthamve gouthamve commented Jul 15, 2022

This is a follow up of #2611 and the Prometheus WG call from June 22, 2022.

Changes

This discusses the zero_threshold field that exists and is used in Prometheus Exponential Histograms and it would be nice to include it in OTel Exponential Histograms as well.

You can see the reference implementation here:
https://github.com/prometheus/prometheus/blob/5937b4f5d4f658758f4aa6b9ab803f022e6e5497/model/histogram/float_histogram.go#L186-L197

https://github.com/prometheus/prometheus/blob/5937b4f5d4f658758f4aa6b9ab803f022e6e5497/model/histogram/float_histogram.go#L770-L790

Prometheus also uses it (increases zero_threshold) to limit the number of buckets: https://github.com/prometheus/client_golang/blob/5a321c7b05264e54d2dbc86f12d9b80d4e7a492b/prometheus/histogram.go#L773-L808

DDSketch from Datadog uses something similar to limit the number of buckets (they expand the lowest bucket, not zero bucket but same principle).

Related issues #2611

specification/metrics/data-model.md Outdated Show resolved Hide resolved
specification/metrics/data-model.md Outdated Show resolved Hide resolved
specification/metrics/data-model.md Outdated Show resolved Hide resolved
This is a field that exists and is used in Prometheus Exponential
Histograms and it would be nice to include it in OTel Exponential
Histograms as well.

You can see the reference implementation here:
https://github.com/prometheus/prometheus/blob/5937b4f5d4f658758f4aa6b9ab803f022e6e5497/model/histogram/float_histogram.go#L186-L197
https://github.com/prometheus/prometheus/blob/5937b4f5d4f658758f4aa6b9ab803f022e6e5497/model/histogram/float_histogram.go#L770-L790

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Copy link
Contributor

@jsuereth jsuereth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming this supports Prometheus compatibility (and I'd love to have a more official specification to ensure that), this looks good to me as an optional extension.

Copy link
Member

@reyang reyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is really important to avoid the "taint mode" 👍

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@jmacd jmacd changed the title Add optional Zero Threshold for Exponential Histograms Add optional Zero Threshold for Exponential Histograms to the metrics data model Aug 4, 2022
@jmacd
Copy link
Contributor

jmacd commented Aug 4, 2022

@gouthamve I changed the PR title to limit its scope to match what's currently reviewed. I think after this merges, we're free to add it in the protocol for OTLP v0.20. Separately, we'll update ./sdk.md to say that expo-histos will accept an option and will have a default behavior w.r.t. this new thing--this we can debate later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants