Initial draft of Prometheus <-> OTLP datamodel specification. #2017
Conversation
| - `le` label is used to identify histogram bucket boundaries and counts. | ||
| - `quantile` label is used to identify quantile points in summary metrics. | ||
| - `__name__` is used to identify the metric name of the data point. | ||
| - `__metrics_path__` is ignored in OTLP. |
There was a problem hiding this comment.
Except for __name__ (which is not part of the OpenMetrics protocol, has built-in meaning in the relabeling machinery, and is special in the Prometheus Remote Write protocol), I believe all the other __-prefixed labels are meant for the relabeling machinery to have access to, they're just "not exported".
There was a problem hiding this comment.
The collector seems to use __schema__, but not sure if that's a PRW detail or not.
Co-authored-by: Joshua MacDonald <jmacd@users.noreply.github.com>
| **Status**: [Experimental](../document-status.md) | ||
|
|
||
| This section denotes how to convert from prometheus scraped metrics to the | ||
| OpenTelemtery metric data model and how to create prometheus metrics from |
There was a problem hiding this comment.
| OpenTelemtery metric data model and how to create prometheus metrics from | |
| OpenTelemetry metric data model and how to create Prometheus metrics from |
There was a problem hiding this comment.
In general, shouldn't Prometheus be uppercase throughout?
There was a problem hiding this comment.
Yeah, this was hastily written, will go through and fix.
|
|
||
| Prometheus metric labels are split in OpenTelemetry across Resource attributes | ||
| and Metric data stream attributes. Some labels are used within metric | ||
| families to denote semantics which open-telemetry captures within the structure |
|
|
||
| Prometheus Stateset is dropped (TBD). | ||
|
|
||
| Prometheus Info is dropped (TBD). |
There was a problem hiding this comment.
Considered Prometheus Info are typically represented with a constant 1 I could see this easily mapping into a OTel Gauge. As if I remember correctly they're already exported as Gauges. (So maybe it doesn't need to be specified)
There was a problem hiding this comment.
Ok looking at Java client it used to be a part of Gauge metric family, but now it's been moved to Unknown. So based on our reference either way it should be a gauge.
There was a problem hiding this comment.
We should specify it in some way. I think coming in as a gauge works, we just need a way to do OpenMetrics -> OTLP -> OpenMetrics here.
|
|
||
| OpenTelemetry Sum follows this logic: | ||
|
|
||
| - If the aggregation temporality is cumulative and the sum is monotonic, |
There was a problem hiding this comment.
Should we specify this on the reverse that Prometheus Counter becomes a cumulative monotonic OTLP Sum ?
| that is not explicitly called out as being handled specially will be included | ||
| in the set of attributes for a metric data stream. | ||
|
|
||
| Here is a table of the sett of prometheus labels that are lifted into Resource |
There was a problem hiding this comment.
| Here is a table of the sett of prometheus labels that are lifted into Resource | |
| Here is a table of the set of prometheus labels that are lifted into Resource |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
| Additionally, in Prometheus metric labels must match the following regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Metrics | ||
| from OpenTelemetry with unsupported Attribute names should replace invalid characters with the `_` character. This | ||
| may cause ambiguity in scenarios where mulitple similar-named attributes share invalid characters at the same | ||
| location. This is considered an unsupported case, and is highly unlikely. |
There was a problem hiding this comment.
| location. This is considered an unsupported case, and is highly unlikely. | |
| location. This is considered as an unsupported case, and is highly unlikely. |
| - `__metrics_path__` is ignored in OTLP. | ||
|
|
||
| Additionally, in Prometheus metric labels must match the following regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Metrics | ||
| from OpenTelemetry with unsupported Attribute names should replace invalid characters with the `_` character. This |
There was a problem hiding this comment.
I would suggest that we take a special case here:
if the first character of the label key is a digit ([0-9]), prepend a _ instead of replace it with _
For example:
| OpenTelemetry attribute name | Prometheus label key |
|---|---|
| "9123" | "_9123" |
Rather than:
| OpenTelemetry attribute name | Prometheus label key |
|---|---|
| "9123" | "_123" |
There was a problem hiding this comment.
I've left some suggestions, and I'll write the prototype in OpenTelemetry .NET (e.g. open-telemetry/opentelemetry-dotnet#2578)
|
|
||
| OpenTelemetry Histogram becomes a metric family with the following: | ||
|
|
||
| - A single `{name}_count` metric denoting the count field of the histogram. |
There was a problem hiding this comment.
Prometheus doesn't have a separate notion of "unit", based on the Prometheus Metric and Label naming convention, it seems the general guidance is to append unit as a suffix. For example:
| OpenTelemetry metric name (from instrument/view) | OpenTelemetry metric unit | Prometheus metric name |
|---|---|---|
| http.server.duration | milliseconds | http_server_duration_milliseconds |
| http.server.active_requests | N/A | http_server_active_requests |
There was a problem hiding this comment.
Given that prometheus is moving the direction of OpenMetrics which does support Units as a concept https://github.com/OpenObservability/OpenMetrics we may want to suggest that path, so not changing the metrics name but instead is metadata on the metric.
| OpenTelemetry can assume some shape/structure to metric family groups, any | ||
| metric belonging to a family that is not treated specially should be exported | ||
| as its own metric stream. For example, while histograms are expected to | ||
| live in a metric family with metrics `{name}_sum`, `{name}_count` and `{name}`, |
There was a problem hiding this comment.
According to the Prometheus best practices for Histogram and Summaries, we should include unit:
- If there is no unit (empty string, null string), histogram should use
{name},{name}_countand{name}_sum. - If there is a valid unit, histogram should use
{name}_{unit},{name}_{unit}_countand{name}_{unit}_sum.
|
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
|
isn't this still active? |
|
It is active |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
I discussed this draft PR w @jsuereth @jmacd @Aneurysm9 earlier this week. We're planning to work towards completing this PR in the Prometheus workgroup. |
|
Closing in favor of #2266 |
DO NOT SUBMIT (draft only)
For shared contribution/discussion and improvements.
Updates data model specification to denote prometheus compatibilty.
Pending items:
Related issues:
cc @alolita for help shaping this up.