Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent duplicate Prometheus description comments #2890

Merged
merged 3 commits into from Oct 21, 2022

Conversation

dashpole
Copy link
Contributor

@dashpole dashpole commented Oct 18, 2022

Fixes #2869

Follow-up to #2703. That PR prevents collisions between data points, but not collisions between the descriptions of the metrics.

OpenMetrics requirements

From https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#metricfamily:

A MetricFamily MUST have a name, HELP, TYPE, and UNIT metadata.
MetricFamily names are a string and MUST be unique within a MetricSet.

Basically, for a given metric name, we MUST have exactly one HELP, TYPE, and UNIT comment.

Changes

This is a bit light on details, but I can add more if people think that would be helpful.

Basically, this is saying that exporters need to deduplicate Prometheus "descriptions" when sending. For TYPE conflicts, some metric has to be dropped, as the data sent might not be valid (e.g. you can't send counter information with a histogram TYPE). For UNIT and HELP comments, it is up to the exporter how to merge, but the exporter MUST NOT keep both comments.

Example approach

  • Keep a map of metric name -> Description when constructing a batch of metrics for a scrape.
  • Each time a new metric (with Description) is added, look up the name in the map:
    • If it isn't found, add it to the map
    • If it is found, and the desc is the same, do nothing
    • If it is found, and has conflicting type information, drop the metric + log a warning
    • If it is found, and has conflicting help or unit, use the stored help + unit instead of the provided one + log a warning.

@dashpole dashpole requested review from a team as code owners October 18, 2022 15:45
@dashpole dashpole changed the title Prevent duplicate Prometheus comments Prevent duplicate Prometheus description comments Oct 18, 2022
@dashpole
Copy link
Contributor Author

cc @jack-berg @reyang @open-telemetry/wg-prometheus

specification/metrics/data-model.md Outdated Show resolved Hide resolved
specification/metrics/data-model.md Outdated Show resolved Hide resolved
Copy link
Member

@reyang reyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reyang
Copy link
Member

reyang commented Oct 21, 2022

@dashpole would you update the changelog?

@dashpole
Copy link
Contributor Author

@dashpole would you update the changelog?

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle conflicting metric descriptions and types in prometheus exporter
7 participants