Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when a counter has an starting value of integer and then a float is pushed #2

Closed
slok opened this issue Oct 17, 2023 · 3 comments

Comments

@slok
Copy link

slok commented Oct 17, 2023

Hi!

First of all thanks for this project, specially gravel-gateway. I think that currently is the most usable/helpful Prometheus gateway out there.

Context

I encontered this problem while using gravel-gateway. However seems that the problem is related with the usage of this library, so I open the issue here, if you prefer to have the issue on the other repo, I will move it happily.

Problem

The problem itself is when I push a counter metric and randomly crashes the gateway with this error:

thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/openmetrics-parser-0.4.2/src/public/model.rs:900:50

Findings

After reading some code (I don't know Rust) and making some tests I found that this happens when a counter has an integer value. E.g:

example_test_counter_total{foo="bar"} 0

And then a float is pushed. e.g:

example_test_counter_total{foo="bar"} 0.01

After this second push, the gateway crashes, but, if the metric starts with a float from the begining, there are no crashes, however is very common to initialize a metric to 0, leading to this use case.

Extra info

More information that could help:

  • Gravel gateway version: v1.6.2
  • Prometheus client: github.com/prometheus/client_golang v1.17.0
  • Using the official process collector leads to this crashes on process_cpu_seconds_total counter metric.
@sinkingpoint
Copy link
Owner

Interesting! I'll take a look

@sinkingpoint
Copy link
Owner

This is fixed - will get a gravel gateway out with this soon

@slok
Copy link
Author

slok commented Nov 7, 2023

Many thanks!!

sinkingpoint added a commit to sinkingpoint/prometheus-gravel-gateway that referenced this issue Nov 17, 2023
This brings in the fix for sinkingpoint/openmetrics-parser#2

Signed-off-by: sinkingpoint <colin@quirl.co.nz>
sinkingpoint added a commit to sinkingpoint/prometheus-gravel-gateway that referenced this issue Nov 17, 2023
This is a bit of a bug fix release that solves:

- #27
- #29
- sinkingpoint/openmetrics-parser#2

Signed-off-by: sinkingpoint <colin@quirl.co.nz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants