Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
estimated histgoram are accumulate twice #3827
Commit fc4416a change the get_histogram method to return an accumulative histogram instead of a regular histogram.
prometheus assumes that the return histogram is regular one and accumulate the result also,
I don't understand how this is the case. Before that patch, the histograms were verified to be bogus. Can you show me an example of the histograms being bad now ?
Also this link : https://prometheus.io/docs/concepts/metric_types/#histogram explicitly says that the buckets are expected to be cumulative: (see first bullet)
Prometheus expect it to be accumulative, but that is taking care of on the Prometheus level.
Point your browser to:
(for example http://localhost:9180/metrics)
and look for scylla_storage_proxy_coordinator_write_latency_bucket for example
see that the total number is less than the value of the last bucket.
The code that does the accumulation is found at:
The issue was only detected now because Prometheus 2.x uses only the text representation, and there is a difference in the implementation between the protobuf and the text.