Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PrometheusPushGateway cuts off current valid metrics #912

Closed
vkorchik opened this issue Dec 3, 2020 · 5 comments
Closed

PrometheusPushGateway cuts off current valid metrics #912

vkorchik opened this issue Dec 3, 2020 · 5 comments

Comments

@vkorchik
Copy link

vkorchik commented Dec 3, 2020

Hi,
I am using Kamon version 2.1.9

I have found it very strange but PrometheusPushGatewayReporter is created with a hardcoded stale period of 365 days, and this forces all my current metrics to be cleaned here. Here we have from debug:

stalePeriod = PT8760H // which is 365 days
currentInstant = 2024-12-18T00:00:00Z
cutoff = 2023-12-19T00:00:00Z

So here i will give you my config file, and the way i emit metrics.

// application.conf

kamon {
  prometheus {
    start-embedded-http-server = no
    pushgateway.api-url = "http://localhost:9091/metrics/job/example"
  }

  modules {
    prometheus-reporter.enabled = false
    pushgateway-reporter.enabled = true
  }
}


// Main.scala

Kamon.init()
Kamon.counter("example-counter-1").withoutTags().increment(1)
Kamon.counter("example-counter-1").withoutTags().increment(1)

PS:
The usual PrometheusReporter works well, since stale period there is 5 years as well as period, so cutoff date is basically the now date.

@muej
Copy link

muej commented Feb 22, 2021

I faced this issue too and have tracked it down to be an issue in the PeriodSnapshot.Accumulator and I guess this change was not applied here PrometheusPushgatewayReporter.scala#L34.

@muej
Copy link

muej commented Feb 22, 2021

If it is of any help, my configuration can be found on gitter

@muej
Copy link

muej commented Feb 22, 2021

#822 seems to be related.

@vkorchik
Copy link
Author

@muej, yes you are right, the problem is exactly in instantiating the snapshot accumulator.
But PR is there for more than 6 months, hope it will be merged anytime soon.

@SimunKaracic
Copy link
Contributor

Fixed in version 2.1.14!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants