gitlab_ci_pipeline_time_since_last_run_seconds isn't updating properly

[Joyjey]

Hi!
I wanna say that it's awesome thing and I'm very thankful about what you did.
Could you help me to understand the logic of polling metrics from branches about last pipelines?
When exporter is getting information about a current pipeline first time the info looks ok, but if I don't starting new pipelines on a branch again, it won't update time about since last run. What did I do wrong?

My configuration about discover and polling:

  maximum_gitlab_api_requests_per_second: 10

  # Interval in seconds to discover projects
  # from wildcards (optional, default: 1800)
  wildcards_projects_discover_interval_seconds: 1800

  # Interval in seconds to discover refs
  # from projects (optional, default: 300)
  projects_refs_discover_interval_seconds: 300

  # Interval in seconds to poll metrics from
  # discovered project refs (optional, default: 30)
  projects_refs_polling_interval_seconds: 300

  # Sets the parallelism for polling projects from
  # the API (optional, default to available CPUs: runtime.GOMAXPROCS(0))
  polling_workers: 4

This is example of metrics I got in one project.

After I started new pipeline on master branch and it has updated since last run metric for master, but for staging/master value hasn't changed and it shows the same value already 30+ minutes.

It looks like I can get info about this only once. Is it true?

[mvisonneau]

👋 hey @Joyjey, thank you for the very kind feedback 🙇

Indeed, thank you for highlighting this. I believe there was some breaking changes introduced in that regard with the recent refactoring that happened. I have pushed a change which I believe should be fixing it (5a91ba2)

If you are using the docker image, can you try to point it to the latest tag in order to try it out ?

[Joyjey]

I'm using tag 0.3.2 now. Isn't it latest now? Let me tell you a little more info about my environment. I'm watching now for 2 main branches in 50 projects. Can it be problems with rate limit of Gitlab?
When I use log-level debug it shows messages like these:

{"for":108218928,"level":"debug","msg":"throttled GitLab requests","time":"2020-05-28T09:08:28Z"}
{"for":111823464,"level":"debug","msg":"throttled GitLab requests","time":"2020-05-28T09:08:28Z"}
{"for":112809162,"level":"debug","msg":"throttled GitLab requests","time":"2020-05-28T09:08:28Z"}

But I tried to change settings about that in Admin Area -> Settings-> Network -> User and IP Rate Limits and it unfortunately didn't help.

I have pushed a change which I believe should be fixing it (5a91ba2)

Thank you, I'll try!

[mvisonneau]

I doubt the issue you are facing is related to the rate-limit throttling, however to prevent it from happening out you can increase the maximum_gitlab_api_requests_per_second parameter (default: 10)

[PPACI]

latest image fixed the problem for me.
Would it be possible to release a 0.3.3 tag ?
We use this feature to monitor daily backup for production environnement.... And a latest tag is not really the kind of tag we want to deploy 😅

Btw, awesome job 👍

[mvisonneau]

I have now released it as part of 0.3.3, will close this issue as I assume it has fixed it :)