Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

missing data in collected time series #140

Closed
madeinoz67 opened this issue Dec 12, 2017 · 5 comments
Closed

missing data in collected time series #140

madeinoz67 opened this issue Dec 12, 2017 · 5 comments

Comments

@madeinoz67
Copy link

version: 0.2.7

I'm seeing on a number of wmi hosts (not all) missing time series data that appears to be determanistic (although not always)

This is only being observed on wmi_exporter nodes, linux node exporter exhibit no missing data.
the wmi nodes are not under any abnormal load during these drops in data.

I'm scraping about 30 wmi_nodes on 3 seperate seperate jobs based on their role , i.e. domain server has general collectors enabled + dns + AD, web servers have general + IIS and then general hosts have default collectors enabled . all jobs have scrape interval set is 20Sec. prometheus dashboard is not reporting any missed scrapes.

Mainly being seen on disk and memory metrics, however in some cases also on system_threads. More than likely is something I'm not doing correctly. These are mainly on default exporter install, although am seeing on on exporters with IIS enabled.

What is the recommended scrap interval for this exporter?

2017-12-12_10-45-34
2017-12-12_10-53-27

@martinlindhe
Copy link
Collaborator

martinlindhe commented Dec 12, 2017

This reminds me about an issue that has popped up before,
in that some WMI queries, on some machines took very long time to respond. This could cause gaps in the timeline.

You can debug this by stopping the wmi service (if you installed the .msi installer), and manually starting wmi_exporter.exe on a machine that you know this happens on, and keep an eye on the terminal output.

You can see more info about this being debugged in this issue: #122

@madeinoz67
Copy link
Author

@martinlindhe thanks for that. I will look at doing the debugging.

my current work around is to make null values connected so it doesn't show up the gaps on the screen

2017-12-13_08-27-22

@leoluk
Copy link

leoluk commented Jan 29, 2018

It's a bug in WMI, see #89.

I think I found a workaround, give me a few days to clean up the code and I'll publish it.

@pete-leese
Copy link

I’ve seen this myself. Has this now been fixed ?

@madeinoz67
Copy link
Author

I will close this one as a duplicated of #89

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants