Missing metrics (exported but not stored by Prometheus)

[mtanatwine]

What did you do?
Attempt to use Prometheus to view node_exporter metrics
What did you expect to see?
Metrics existing in Prometheus
What did you see instead? Under which circumstances?
Missing metrics (node_network*, to be specific)
Environment

• System information:

Linux 4.9.20-10.30.amzn1.x86_64 x86_64

• Prometheus version:

prometheus, version 2.0.0 (branch: release-2.0, revision: b3ddaf5)
build user: kitchen@6e402e1ef1af
build date: 20171113-10:50:17
go version: go1.8.3

• Alertmanager version:

N/A

• Prometheus configuration file:

N/A

• Alertmanager configuration file:

N/A

• Logs:

There appear to be no relevant logs pertaining to this issue.

We have a number of instances in AWS which are running node_exporter. I've connected to the metric endpoint manually and verified that they are exposing the network statistics, which they indeed are. I've also performed a curl from the Prometheus server, and this also returns said network statistics, so I'm at a loss as to why the metrics aren't making themselves into the database. As you can see from the screenshot above, surely there should be at least the same amount of node_network* metrics as there are instances. I realise that some instances may have more than one network interface, but that would result in the count being higher, not lower than the memory metric.

I've tried enabling debug logging but that just spits out unrelated errors (instances discovered via the ec2 config but those which are not yet running node_exporter).

Any ideas?

[brian-brazil]

It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.

[mtanatwine]

I did see the comment about usage questions, but I thought it was worth logging it here as it would appear to be a bug, given the metrics are all being exported but not consumed by Prometheus.

[brian-brazil]

There's insufficient information here to indicate a bug, and usually these issues turn out to be usage issues, so the default presumption is that it's an issue outside of Prometheus. As we get quite a few of these, we direct users to the mailing list to keep the bug tracker tidy.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.