filter for node_filesystem_* and node_network_*

[neelance]

We're running node_exporter on machines on which we use Docker. There are a lot of docker containers that get spawned and destroyed. Over time, node_exporter accumulates hundreds of entries for node_filesystem_*{filesystem="/var/lib/docker/devicemapper/mnt/*"} and node_network_*{device="veth*"}. I think that could be a reason why our prometheus is using a lot of memory. I see two possible solutions: One is to filter those values completely, the other is to forget about old entries after some time. What do you think?

[brian-brazil]

There's already a --collector.filesystem.ignored-mount-points for the former and we'd accept a PR to add similar for the latter.

Do these filesystems and devices still exist? If not, that's a bug in the node_exporter as it should only be exporting things that exist right now.

[neelance]

No, most of the filesystems and devices don't exist any more.

[juliusv]

@neelance Ok, so they don't exist on the host anymore, but you can still see them on your node exporter's /metrics HTTP endpoint (could you confirm)? In that case, it's indeed a bug that should be fixed in the node exporter.

[neelance]

Yes, exactly.

[juliusv]

@neelance Thanks, I opened #61 to track this then. Closing this one then (please reopen if you feel like there's something missing).

[neelance]

What about the network interfaces? Shouldn't it also drop those when they don't exist any more?

[juliusv]

@neelance Ah thanks, I missed that second part. Filed an issue about that too: #62

[neelance]

Great, thanks!