outdated nodes on monitor.berlin.freifunk.net

[cholin]

We have plenty of non-reporting entries for our monitor service. 63 of 268 nodes are outdated for http://monitor.berlin.freifunk.net. I wrote a snippet to detect these nodes (see https://gist.github.com/cholin/3378b204ab7513bc6024 - output: http://paste.debian.net/hidden/2dbd50bb/).

I would suggest we remove these outdated entries automatically via a cronjob. For example after 14 days of inactivity.

[booo]

We may loose some data but I don't care. I think we should increase the inactivity time to a month.

[cholin]

I'm fine with a month. On the mailing list someone proposed 12 month (see archive) but I think that barely helps. Often people misconfigure their node and therefor we have orphaned entries and my goal is to delete these ones. With 12 month we won't archive this goal...

[cholin]

I would configure a cronjob to delete orphaned entries after one month of inactivity if no one complains in the next days.

[booo]

Sounds good. Thanks! I can integrate the script in puppet if you like.

[cholin]

Would be cool!

[andrenarchy]

+1 for 1 month.

[egmont1227]

+1 for 1 month

but before running the removal live, i suggest to annouce to the mailing list to ask to fix the reporting. I see a lot of known locations which just be misconfigured / not to much cared about the stats but still operational.

[cholin]

Just a short update: Seems I forgot to mention that the cronjob for deleting non-repoting nodes (with inactivity of 30 days) is configured since 2.02.2015. Seems integration into puppet is still missing.

[pmelange]

It seems like this isn't really working for nodes which no longer report a specific statistic. For example, the old ffvpn interface is not longer used by many nodes, but since the node itself still reports for example it's load average, the old ffvpn rrd files is still on the system.

/var/lib/collectd/rrd$ find . -type f -mtime +365 | wc -l
13042

Currently there are over 13000 rrd files which are over a year old. RRD files only store statistics for up to a year, so I think these files should be deleted too.