No Graphs from graphite for 12 hours or more (Days, Weeks, Months and Years)

[geossh]

Hi,

First of all big thanks to Jordan and all the other contributors for your amazing work.

Icinga2 with Icingaweb2 and Graphite runs quite perfect in drawing graphs for 5 min up to 6 hours, but fails to draw the graph for days, weeks, months and years (at least in my environment).

Anyone an Idea?

Thanks,
Hans

[jjethwa]

Hi @geossh

I'm not an expert on Graphite unfortunately, but I think you might need to tweak the carbon configuration with your retention and collation policies: https://graphite.readthedocs.io/en/latest/config-carbon.html

[geossh]

Hi Jordan,

You are an expert on graphite too :-)

It was exactly the retensions in data/graphite/conf/storage-schemas.conf. First I realized that Icinga2 is doing a check every 5 minutes only and that did not correspond with the retensions set in storage-schemas.conf. Which was
retentions = 10s:6h,1m:6d,10m:1800d

So I changed that to
retentions = 5m:10d,15m:30d,30m:90d,360m:4y

Now new *.wsp files should be correctly filled. Next I wanted to migrate the already existing *.wsp files to the new retensions. First I had to install python3-whisper (my host is a Ubuntu-Server):

apt install python3-whisper

And after that I ran:
find /opt/icinga2/data/graphite/storage/whisper/icinga2/ -type f -name '*.wsp' -exec /usr/bin/whisper-resize --nobackup {} 5m:10d 15m:30d 30m:90d 360m:4y ;

And now I have a perfect drawing for days and weeks too :-)
Maybe it is possible for the next upgrade of your amazing docker image (if there will be one once a time) to change the retensions of storage-schemas.conf to the icinga default (which is a check every 5 minutes).

Thanks again and all the best,
Hans

[jjethwa]

That's great news, @geossh and thanks so much for the information, it's really helpful! The storage-schemas.conf is default from the graphite official image, so maybe we need to figure out a way to inject the updated retention config line into it.

[geossh]

Dear Jordan, I have to thank you :-) And maybe it's enough to make a note in the readme (https://github.com/jjethwa/icinga2#graphite). All the best and many greetings, Hans

…

On 21.02.22 17:17, Jordan Jethwa wrote: That's great news, @geossh <https://github.com/geossh> and thanks so much for the information, it's really helpful! The storage-schemas.conf is default from the graphite official image, so maybe we need to figure out a way to inject the updated retention config line into it. — Reply to this email directly, view it on GitHub <#275 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVLAQFH5P7MGMDVFW3TVK4DU4JQSTANCNFSM5OT54ABA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.

[jjethwa]

Thanks @geossh 😄 Added a link to your comment in the README and if I figure out a good way to inject the config, I'll let you know.

[4rne]

And after that I ran: find /opt/icinga2/data/graphite/storage/whisper/icinga2/ -type f -name '*.wsp' -exec /usr/bin/whisper-resize --nobackup {} 5m:10d 15m:30d 30m:90d 360m:4y ;

Github seems to have stripped the \; at the end so the command should be:
find /opt/icinga2/data/graphite/storage/whisper/icinga2/ -type f -name '*.wsp' -exec /usr/bin/whisper-resize --nobackup {} 5m:10d 15m:30d 30m:90d 360m:4y \;