Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for creating sparse files with whisper-resize (or read the configuration from carbon) #331

Closed
SJrX opened this issue Jan 8, 2024 · 1 comment · Fixed by #332

Comments

@SJrX
Copy link

SJrX commented Jan 8, 2024

Please ask feature request only for whisper component here, if you want to to do so for Graphite, please use graphite-web repo

Is your feature request related to a problem? Please describe.
I wanted to change the retention period of a bunch of files. The instance and metrics are tuned very well and makes heavy use of sparse files to allow us to have a huge number of metrics with low disk space.

It is surprising and counter-intuitive that if you shrink files to truncate some of them, that your disk space can blow up because the new files aren't sparse.

Describe the solution you'd like
A new argument --sparse that causes the created files to be sparse.

Describe alternatives you've considered
I guess I have a work around which is just:

find ./ -type f -name '*.wsp' -exec echo sh -c '/opt/graphite/bin/whisper-resize.py --nobackup {} 30s:7d 5m:90d; fallocate -d {};

However in theory that might be twice as slow since it rewrites the file twice, although maybe fallocate is faster at this than python would be 🤷

Additional context
Carbon supports sparse creation of files here: https://github.com/graphite-project/carbon/blob/master/conf/carbon.conf.example#L205

@deniszh
Copy link
Member

deniszh commented Jan 9, 2024

Oops, I recently did resize with sparse, but completely forgot to submit my changes. Will do when recover, got some nasty flu.
Or submit your pr if you have it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants