Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sparse/fallocate/uid/gid options for whisper-resize.py #332

Merged
merged 5 commits into from
Jan 19, 2024

Conversation

deniszh
Copy link
Member

@deniszh deniszh commented Jan 19, 2024

Fix #331

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (07f3db9) 87.83% compared to head (462261b) 87.83%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #332   +/-   ##
=======================================
  Coverage   87.83%   87.83%           
=======================================
  Files           2        2           
  Lines        1143     1143           
  Branches      206      191   -15     
=======================================
  Hits         1004     1004           
  Misses         99       99           
  Partials       40       40           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@deniszh deniszh merged commit f4721ac into graphite-project:master Jan 19, 2024
10 of 11 checks passed
@deniszh deniszh deleted the dzhdanov/resize-sparse branch January 19, 2024 11:51
@BBQigniter
Copy link

just a note - I love the --quiet and --silent changes. (Only found the new script it too late)

I lately had to do a mass resize and tried to figure out how to do it the fastest way. Using the xargs-approach for example via find ./ -name "*.wsp*" | xargs -P750 -n1 -I'{}' /opt/graphite/bin/whisper-resize.py --aggregate --nobackup {} 10s:6h 1m:6d 10m:90d, the resize would have taken 6h+ - as we are currently using a the latest available graphiteapp/graphite-statsd:1.1.10-5 container, were the old whisper-resize.py script is included.

After checking the script in the container, I just commented the line print("oldtimestamps: %s" % oldtimestamps). The resize took on a test-maschine 1-2h instead. Also memory usage dropped from roughly 15-16G to about 4G max while running that amount of resize-tasks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for creating sparse files with whisper-resize (or read the configuration from carbon)
3 participants