Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Update backup script to utilize all cores #36

Closed
SudosFTW opened this issue Apr 16, 2022 · 1 comment
Closed

Suggestion: Update backup script to utilize all cores #36

SudosFTW opened this issue Apr 16, 2022 · 1 comment
Assignees

Comments

@SudosFTW
Copy link

As per the examples in the post on this blog, one can decrease the time needed to compress files into a gzipped tarball by adding in an external compressor. This would be ideal for multi-core Pi/other embedded ARM boards/etc. (and any other systems that may utilize your scripts, as I'm personally using it to make my life easier on a low-end quadcore amd64 thin client with the network checks commented out, which works quite well in fact!)

by changing the script to read as such...

tar -I pigz --exclude='./backups' --exclude='./cache' --exclude='./logs' --exclude='./jre' --exclude='./paperclip.jar' -pvcf backups/$(date +%Y.%m.%d.%H.%M.%S).tar.gz ./*

...it will now utilize all available cores to make the backup instead of only utilizing a single core and allow the backup to go faster by a factor of 2-3 times. This makes sense both for a faster startup experience for a small server instance and also for a server that has been running for a long time with many chunks, or a pre-generated world with many chunks. Additionally, after this it might be a good idea to throw in a sync command to be sure the data is fully flushed to disk/SD card/etc. before starting the java instance since it doesn't hurt anything.

I've already tested this in my own startup script and it works splendidly, and while I don't have a modern pi to test it out with, it should be safe to use going forward on those so long as tar is able to recognize the -I pigz argument with -pvcf at the end instead.

@TheRemote
Copy link
Owner

Hey SudosFTW,

This is a great idea! It adds a 36KB dependency of "pigz". I figured out how to work that in where if pigz is present it will do the multicore backup but if it's not present it will do the regular "slow" backup.

I've added it in here. Thanks for the suggestion!

@TheRemote TheRemote self-assigned this May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants