-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CLI: Improved customizability and scriptability of verdi storage maintain
#5936
Conversation
verdi storage mantain
verdi storage mantain
for more information, see https://pre-commit.ci
…to feature-no_repack
for more information, see https://pre-commit.ci
…to feature-no_repack
for more information, see https://pre-commit.ci
verdi storage mantain
verdi storage maintain
hey @Crivella, just checking: have you talked to anyone from the AiiDA team about this already who might be able to review this PR? |
Hi @ltalirz, I got in touch with @chrisjsewell on slack last week, he said he should give it a look this week. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great cheers!
Thanks @Crivella , useful addition. I was just wondering if we could have better used the |
Hi @sphuber I am not against it. Sorry, didn't notice that |
Introduced changes
This pull requests, adds 2 option to
verdi storage maintain
:--no-repack
: disables thedo_repack
option when running a full maintenance with theDiskObjectStore
backend.--yes
or-y
: skip the confirmation prompt to use command in scripts (eg periodic maintenance with CRON)Not using the flags leave the behavior of the command unaltered for back-compatibility
Reasoning
When performing a storage maintenance operation through the verdi CLI, the only two available options right now is to do either a live or non live operation.
Using the
DiskObjectStore
backend this translates to doing one of the 2:pack_all_loose
only, which store all loose files inside packs, but does not remove the loose files itself.pack_all_loose
,do_repack
,clean_storage
,vacuum
I think it would be useful to have a more granular control of the live maintain operation.
Especially the
do_repack
option can be undesired, when performing maintenance on a huge repository that is being backed up via rsync. Instead, performing theclean_storage
periodically is almost mandatory for having efficient backups. Without it we end-up in the same situation as the old storage of having a huge amount of IO nodes to rsync, plus the redundancy of having both the loose and packed version of a file ifverdi storage maintain
is being run periodically.