Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: compacting schedule like we do for pruning #1718

Open
marco44 opened this issue May 22, 2023 · 10 comments · May be fixed by #1981
Open

FR: compacting schedule like we do for pruning #1718

marco44 opened this issue May 22, 2023 · 10 comments · May be fixed by #1981
Labels
priority:high Security issue, or critical to functionality or usability (Cannot function or hard to use without) type:enhancement Improvement of an existing function

Comments

@marco44
Copy link
Contributor

marco44 commented May 22, 2023

The problem

I just recently upgraded to borg 1.2. The repo started growing very fast, and I learn that now we need to run compact at regular intervals to reclaim space. The compact button allowed me to recover the lost space for my repo. But I feel it should be run regularly, like we do for pruning or validation

Requested Solution

Having the possibility to compact after each backup or at regular interval.

Additional context

I saw some discussions about this in other issues about compact, but nothing clearly about this. Maybe I missed something?

@marco44 marco44 added the type:enhancement Improvement of an existing function label May 22, 2023
@m3nu
Copy link
Contributor

m3nu commented May 22, 2023

Would fit nicely under Schedule. Similar to Validation. Ever X weeks etc.

Screenshot 2023-05-22 at 15 13 07

@marco44
Copy link
Contributor Author

marco44 commented May 22, 2023

Yeah, but which one would be better ? Like autopruning, or like validation ? I think both have their merits. I suppose borg developers had a reason for not compacting on each run...

@real-yfprojects
Copy link
Collaborator

I suppose borg developers had a reason for not compacting on each run...

I'd do it like validation.

@masterjuggler
Copy link

Is this something that is in line to be added at some point? Right now I have to either set a cron job or add it as a post-backup shell command, which works but isn't obvious for people setting up a new repo.

@m3nu
Copy link
Contributor

m3nu commented Oct 21, 2023

Could be added and grouped with pruning. Like Run post-backup tasks > Prune, Compact

@masterjuggler
Copy link

If it's not a major effort to add, I think it would be worthwhile because it's something that's very easy for most people to not realize is a necessary manual step, which results in a very bloated repo.

@Gerben321
Copy link

Would indeed be a nice feature. I noticed some unexplainable large files on my Unraid server and traced it back to my Vorta destination folder on another server. My supposedly ~5GB backup has become 262GB. I am now compacting it. Would be great to have the option to do it automatically once in a while or after each prune or something. Shouldn't be that difficult.

@AnthillSudoku
Copy link

could be adding this command to "schedule > shell commands > post-backup" a workaround?

borg compact $repo_url

@ThomasWaldmann
Copy link
Collaborator

ThomasWaldmann commented Jan 7, 2024

I just put this into next milestone, this should get a high priority.

If implementing something advanced takes longer, something simple should be done ASAP so that "borg compact" is invoked. Not invoking it at all without user interaction is no good idea, backup space usage is exploding for some people until they notice they have to click on "compact" manually. If people did not set up repository quota or reserved space somehow they might even run completely out of free fs space, which is a known very bad situation with borg.

So, a quick fix would be vorta just doing a borg compact implicitly after each borg prune, no matter whether it was invoked manually or automatically.

If vorta supports borg < 1.2 still, borg compact must only be called for borg >= 1.2.

@ThomasWaldmann ThomasWaldmann added the priority:high Security issue, or critical to functionality or usability (Cannot function or hard to use without) label Jan 7, 2024
@shivansh02
Copy link
Collaborator

Hi I'd like to work on this. Should I go for the quick fix first, by running borg compact after each prune, or start working on the scheduling?

@shivansh02 shivansh02 linked a pull request Apr 5, 2024 that will close this issue
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:high Security issue, or critical to functionality or usability (Cannot function or hard to use without) type:enhancement Improvement of an existing function
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants