Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune append-only repo if client has pruned the repo already #5950

Closed
reox opened this issue Aug 7, 2021 · 4 comments
Closed

Prune append-only repo if client has pruned the repo already #5950

reox opened this issue Aug 7, 2021 · 4 comments

Comments

@reox
Copy link

reox commented Aug 7, 2021

Have you checked borgbackup docs, FAQ, and open Github issues?

yes

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.9

Operating system (distribution) and version.

Debian Buster

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

Full borg commandline that lead to the problem (leave away excludes and passwords)

Server in append-only mode, configured via .ssh/authorized_keys: https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode

Client:

borg prune --verbose --show-version --show-rc --stats --list --keep-daily=7 --keep-weekly=4 --keep-monthly=12 --keep-yearly=-1 "$REMOTEURL"

Admin Machine tries the same prune command later.

Describe the problem you're observing.

The documentation for the append-mode says under "Drawbacks":

Be aware that as soon as you write to the repo in non-append-only mode (e.g. prune, delete or create archives from an admin machine), it will remove the deleted objects permanently (including the ones that were already marked as deleted, but not removed, in append-only mode).

See https://borgbackup.readthedocs.io/en/stable/usage/notes.html#drawbacks

However, I found that running borg prune on the repository from the admin machine does not work if the client (who can only append) runs prune on its own and thus the admin would not prune any archives.
The documentation suggest to me, that it should be possible for the client to tag the data as removed and the admin eventually removes it completely. But, in that case the admin has to find a time when the client has not yet pruned the repo and would have tagged any new archives.
Maybe I do not understand the documentation here correctly but it sounds like that running prune on the admin machine should remove all tagged archives, but it seems like it does this only if there are any archives to prune on the admin machine as well. Or does "as soon as you write" means that the pruning is only triggered if and only if something else is actually pruned, thus running borg prune without any archive pruned is only reading?

Furthermore, is there a way to remove only the tagged archives as the admin, even if no other archives would be pruned?

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

yes

Include any warning/errors/backtraces from the system logs

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Aug 7, 2021

If you want to compact the repo (== really remove deleted data) from the admin machine, you must not have "append-only" in the authorized_keys entry for the key used by the admin.

And as the docs tell, before compacting, you should make sure the repo is in the desired state (no malicious or accidental bad operations have been run).

borg 1.1: If you run prune from the admin machine, you would not also run prune on the append-only clients.

@reox
Copy link
Author

reox commented Aug 7, 2021

yes that is clear, maybe lets explain it a different way:

# client --> can only append-only
# admin --> has read/write access
client$ borg commit [...]  # runs daily

# now the client calls prune at some point in time:
client$ borg prune --keep-daily 7 [...]  # keeps 7 archives but only tags for removal

# After that, the admin wants to do the same:
admin$ borg check [...]
admin$ borg prune --keep-daily 7 [...]  # does not prune anything and does not compact the repo
admin$ borg prune --keep-daily 6 [...]  # removes one archive and compacts the repo 

@ThomasWaldmann
Copy link
Member

Run prune only from admin machine. Way easier to manage.

@reox
Copy link
Author

reox commented Aug 8, 2021

okay, that is good to know. That indeed answers my question.
Maybe that should be added to the documentation to clearly state this? Or did I just read over that in the documentation?

@reox reox closed this as completed Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants