Borg and incremental backups...

[rofo69]

I'm a total noob to Borg and wanted to understand a little more about how it works.
I've seen videos where people are creating archives with names appended by dates via bash scripts. All seems fine.

To keep it simple, lets say I have a single repo containing a several Tb of media files. If I schedule a daily backup of this, using this same approach, it would create archive files, each with a different date as part of the archive name.

Now, given borg is a de-duplicating backup process, does this span different archives in the same repo ? Lets say I have 1tb of data to backup that results in a 1tb archive. I assume that 7 days of backups would result in still only 1tb of actual archive space taken if no changes happened over that period ?

If this is true how does archive rotation work if all the backups are incrementals based on the first archive. ? Is there a way you re-baseline the data ?

[FelixSchwarz]

You need to switch your mental model completely: There are no "incremental" backups. Each archive is basically a full backup but the same data is stored only once (deduplication). You can prune older archives and borg will remove all chunks which are not needed anymore.

This elegant design is the reason why people use borg and similar tools like restic.

The downside is that a single corrupted block can affect many backups so you need to handle that yourself (e.g. backup to multiple locations).

[rofo69]

Ah ok, so manually deleting archives is basically a no no. Leave Borg to manage it via pruning. This starts to make sense. It also means surely that having archives with dates appended to them are irrelevant ? Just let Borg handle it.

[FelixSchwarz]

What do you mean by "manually deleting archives"? You can run borg delete. Just do not delete some of borg's files directly.

[rofo69]

thats what I meant. Don't go into the file system and delete them.

[FelixSchwarz]

Each backup run needs a unique name, so adding a date can be helpful. However, the internal Borg storage is entirely opaque, meaning you shouldn't manually do some file system operations.

[rofo69]

Thanks @FelixSchwarz - thats been super helpful in my journey to understand it better. More reading to be done.

[ThomasWaldmann]

Closing this as outdated: it has seen no activity for a long time and predates the current borg releases.

If this still affects you on a current version - borg 1.4.x (stable) or the borg 2.0.0 beta - please open a new discussion with up-to-date details.