borg2: how to deal with incomplete backups

[tve]

I'm wondering how to deal with incomplete backups. Here's a specific example:

etc: stat: [Errno 2] No such file or directory: 'etc'..ssbook/system/contacts.db
home: stat: [Errno 2] No such file or directory: 'home'
opt: stat: [Errno 2] No such file or directory: 'opt'
root: stat: [Errno 2] No such file or directory: 'root'
[...]
Number of files: 29190
Original size: 45.32 GB
[...]
Saving files cache
Saving chunks cache
Saving cache config

[chan 0] sftp session closed.
terminating with warning status, rc 107
== exit code: 107

For comparison, a complete backup says:

Number of files: 2226211
Original size: 292.48 GB

The two problems I have are:

1. how to tell whether I got a complete backup or not
2. what to do with incomplete backups

It seems that the answer is that any run that doesn't produce a 0 exit code is no good and that such runs should be deleted. But that begs the question why they don't error out on their own as soon as a problem comes up and why they produce a seemingly good backup.

By the way, I see that I can get the number of files and original size from borg info, would it make sense to store the result code as well? Or some similar information about "errors and warnings encountered".

[ThomasWaldmann]

If it terminates with != 0, you have to check the logs. borg can't know if a missing or unaccessible file is important or not.

The usual way to deal with source file access issues that is to skip the file, emit a warning (and change to a warning rc) and proceed with the remaining files. A backup that missing some files is still better than one that aborts or misses even more files.

It is more efficient to delete "bad" backups AFTER you made a good one due to deduplication.

Storing the result code (or even emitting it as json) is difficult, because it is only final when borg actually terminates. Storage of the warnings is usually done in the logs, not sure if we should store that in archive metadata.

[tve]

OK, it seems to me that your responses address the case where I'm watching the backup. I have a script that follows the borg create with a borg check and a borg prune. As it stands the borg prune can easily keep the bad backup and cut out a nearby good one.

Maybe storing the final result code is impossible but storing the number of errors and warnings encountered shouldn't be.

WRT efficiency, does the order matter if there is no compaction in-between?

[tve]

I would also add that I would consider the safe default to be to error out or somehow mark the archive as bad/questionable if there were any warnings or errors. The user can decide to keep/rename the archive if it has value nevertheless, or perhaps pass a --always-keep flag.

[ThomasWaldmann]

The problem is that warnings in general are not rare, especially if one backs up a live filesystem. If borg scans the directories and invokes archiving of the dir items, it can easily happen that a file vanishes due to a rename or delete and that will result in an appropriate warning.

If we would mark all archives questionable that had such issues, you could end up with many marked ones.