More efficient backup log

[ewxrjk]

$ cat * | wc -c
282855
$ du -ks .
4100    .

i.e. more than 10 times as much space is being used as is really needed. The logs could be combined into one file, or just a few files (e.g. one per host or one per volume).

There is an argument that old-style logs should still be written for a while even when a new-style combined format is written and read, so that after a partial restore of a backup host that reverts an upgrade but does not revert the backup log, backups can continue to work properly without replaying the upgrade. However, pruning cannot work in such a downgrade situation - the new-style data will become inconsistent with the old-style data. Therefore no attempt will be made to support downgrades.

[ewxrjk]

The current log contains:

• the date (but not a timestamp)
• what host/volume was backed up
• what device is was backed up to
• an exit status
• an output string

Thoughts:

• The above list translates into a SQL table naturally.
• This seems like an opportunity to upgrade the date to a second-resolution timestamp, allowing for the possibility of more than one backup per day in the future (probably not as part of this issue). Precise timestamps can be recovered from st_mtime during the upgrade process.

[senji-rsbackup]

Looking at my log directory filesizes vary from 19 bytes; containing just OK: device=tenneth from a volume with no backup-hooks and just the output "OK: device=tenneth" to 212241 bytes in the worst case with a hell of a lot of diagnostic output from the backup-hook (the next longest is a mere 25406 bytes).

I guess the diagnostic output from backup-hooks can be stored in an SQL TEXT field; or they could be dumped into a per-day or per-host log file.

[ewxrjk]

The issue11 branch contains an (experimental) implementation using SQLite. I've not deployed it locally yet (though it does pass its tests).