Multiple devices leads to snapshotting the same filesystem more than once

[senji-rsbackup]

pre-backup-hook and post-backup-hook are executed for each individual backup; where a backup is a backup of an individual volume onto an individual device. This means that if you're backing up to multiple devices they get run once per (volume,device) tuple.

This would only be ideologically annoying except that the (udev,lvm) system has a number of races/leaks; which seem to be exacerbated if you do a lot of snapshotting in a short space of time.

I can see a number of ways of addressing this issue:

• The backup-hooks could be run only once per volume, before the first backup and after the last.
• backup-hooks could be made to opportunistically reuse snapshots and then a post-host-hook added to clean up snapshots.
• A pre- and post-host-hook could be added that can be used to set up and tear down snapshots host-wide (this could also be used to set locks / warnings about running backups and other good things)

The latter two options have the problem that you might end up filling up your lvm with snapshots; whereas if everything is working OK currently you only have one snapshot at once.

[ewxrjk]

rsbackup could tell each hook whether it's first, last or somewhere in the middle (although it'd have to work out which of these was true). I wonder how this would interact with the request for more concurrency too...

[senji-rsbackup]

I don't think there is much advantage in attempting concurrency within a single host - you're likely to end up trying to read from the same physical medium; so hopefully the two issues are orthogonal.

[ewxrjk]

We now have https://github.com/ewxrjk/rsbackup/blob/master/src/Action.h to provide concurrent sequencing of actions, only used for pruning operations so far. Obviously the strategy for issue #24 is to use it for backups too.

For hooks, some additional concepts are needed.

1. Actions need to be able to depend on one another, so that pre-backup-hooks can be sequenced before all relevant backups and post-backup-hooks after them.
2. (Some) actions need to be suppressed if nothing depends on them, so that hooks are not run if nothing actually requires them.
3. Dependencies need to be either conditional, so that backups and post-backup-hooks aren't run if pre-backup-hooks failed, or unconditional, so that post-backup-hooks are still run even if backups failed.
4. Dependencies should be computed by pattern-matching at both ends, so we can say that all backups of a volume depend on that volume's pre-backup-hook, while a post-backup-hook depends on all backups of the relevant volume.

Can we do the *-access-hooks this way too? I think so but haven't thought through the details.

[ewxrjk]

The above leaves out the detail of how to ensure that snapshots only happen once (in favour of making sure that they happen at the right times). Running hooks only once per volume is attractive; it breaks compatibility but since the motivating application is snapshotting that doesn't seem particularly problematic (notice that the supplied snapshot hook does not access RSBACKUP_DEVICE).

I can't think of an easy way to issue deprecation warnings for accessing RSBACKUP_DEVICE (or rather, I can, but running the hook with an LD_PRELOAD module that intercepts getenv seems unreasonably intrusive), though it could be de-documented in an intermediate release.

[ewxrjk]

Volume hooks now only run once per volume. This hasn't had real-world testing yet so be a little cautious.