The automatic parent selection of cmd_backup is not smart enough.

[middelink]

Currently the code sets the current hostname as default. This is then used in FindLatestSnapshot to discover the latest snapshot for that host.

However, what if I make 2 snapshots for a host, because I have different retention policies for them? E.g. each night the host makes backups of /etc and of /home. The next night backup runs for /etc again and picks the /home backup as parent, as it is the most recent one. Consequently the /home backup picks the just made /etc as parent...

So none of this would be especially worrisome as restic would still dedup files at file and blob level, but as you (@fd0) explained so eloquently in in #652, picking the wrong parentID will result in lots of compute cycles as it cannot determine if the file has changed upfront.

So, I think the way forward would be to create a FindOptimalParentSnapshot(repo, tags, paths, host) function, which uses host as a hard constraint, but the tag and paths ones as soft constraints. Bonus points if it would notice supersets. If I created a backup of /etc,/home and later on do a backup of just /home, it could pick the /etc,/home backup as it's parent. (unless there is a more recent single /home of course). Then of course making a new backup of /etc,/home becomes hairy. what do we pick, the most recent one of /home or the most recent one of /etc,/home ? My bet is on the later as they will have the most overlap.

[sanmai]

You can set your own hostname for each and every backup, if that helps.

[fd0]

Good idea for improving parent snapshot detection!

Unfortunately I think you've got a tiny bit wrong:

However, what if I make 2 snapshots for a host, because I have different retention policies for them? E.g. each night the host makes backups of /etc and of /home. The next night backup runs for /etc again and picks the /home backup as parent, as it is the most recent one. Consequently the /home backup picks the just made /etc as parent...

Suppose I have a backup of /home and a separate, later backup of /etc on the same host. On the next backup run of /home, restic will find the first snapshot (which just contains /home):

• in cmd_backup.go the function FindLatestSnapshot is called with the set of paths and the hostname as parameters: https://github.com/restic/restic/blob/master/src/cmds/restic/cmd_backup.go#L398
• This function then loads all snapshots and selects the latest one with the same hostname and set of paths: https://github.com/restic/restic/blob/master/src/restic/snapshot.go#L156

Is there a bug that I'm not aware of?

[middelink]

Aww, shoot. I completely missed that. <ponders> ...but there was something wrong with the code when I went over it yesterday...

Ah. its SamePath() which threw me off. Say I have 2 existing backups, /,/boot and a newer backup of / (of the same host; who knows why ^^). Then when I make a new backup of /,/home, the one with / will be it's parent.

SamePaths() is not really checking if the paths are exactly the same, but rather a subset. (expected="/", actual="/,/boot") will return true. And since FindLatestSnapshot picks the most recent matching backup. Bingo, problem.

PS. Why does that function not take a Snapshot receiver, like HasTags? All 4 use-cases take sn.Paths as first argument

[fd0]

Oh, interesting. SamePaths() does actually only test whether actual is a subset of expected. That's... unexpected (from the function's name).

And we can certainly make SamePaths a function of Snapshot.