Option for symlink dereference

[cafuego]

A few of my backup repositories point at a 7 day rotating database dump directory. Each day, a dump of each database is written to /mnt/backup/database/YYYY/MM/DD/HH/II.

Originally I had restic backing up /mnt/backup/database, but that included 7 full copies of all data in each backup. Whilst redundancy is nice, this means a restore takes much longer than a restore of a single copy and there is seven times the data recovery and transfer charge.

I though I was smart by linking /mnt/backup/database/YYYY/MM/DD/HH/II to /mnt/backup/database/current, but in that scenario restic only backs up the symlink. Fast, but not helpful ;-)

I have a solution now that only backs up single copies of data by backing up /mnt/backup/database/current/* but that means a list of all database dumps is used by restic as the backup path. It also means that when I add or delete a database, the path changes and the expiry jobs see the snapshot with the different path as a new snapshot.

That leaves me with way more data stored offsite than is strictly necessary and thus a higher storage bill.

It would be wonderfully helpful if restic had an option to dereference symlinks, so I could point it at /mnt/backup/database/current and have it backup the contents of the directory that links points at.

[pvgoran]

There is a discussion about symlink dereference in #542.

For your situation, I believe a more promising approach would be to introduce some syntax to tell restic during backup "this file/directory should be recorded in the repository under that path". (I think it was discussed at some point, but I'm not sure.) This would allow you to dereference /mnt/backup/database/current in your backup script, pass the resulting path to restic, but tell it to backup it as something like database/current.

Symlink dereference is, of course, looks like a simpler approach, at least from the user's point of view, but it has some problems (again, see #542).

[fd0]

Hey, thanks for taking the time to raise this issue and describing your use case in such detail! That's very valuable to us.

Making backups with restic with changing/unstable paths is not so easy and may cause additional work. There's a PR in the works (#1196) which will allow grouping for forget by tags only, so maybe that'll help once it's added.

I'm not convinced (as outlined in #542) that an option to de-reference symlinks is a good idea. Since all this is included in #542, I'm going to close this as a duplicate. I've spent some time thinking about the issue and I'll add a comment over there. Please subscribe to #542 if you'd like to be notified on any progress.