Merge multiple snapshots into one OR Append operation

[samuelxhu]

I am faced with a task using restic as a archive solution. The use case is as follows: The main disk is on SSD, fast but with limited capacity, and a second HDD on USB as an archive disk. The idea is to (from time to time or on a schedule) store (archive) data on SSD to HDD, and delete those already-archived data on SSD to leave space for new data.

Thus, is there a way to merge multiple snapshots into one (Archive)? or is it possible for restic to support archive or append operation without breaking back compatibility?

thanks a lot in advance

[greatroar]

This would be possible in theory given restic's file formats, but the question is, why do you think you need it? If disk space is the main concern, restic's deduplication should already take care of that: the same file written to the same repository twice will only get stored once, even if it's part of two distinct snapshots.

[MichaelEischer]

That sounds similar to what #3405 is proposing.

[samuelxhu]

My use cas is to archive, therefore I would like to have all files on the last snapshot.

…

On Sat, Jun 11, 2022 at 12:58 PM MichaelEischer ***@***.***> wrote: That sounds similar to what #3405 <#3405> is proposing. — Reply to this email directly, view it on GitHub <#3792 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYMCAIRV5QFEG34R25GC3TVORWNVANCNFSM5YKA2USQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[MichaelEischer]

Then why don't you just create a snapshot that includes all relevant paths, that is in your case both disks?

[greatroar]

I was assuming the restic repository was on the USB disk. Is that not correct?

[samuelxhu]

Indeed, the intended archive disk on a USB3 disk or a remote S3 bucket. The problem with multiple snapshots as archives is as follows: user has to search for files among tens of such snapshots, which is cumbersome.

…

On Sat, Jun 11, 2022 at 7:23 PM greatroar ***@***.***> wrote: I was assuming the restic repository was on the USB disk. Is that not correct? — Reply to this email directly, view it on GitHub <#3792 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYMCAKZEC2L5T5IWFJFDJLVOTDPJANCNFSM5YKA2USQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[greatroar]

This could be solved by #3405. Alternatively, it can be solved by a view in restic mount that combines snapshots, or even a union mount on top of restic mount.

(It is stretching restic's intended purpose from backup to archiving, but I admit I would find this useful too...)