[Feature Request] Support for restic copy command

[cqjjjzr]

Hi, just discovered this awesome project and I wonder if we could support the restic copy command: https://restic.readthedocs.io/en/stable/045_working_with_repos.html#copying-snapshots-between-repositories

When backing up to multiple backends, the current solution is to run restic backup command multiple times. However, if all the backends have similar options, then: 1) backing up to one repository first; 2) then copying the whole repository to all other backends; would save a lot of time and disk I/Os. However, since all data is needed to be downloaded and then uploaded, the first repository (the one as the destination of restic copy) must be fast and responsive.

The ideal solution is: For configuration, add a copy section in the location nodes in locations. This section consists of two elements:

• from - must be one name from backends in to list of the current location;
• to - a list of backends

When the backup is completed for the current location + from backend specified in the copy section, (should the time and whether to run the copy command be customizable?) autorestic should run restic copy for all to backends specified in the copy section to synchronize the snapshots.

[cupcakearmy]

Thats a very interesting suggestion. What about something like this?

location:
  from: ...
  to: backend1
  copy:
    - backend2
    - backend3

[cqjjjzr]

Factually I believe it could be:

location:
    from: ...
    to:
        - backend1 # fast
        - backend2 # slow
    copy:
        from: backend1 # disambiguity
        to:
            - backend3
            - backend4

But this syntax seems to be a little verbose..

[cupcakearmy]

Mhh, it is a bit verbose. Though more flexible for sure.

location:
    from: ...
    to:
        - backend1 # fast
        - backend2 # slow
    copy:
        - backend1: backend3
        - backend1: backend4

or

location:
    from: ...
    to:
        - backend1 # fast
        - backend2 # slow
    copy:
      backend1:
        - backend3
        - backend4

Don't know which I would prefer, all threee seem reasonable. Maybe I'll ask discord 😅

[cqjjjzr]

The second one is even more flexible since it supports many-to-many copying.

Additionally, I noticed that we don't need to place the copy section in a location! It could be a global section because copy command doesn't depend on any location.

[cupcakearmy]

2 and 3 shold be equivalent mostly, you can do many to many with both. But maybe the 2nd is more clear? I'll ask discord about 2 vs 3.

I thought too about a global config option. But if it's a global setting every location will be mirrored. I think it's better to scope it to a location?

[cqjjjzr]

The copy command is not based on source locations, but are about backends (aka restic repositories) only, so it makes sense to make it global.

[cupcakearmy]

Yes, but I think its better to only copy the current snapshot made for a location than every snapshot ever made on that backend.

[cqjjjzr]

Hmm.. The filter criteria can be tags, hosts, paths, snapshot ID, maybe we could allow both one global section and one in the location (which uses paths for filtering)?

[cupcakearmy]

Paths are a bit of a pain to work with as they are an array tbh. thats why I introduced tags for each location. But then the usecase is replicate an entire repo?

[cqjjjzr]

According to the restic document, you could replicate the entire one, but can also pick out some snapshots (based on those criteria) to copy.

But anyway, it's a operation between two repos.

[cupcakearmy]

Yes, the operation is between two repos, but semantically it's still relevant to a location right? because I though you wanted to copy the backup that was just made to another repo right? and a backup in the "world" of autorestic is location bound.

[cqjjjzr]

Hmm. That's correct.

[cupcakearmy]

Discord has spoken https://discord.com/channels/252403122348097536/834380500315537478/923163095801688106

location:
  from: ...
  to:
    - backend1 # fast
    - backend2 # slow
  copy:
    backend1:
      - backend3
      - backend4

seems to be the more popular version

[FarisZR]

Until this is implemented, can I use the normal Restic copy command to copy autorestic repositories ?

[cupcakearmy]

You should be able to simply use autorestic exec to run any native restic commands :)