Inconsistency in elapsed time while restoring the same backup to different destinations

[tricarte]

Rustic version: 0.5.0

Same issue also applies to v0.4.4.

OK, all my backups are in an external drive. Today, I restored one of my small sized backups to a ~/test directory and it was fast enough. Then I tried restoring the same backup to ~, it took longer. Here is the difference for a small sized backup.

# Restore to ~/test
$ time rustic --log-level OFF --no-progress -P repos restore latest \
    --filter-paths /home/username/repos/wpsite /home/username/test
using config /home/username/.config/rustic/repos.toml
Files:  629 to restore, 0 unchanged, 0 verified, 0 to modify, 62 additional
Dirs:   244 to restore, 0 to modify, 33 additional

real 0m7,083s
user 0m0,975s
sys 0m0,550s

# ******************************************************

# Restore to ~
$ time rustic --log-level OFF --no-progress -P repos restore latest \
    --filter-paths /home/username/repos/wpsite /home/username
using config /home/username/.config/rustic/repos.toml
Files:  629 to restore, 0 unchanged, 0 verified, 0 to modify, 1127748 additional
Dirs:   244 to restore, 0 to modify, 206988 additional

real 0m45,774s
user 0m8,314s
sys  0m10,562s

And here is another backup that is 874MB.

# Restore to ~/test
$ time rustic --log-level OFF --no-progress -P wpready-based-sites-backups restore latest \
    --filter-paths /home/username/backups /home/username/test
using config /home/username/.config/rustic/wpready-based-sites-backups.toml
Files:  22 to restore, 0 unchanged, 0 verified, 0 to modify, 62 additional
Dirs:   24 to restore, 0 to modify, 33 additional

real 0m52,199s
user 0m5,693s
sys 0m3,394s

# *************************************************

# Restore to ~
$ time rustic --log-level OFF --no-progress -P wpready-based-sites-backups restore latest \
    --filter-paths /home/username/backups /home/username
using config /home/username/.config/rustic/wpready-based-sites-backups.toml
Files:  22 to restore, 0 unchanged, 0 verified, 0 to modify, 1127141 additional
Dirs:   24 to restore, 0 to modify, 206768 additional

real 1m33,330s
user 0m13,904s
sys 0m14,888s

Removing the --no-progress argument reveals where it spends more time but no clue about why:

# Restore to ~
[00:00:48] collecting file information...
[00:00:41] restoring file contents...     ████████████████████████████████████████ 762.61 MiB/762.61 MiB 18.55 MiB/s  (ETA 0s)
# Subsequent run
[00:00:45] collecting file information...
[00:00:45] restoring file contents...     ████████████████████████████████████████ 762.61 MiB/762.61 MiB 16.72 MiB/s  (ETA 0s)

# Restore to ~/test
[00:00:00] collecting file information...
[00:00:24] restoring file contents...     ████████████████████████████████████████ 762.61 MiB/762.61 MiB 30.83 MiB/s  (ETA 0s)
# Subsequent run
[00:00:00] collecting file information...
[00:00:14] restoring file contents...     ████████████████████████████████████████ 762.61 MiB/762.61 MiB 50.93 MiB/s  (ETA 0s)

Thanks in advance.

[aawsome]

If you restore to an already existing dir, rustic scans this dir. It then shows the result under the statistics as additional.

If you specify the --delete option, also all of these additional files/dirs will be removed.

This explains your observation: Restoring to an empty dir is faster than restoring to a dir containing lots of entries...

[aawsome]

rustic --log-level OFF --no-progress -P wpready-based-sites-backups restore latest \
    --filter-paths /home/username/backups /home/username/test

FYI, all of these options can also (but of course must not) be specified in the config file:

[global]
log-level = "OFF"
no-progress = true

[snapshots-filter]
filter-paths = ["/home/username/backups"]

[tricarte]

I didn't realize that 'additional' section. But I'm still confused.

rustic -P profile_name restore latest \
    --filter-paths /home/username/backups /home/username

First, a question: Does this command mean 'restore the latest snapshot that contains the path /home/username/backups to /home/username?

If true:

If I want to restore the path '/home/username/backups' to /home/username, after the restore, what I have is:
/home/username/home/username/backups. So why does rustic scan those additional, almost 1 million files and 200K dirs in my home dir where I would expect rustic to scan for just a single directory called 'home' in my home dir.

Also your snapshots-filter example makes me completely confused. Is this a source-to-destination like pairing or just two paths that I want to restore. I think, it's not a source-to-destination like pairing, because it didn't work. If they are two paths that I want to restore, then you got me wrong because in my first examples, /home/username/test is not what I want to restore. It's the destination. Maybe you just come up with /home/username/test dir as an example path in the repository, I don't know.

Sorry for not making it short. Thanks.

[aawsome]

First, a question: Does this command mean 'restore the latest snapshot that contains the path /home/username/backups to /home/username?

Exactly!

Sorry for the confusion about the snapshot filter. You filtered just one path - I corrected the example.
The snapshot filter only ever filters snapshots before processing them. In your example, the filter is applied before getting the latest snapshot of the filtered snapshot list. Than that latest snapshot is used to do the restore.

If you want to restore a sub-dir within your snapshot to a specific dir in your filesystem (maybe this is what you mean by "source-to-destination like pairing"), you can use the <SNAP>:PATH syntax. In your example:

rustic restore --filter-path=/home/username/backups latest:/home/username/backups /home/username

I'll turn this issue into a discussion as it seems like it fits better there.

[tricarte]

One last question then: Is it possible to restore every path under each [[backup.sources]] section in a profile without listing the paths manually where every restored path must be from the latest snapshot of its own?

[aawsome]

I thought about using the saved path as default when restoring using rustic restore <SNAP> /restore/path.
But then:

• restore <SNAP> would behave differently to restore <SNAP>:/ and people coming from restic would expect the second behavior when specifying the first
• The behavior for multiple paths is still not defined.

So, no, there is so far no possibility.