Inform about downloading index

[rawtaz]

When using restic on a "slow" connection, it takes a while to download the index form the remote repository, and during this time restic just sits there without saying anything, leaving the user in the dark unless they check and see that there is much active network traffic.

It would be nice if restic at least notified the user that it's working on downloading the index, and even better output a progress/percentage during that time to keep the user updated. Something along the lines of Downloading index... 42%.

[fd0]

Yes, this is something we definitely need to work on, it also applies to other operations.

[fd0]

Still relevant :/

[arjunajesh]

@MichaelEischer Can you point me towards what functions I should look at to implement the percentage of index files downloaded

[MichaelEischer]

@arjunajesh The index loading is implemented at internal/repository/repository.go in the method LoadIndex. That method would have to take an additional progress.Counter parameter (see internal/ui/progress/counter.go). Most commands will be able to use the counter implementation from cmd/restic/progress.go. For the backup and restore command, we'll need some adapter code which converts the progress bar output into something suitable for the termstatus.Terminal that is used to handle the output for these two commands.

[arjunajesh]

@MichaelEischer Here is a solution that I came up with. I use the repo.list method to count how many index files exist in the backend. I was able to display a progress bar with this code. Is there a simpler way to get the number of index files?

repo.List(ctx, restic.IndexFile, func(id restic.ID, size int64) error {
    numIndexFiles += 1
    return nil
})
bar := newProgressMax(!gopts.Quiet, uint64(numIndexFiles), "index files loaded")
defer bar.Done()

err = repo.LoadIndexModified(ctx, bar)

Additionally, could you elaborate on how the adapter code should work for backup and restore command? How should I write to the termstatus.Terminal?

[MichaelEischer]

Is there a simpler way to get the number of index files?

Not really, but the index counting can be moved inside LoadIndex: The Counter used for the progress bar has a SetMax method which allows updating the limit. That is the progress bar can be created with a hardcoded maximum of 0, which is afterwards updated by LoadIndex.

Additionally, could you elaborate on how the adapter code should work for backup and restore command? How should I write to the termstatus.Terminal?

newProgressMax uses a progress.Func that directly prints to os.Stdout. It might be enough to create an alternative implementation of newProgressMax that uses something like term.SetStatus([]string{formattedProgress}) to print the output. As both functions would return a progress.Counter, this would then allow LoadIndex to work with either one.

[arjunajesh]

Thanks for the help, I am getting close to a PR. I just need some clarification on how I should handle all the calls of LoadIndex (which now takes an additional progress.Counter) in /internal. The implementation of newProgressMax is in cmd/restic/progress.go and so I can't use it in a lot of the test files that call LoadIndex in the internal directory. My instinct is to modify LoadIndex to handle nil for progress.Counter parameter, but I wanted to ask before I implement this. Thanks!

[MichaelEischer]

My instinct is to modify LoadIndex to handle nil for progress.Counter parameter, but I wanted to ask before I implement this.

@arjunajesh For most tests using nil in place of a progress.Counter is the best solution.