chore(ic-backup): don't hold locks during sleeps between rsync retries and ignore *.tmp files while rsyncing

[kpop-dfinity]

1. When a replica backs up an artifact in the backup partition, it first creates a temporary file $dst.tmp, writes to it, and only then renames the file to $dst (see

   ic/rs/artifact_pool/src/backup.rs

   Line 583 in 2450f6d

   ic_sys::fs::write_using_tmp_file(full_path, |writer| writer.write_all(&serialized))

   ).
   This means that if we attempt to rsync the content of the backup partition, we might run into errors because we might initiate a download while a tmp file (very temporarily) exists. To avoid running into file has vanished errors, we propose to ignore all *.tmp files.
2. Currently, we acquire an exclusive lock for the whole duration of rsync_spool function, including the delays between retries after failures. We propose to hold the lock only for the duration of the rsync command, and drop the lock immediately if the rsync command fails.

This should help with deflaking //rs/tests/consensus/backup:backup_manager_{downgrade|upgrade}_test

[pierugo-dfinity]

Nice