"Stop after current file" with --usn-policy may never back up remaining files

[ts678]

• I have searched open and closed issues for duplicates.

---

Environment info

• Duplicati version: 2.0.4.32
• Operating system: Windows 10 Version 1903
• Backend: Local folder

Description

The "Stop after current file" ability added in v2.0.4.29-2.0.4.29_canary_2019-09-17 seems incompatible with --usn-policy using USN. Files which are not backed up before the Stop might be skipped forever.

See 2.0.4.31 “Stop after current file” with Windows USN never backs up some files on future backups, which speculated on cause, and invited user tests which didn't happen, so here's a formal bug report. Some of the specifics are erratic, but it's reproducible enough and sticky enough to get some debug.

Steps to reproduce

1. Basic recipe is to start a backup that isn't too instant, Stop, then Backup more times. My test was:
2. Set --usn-policy=Required after making sure that Duplicati is running with Administrator privilege
3. Use a copy of C:\Program Files\Duplicati 2 as test material. Use File Explorer to measure total size.
4. Slow with Remote Volume Size at 1 MB, --throttle-download and --throttle-upload at 100 KByte/s
5. To spot when to Stop, and see other info, open a tab to About --> Show log --> Live --> Verbose
6. "Run now" Backup, go to live log tab, wait for uploading to begin, then do "Stop after current file"
7. Home screen "Source" size will be less than total. "Restore" dropdown will show partial. OK so far.
8. "Run now" again. Not much changes.
9. Try to get it to stop being partial by adding one-byte .txt file to the folder that's being backed up.
10. "Run now" again.

• Actual result:
  Backup may (often but not always) declare success, no longer be partial, but be missing many files.
• Expected result:
  Files are backed up.

Screenshots

Debug log

[drwtsn32x]

Was easily able to reproduce problem. I think this should be an easy fix... testing now....

[drwtsn32x]

PR #3995 has been merged... @ts678 are you able to try now or do you need to wait for the next release?

[ts678]

I'm not set up to do builds. This is the sort of pre-release testing that a nightly build would help with.

[drwtsn32x]

Yeah, nightly builds sounds like a nice idea

BTW if you have a Windows machine you can get Visual Studio 2019 Community Edition for free. That's what I did. I can give you pointers if you want to try it out.

[warwickmm]

I'm going to close this since the pull request was merged. We can reopen this if we find that there are some unresolved issues.