Handle removing files better

[Bodigrim]

Closes #11606.

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

[zlonast]

In Haskell, we have very poor support for Windows operating system errors. The libraries provided by the compiler don't have a comprehensive manual catch for these errors for Windows. Essentially, we only catch errors on Linux and Mac (so-called errno). I think this will also silently crash, like renameFileWithRetry.

[zlonast]

Personally, I like this pull request, just like renameFileWithRetry. But it doesn't prevent the silent death of the build on Windows.