Windows backend can really delete files in some conditions

[takluyver]

This is a follow up to an issue opened on Jupyter: jupyter/notebook#3631

send2trash on Windows uses the SHFileOperation interface with the FOF_ALLOWUNDO flag to move files to the recycle bin. However, the Windows docs only say that this means "Preserve undo information, if possible", and a couple of SO questions seem to confirm that that means files may be properly deleted in some conditions:

• https://stackoverflow.com/questions/1911907/what-does-shfileoperation-do-when-the-recycle-bin-is-full
• https://stackoverflow.com/questions/23720519/how-to-safely-delete-folder-into-recycle-bin

In particular, it seems that there may be a maximum percentage of disk space for the recycle bin, and deleting files bigger than the available space might cause them to be really deleted. This is a pretty important caveat!

The second of those SO questions and this linked codeguru thread mentions a few possible ways around this, none of them great:

• The FOF_WANTNUKEWARNING will display an interactive Windows 'are you sure' dialog box. But that assumes that there's an interactive local user to respond to the dialog
• There's apparently some way to do it with the newer IFileOperation interface, but the sample code is pretty complicated in C++; I'm not volunteering to do that through ctypes. And the author made a cryptic comment on SO that what he had found didn't cover all cases, without mentioning which cases were affected.
• There's an undocumented IRecycleBinManager interface that can apparently do it. But... undocumented!

So I've no idea what should happen, but I thought it was worth opening an issue here about it.

[Atero]

Hi,
Given your observation "In particular, it seems that there may be a maximum percentage of disk space for the recycle bin, and deleting files bigger than the available space might cause them to be really deleted. " It seems that there is no way back for recovering ? No command or function whatsoever that can help here ?

Thanks in advance

[takluyver]

Recovery tools may still be able to help: deleting a file doesn't usually overwrite the data, it just marks the area as not in use. So recovery tools work by scanning 'empty' disk space for data that looks like files. But it involves a degree of luck: once the disk space is marked as empty, a new file could be written in that space, which replaces the old data. And I don't know enough about recovery tools to help you with them.

[arsenetar]

I started looking into IFileOperation due to another issue, and was able to get it working. I am not sure what the best way to implement the check for the delete actually working (as not sure how to setup a test scenario). I was able to get the IFileOperationSink, which appears to at least be partially involved in some of those links on how to handle this, to work as well. All of this was using pywin32 to interface to IFileOperation. Still some work to be done.

[arsenetar]

Looks like IFileOperationSink is at least the answer, I noticed Qt 5.15 also picked up a cross platform trash/recycle interface. It looks like they have used this to prevent deletion https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_win.cpp#n493. I will be making a PR to adapt that for the IFileOperation version (currently for Vista and newer and requires pywin32).

[arsenetar]

Have confirmed in recent work that IFileOperationSink can detect when the file would be deleted, but I was not able to get it to stop the operation. I need to look into it a bit more.