New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On Windows, Remove-Item -Recurse is unexpectedly asynchronous with directories #8211

Open
mklement0 opened this Issue Nov 8, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@mklement0
Contributor

mklement0 commented Nov 8, 2018

Update: The problem affects CoreFx too, so the problem is likely at a lower level; I'll update this issue with more findings later.


Note: The problem occurs only on Windows (and there also in Windows PowerShell).

The infinite loop below errors out after a few seconds, because Remove-Item -Recurse hasn't fully finished removal of the target directory before New-Item tries to recreate it.

Steps to reproduce

While ($true) { Write-Host -NoNewline .
                $null = New-Item -EA Stop -Type Directory tmpDir
                1..10 | % { $null > tmpDir/$_ }; Remove-Item -EA Stop -Recurse tmpDir }

Expected behavior

In the absence of the bug, the loop should run forever (terminate it with Ctrl+C), emitting a . in each iteration.

Actual behavior

After a few seconds, the command terminates with an error message such as the following;

New-Item : An item with the specified name C:\Users\jdoe\tmpDir already exists.

While the symptom in Windows PowerShell is fundamentally the same, the error message differs:

New-Item : Access is denied ... ItemExistsUnauthorizedAccessError

Environment data

PowerShell Core 6.2.0-preview.1 on Microsoft Windows 10 Pro (64-bit; Version 1803, OS Build: 17134.165)
Windows PowerShell v5.1.17134.228 on Microsoft Windows 10 Pro (64-bit; Version 1803, OS Build: 17134.345)
@SteveL-MSFT

This comment has been minimized.

Member

SteveL-MSFT commented Nov 8, 2018

Is this potentially a problem with when the file system itself decides to flush?

@mklement0

This comment has been minimized.

Contributor

mklement0 commented Nov 8, 2018

Seems like it, @SteveL-MSFT.

I'm still looking into this, but it seems that the WinAPI RemoveDirectory() function is inherently asynchronous, which is why .NET and cmd are equally affected.

From the docs:

"The RemoveDirectory function marks a directory for deletion on close. Therefore, the directory is not removed until the last handle to the directory is closed."

So it may come down to the following choice for us:

  • Live with the behavior (as cmd and .NET do) and and document it (which cmd and .NET have not done).

  • As a courtesy - if technically feasible and reasonable - implement the synchronous behavior ourselves.

I'm looking into the latter option, as it seems preferable.

@SteveL-MSFT

This comment has been minimized.

Member

SteveL-MSFT commented Nov 8, 2018

@mklement0 is this something you hit on a regular basis outside of your synthetic test?

@mklement0

This comment has been minimized.

Contributor

mklement0 commented Nov 8, 2018

@SteveL-MSFT: As I now realize, the problem is actually more fundamental and more insidious, due to its intermittent, unpredictable nature:

Because file deletion is equally asynchronous, Remove-Item -Recurse itself fails intermittently - see https://serverfault.com/q/199921/176094

And, yes, I have seen such failures in the wild.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment