Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

Closed
bcmills opened this issue Mar 2, 2022 · 9 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows release-blocker
Milestone

Comments

@bcmills
Copy link
Member

bcmills commented Mar 2, 2022

--- FAIL: TestRWithNonexistentFile (0.83s)
    testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\gopher\AppData\Local\Temp\1\TestRWithNonexistentFile1599077814\001\pack.exe: The process cannot access the file because it is being used by another process.
FAIL
FAIL	cmd/pack	4.916s

This can probably be fixed by expanding the set of errors handled by the retry loop added for #50051.

greplogs --dashboard -md -l -e 'TempDir RemoveAll cleanup: .* The process cannot access the file because it is being used by another process\.'

2022-02-25T23:38:11-01e522a/windows-amd64-2012
2021-10-28T18:10:50-2bc8ed8/windows-amd64-2012
2021-10-14T18:43:59-765c911/windows-amd64-2012
2021-09-30T19:56:27-70b1a45/windows-amd64-2012
2021-09-17T19:32:44-74e384f/windows-amd64-2012
2021-03-09T08:19:14-b70a2bc/windows-386-2008
2021-03-09T08:19:14-b70a2bc/windows-amd64-2008
2021-03-09T08:19:14-b70a2bc/windows-amd64-2012
2021-03-09T08:19:14-b70a2bc/windows-amd64-2016
2021-03-09T08:19:14-b70a2bc/windows-amd64-longtest
2021-03-09T08:19:14-b70a2bc/windows-amd64-race
2021-03-09T08:19:14-b70a2bc/windows-arm-zx2c4
2021-03-09T07:44:18-437d229/windows-amd64-2008
2021-03-09T07:44:18-437d229/windows-amd64-2012
2021-03-09T07:44:18-437d229/windows-amd64-race
2021-03-09T07:44:18-437d229/windows-arm-zx2c4
2021-03-09T02:20:55-618b66e/windows-amd64-2008
2021-03-09T02:20:55-618b66e/windows-amd64-2012
2021-03-09T02:20:55-618b66e/windows-amd64-2016
2021-03-09T02:20:55-618b66e/windows-amd64-race
2021-03-09T02:20:55-618b66e/windows-arm-zx2c4
2021-03-08T23:13:52-b6def6a/windows-amd64-2008
2021-03-08T23:13:52-b6def6a/windows-amd64-2012
2021-03-08T23:13:52-b6def6a/windows-amd64-2016
2021-03-08T23:13:52-b6def6a/windows-amd64-race
2021-03-08T20:41:06-a08adda/windows-386-2008
2021-03-08T20:41:06-a08adda/windows-amd64-2008
2021-03-08T20:41:06-a08adda/windows-amd64-2016
2021-03-08T20:41:06-a08adda/windows-amd64-longtest
2021-03-08T20:41:06-a08adda/windows-amd64-race
2021-03-08T20:03:01-bd37284/windows-386-2008
2021-03-08T20:03:01-bd37284/windows-amd64-2008
2021-03-08T20:03:01-bd37284/windows-amd64-2012
2021-03-08T20:03:01-bd37284/windows-amd64-2016

@bcmills bcmills changed the title testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process.| testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." Mar 2, 2022
@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 2, 2022
@bcmills bcmills added this to the Go1.19 milestone Mar 2, 2022
@bcmills bcmills self-assigned this Mar 2, 2022
@gopherbot gopherbot removed the NeedsFix The path to resolution is known, but the work has not been done. label Mar 2, 2022
@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. release-blocker labels Mar 2, 2022
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 2, 2022
@bcmills
Copy link
Member Author

bcmills commented Mar 2, 2022

(This error message is 0x20 ERROR_SHARING_VIOLATION.)

@gopherbot
Copy link

gopherbot commented Mar 2, 2022

Change https://go.dev/cl/388916 mentions this issue: testing: include ERROR_SHARING_VIOLATION in Windows cleanup retries

@bhasden
Copy link

bhasden commented May 18, 2022

Apologies for commenting on a closed issue. However, I wanted to ask if there is any chance this makes its way out prior to 1.19?

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented May 18, 2022

@bhasden Are you encountering this yourself? What Go version and what GOOS/GOARCH? Thanks.

@bcmills
Copy link
Member Author

bcmills commented May 18, 2022

FWIW, this fix would be easy to backport to Go 1.18, but somewhat less trivial to backport to Go 1.17 (it would also need CL 371296, which is a bit more invasive).

There is also a workaround that you can apply for your own specific tests independent of the Go version: you can defer (or t.Cleanup) your own function that iterates os.RemoveAll on the directory returned by TempDir until it is successfully removed.

@bhasden
Copy link

bhasden commented May 18, 2022

@bhasden Are you encountering this yourself? What Go version and what GOOS/GOARCH? Thanks.

I believe I'm running into this same issue in Golang 1.18.1 and 1.18.2 (only two versions I've tested) with some casbin tests that are loading a model configuration file. It's only reproducible in Windows. Team members on non-Windows machines are not running into the issue.

go env output:
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\BrianHasden\AppData\Local\go-build
set GOENV=C:\Users\BrianHasden\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\BrianHasden\go\pkg\mod
set GONOPROXY=######
set GONOSUMDB=######
set GOOS=windows
set GOPATH=C:\Users\BrianHasden\go
set GOPRIVATE=######
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\BrianHasden\go\go1.18.2
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\BrianHasden\go\go1.18.2\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=######
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\BRIANH~1\AppData\Local\Temp\go-build2379612813=/tmp/go-build -gno-record-gcc-switches

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented May 19, 2022

@gopherbot Please open backport to 1.18.

At least one person is encountering this problem in 1.18. There are workarounds but they are awkward. The fix would be harder to backport to earlier releases.

@gopherbot
Copy link

gopherbot commented May 19, 2022

Backport issue(s) opened: #52986 (for 1.18).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@gopherbot
Copy link

gopherbot commented May 23, 2022

Change https://go.dev/cl/407877 mentions this issue: [release-branch.go1.18] testing: include ERROR_SHARING_VIOLATION in Windows cleanup retries

@rsc rsc unassigned bcmills Jun 22, 2022
gopherbot pushed a commit that referenced this issue Jul 25, 2022
…indows cleanup retries

Fixes #52986
Updates #51442
Updates #50051

Change-Id: I1bfbc08c907077467fd50febbec6299a9b73af41
Reviewed-on: https://go-review.googlesource.com/c/go/+/388916
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit eeb9f09)
Reviewed-on: https://go-review.googlesource.com/c/go/+/407877
Reviewed-by: Nooras Saba‎ <saba@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
nao1215 added a commit to nao1215/gup that referenced this issue Sep 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows release-blocker
Projects
None yet
Development

No branches or pull requests

4 participants