Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: race causes `go mod download` to error with `mkdir /go/pkg/mod..: no such file or directory` #30068

Open
dnephin opened this Issue Feb 2, 2019 · 5 comments

Comments

Projects
None yet
6 participants
@dnephin
Copy link
Contributor

commented Feb 2, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11.5 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

linux/amd64

What did you do?

We run builds in a docker container as part of CI. From a container with a fresh git clone of a project we run the following:

mkdir -p /go/pkg/mod
go mod download

or

mkdir -p /go/pkg/mod
go test -v ./...

These commands will download dependencies. Frequently (roughly 1 in 3 builds) these commands will fail with an error like one of the following.

mkdir /go/pkg/mod/cache/vcs: no such file or directory
mkdir /go/pkg/mod/cache/vcs/da5145fda272732cd74527dacbe4967cc6a648dad514ccb83c288a28ea4c0671: no such file or directory
mkdir /go/pkg/mod/cache/vcs/6df073f72200185af52f4179ee546de39a8cde072c63a5c452831c700297adb1: no such file or directory

Re-running the build will eventually fix the problem. The directory is different almost every time.

What did you expect to see?

I expected to see the go: finding... output, followed by a successful download or test run.

What did you see instead?

go: github.com/mitchellh/multistep@v0.0.0-20170316185339-391576a156a5: mkdir /go/pkg/mod/cache/vcs: no such file or directory
go: github.com/anmitsu/go-shlex@v0.0.0-20161002113705-648efa622239: mkdir /go/pkg/mod/cache/vcs/6df073f72200185af52f4179ee546de39a8cde072c63a5c452831c700297adb1: no such file or directory
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/aws/aws-sdk-go v1.13.48
...
go: error loading module requirements
@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Feb 4, 2019

Probably unrelated, but #29667 is another cache locking issue (but for the build cache, not the module cache).

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

Duplicate of #26794: the go command in Go 1.11 is not safe to invoke concurrently in module mode.

Workaround is to use go1.12beta2 instead.

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

@dnephin please do let us know if you can still reproduce these failures with 1.12.

@bcmills bcmills closed this Feb 4, 2019

@yangchenyun

This comment has been minimized.

Copy link

commented Apr 8, 2019

@bcmills After upgrading to 1.12.2, the same issue still comes up from time to time in production.

Our CI and night build pipeline all observes this failure multiple times in a day.

go: github.com/erikstmartin/go-testdb@v0.0.0-20160219214506-8d10e4a1bae5: mkdir /go/pkg/mod: no such file or directory

@bcmills bcmills reopened this Apr 13, 2019

@bcmills bcmills added this to the Go1.13 milestone Apr 13, 2019

@rsc

This comment has been minimized.

Copy link
Contributor

commented Apr 24, 2019

@yangchenyun Concurrency problems here should cause problems where two processes might try to create the same directory at the same time.
But the transcript shows a directory being removed: mkdir /go/pkg/mod: no such file or directory.
That is, the mkdir failed because /go/pkg or /go does not exist.
Is something in your CI system doing rm -rf /go/pkg?
The go command itself is not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.