Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: test, list etc. fails when run in parallel on Windows #27372

Closed
bep opened this issue Aug 30, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@bep
Copy link
Contributor

commented Aug 30, 2018

The workaround is straightforward (serialize build), but I think this deserves an issue as 1) this worked fine in Go 1.10 and 2) people in the same boat as me can get something to search for when scratching their head

Also note that this works fine on macOS and Linux.

Environment:

go version
go version go1.11 windows/amd64
go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\appveyor\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\GOPATH\
set GOPROXY=
set GORACE=
set GOROOT=C:\go
set GOTMPDIR=
set GOTOOLDIR=C:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\appveyor\AppData\Local\Temp\1\go-build440396294=/tmp/go-build -gno-record-gcc-switches
  1. The commit that fixed the build:

gohugoio/hugo@ea8ef57

  1. Extract of the error log:
# github.com/gohugoio/hugo/common/maps
bufferpool\bufpool_test.go:17:2: unknown import path "github.com/stretchr/testify/assert": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/maps [setup failed]
# github.com/gohugoio/hugo/common/math
package github.com/gohugoio/hugo/common/math (test)
	imports github.com/alecthomas/assert
	imports github.com/alecthomas/repr: unknown import path "github.com/alecthomas/repr": target directory C:\GOPATH\pkg\mod\github.com\alecthomas\repr@v0.0.0-20180724025338-ead21651dde3 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/math [setup failed]
# github.com/gohugoio/hugo/common/types
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/types [setup failed]
# github.com/gohugoio/hugo/config
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config [setup failed]
# github.com/gohugoio/hugo/config/privacy
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config/privacy [setup failed]
# github.com/gohugoio/hugo/config/services
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config/services [setup failed]
# github.com/gohugoio/hugo/create
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty

/cc @natefinch

@bcmills

This comment has been minimized.

Copy link
Member

commented Aug 30, 2018

Please provide more detail. I don't see answers for the two most important questions from the issue template:

What did you do?

What did you expect to see?

@bep

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2018

Please provide more detail. I don't see answers for the two most important questions from the issue template:

It's in there if you read more carefully. I have spent enough time on this issue. Use my issue report to improve your software/documentation, or don't.

@FiloSottile FiloSottile added the modules label Aug 30, 2018

@bep

This comment has been minimized.

Copy link
Contributor Author

commented Sep 6, 2018

I have another case, which is more problematic (and I think a more common problem), now in a Linux container on CircleCI:

See error log below.

da00d00bdef65874ff644eaa26: exit status 128:
	fatal: cannot copy '/usr/share/git-core/templates/hooks/update.sample' to '/go/pkg/mod/cache/vcs/cb1953cbdfd14fc2ffce4dfd06487e8d5a0c96da00d00bdef65874ff644eaa26/hooks/update.sample': File exists
go: github.com/dsnet/golib@v0.0.0-20171103203638-1ea166775780: git init --bare in /go/pkg/mod/cache/vcs/e56cb0fb3504244c08f9270d26097e67273a5d49f0367a918d5322d12df3a9b8: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/e56cb0fb3504244c08f9270d26097e67273a5d49f0367a918d5322d12df3a9b8/config: No such file or directory
	fatal: could not set 'core.symlinks' to 'false'
go: golang.org/x/sync@v0.0.0-20180314180146-1d60e4601c6f: git init --bare in /go/pkg/mod/cache/vcs/55179c5d8c4db2eaed9fae4682d4c84a1fd3612df666b372bef3bbb997c9601f: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/55179c5d8c4db2eaed9fae4682d4c84a1fd3612df666b372bef3bbb997c9601f/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: golang.org/x/net@v0.0.0-20180826012351-8a410e7b638d: git init --bare in /go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: gopkg.in/yaml.v2@v2.2.1: git init --bare in /go/pkg/mod/cache/vcs/4f939cf8921e8662ef2d8cf78b0db3bcacccda0d79c9e1b4cef06c640b4a8cef: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/4f939cf8921e8662ef2d8cf78b0db3bcacccda0d79c9e1b4cef06c640b4a8cef/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: golang.org/x/text@v0.3.0: git remote add origin https://go.googlesource.com/text in /go/pkg/mod/cache/vcs/5b03666c2d7b526129bad48c5cea095aad8b83badc1daa202e7b0279e3a5d861: exit status 128:
	error: could not lock config file config: File exists
	fatal: could not set 'remote.origin.url' to 'https://go.googlesource.com/text'
go: error loading module requirements

/cc @caarlos0

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 6, 2018

Which go commands were you trying to run in parallel, and which of those failed?
Did you run go mod tidy and/or go mod download beforehand?

@bep

This comment has been minimized.

Copy link
Contributor Author

commented Sep 6, 2018

@bcmills the links above are to the actual scripts with reference to Docker image used etc. For the last one, I suspect a go mod download before starting the parallel builds would have done the trick.

@caarlos0

This comment has been minimized.

Copy link

commented Sep 10, 2018

@bep yes, parallel go mod download will likely not work... this is a WIP on GoReleaser side as well: goreleaser/goreleaser#779

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 11, 2018

Closing as a duplicate of #26794.

@bcmills bcmills closed this Sep 11, 2018

wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018

Simplify travis-ci build
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.

wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018

Simplify travis-ci build
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.

wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018

Simplify travis-ci build
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.