cmd/go: race TestScript failures on linux/amd64

[mvdan]

Seems to happen almost every run on my machine. go test -race -run TestScript/vendor seems to work as well, which is good since the entire run takes a long time.

$ go version
go version devel +fc42cf8b8c Sun Mar 3 10:33:18 2019 +0000 linux/amd64
$ go env
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOEXE=""
GOFLAGS="-ldflags=-w"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/mvdan/go"
GOPROXY=""
GORACE=""
GOROOT="/home/mvdan/tip"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build154362150=/tmp/go-build -gno-record-gcc-switches"

--- FAIL: TestScript (0.08s)
    --- FAIL: TestScript/mod_vendor_build (4.94s)
        script_test.go:188:
            # initial conditions: using sampler v1.3.0, not listed in go.mod. (1.279s)
            # update to v1.3.1, now indirect in go.mod. (1.309s)
            # vendoring can but should not need to make changes. (1.091s)
            # go list -mod=vendor (or go build -mod=vendor) must not modify go.mod.
            # golang.org/issue/26704 (1.253s)
            > go list -mod=vendor
            [stdout]
            m
            [stderr]
            ==================
            WARNING: DATA RACE
            Read at 0x00c0001a4000 by goroutine 9:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:859 +0x572
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Previous write at 0x00c0001a4000 by goroutine 8:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:870 +0x4dc
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Goroutine 9 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:161 +0x473
              cmd/go/internal/load.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1888 +0xa0
              cmd/go/internal/load.PackagesAndErrors()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1839 +0x76
              cmd/go/internal/load.Packages()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1820 +0x6a
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:424 +0x3934
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21

            Goroutine 8 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:161 +0x473
              cmd/go/internal/load.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1888 +0xa0
              cmd/go/internal/load.PackagesAndErrors()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1839 +0x76
              cmd/go/internal/load.Packages()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1820 +0x6a
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:424 +0x3934
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21
            ==================
            Found 1 data race(s)
            [exit status 66]
            FAIL: testdata/script/mod_vendor_build.txt:19: unexpected command failure

    --- FAIL: TestScript/mod_getmode_vendor (4.38s)
        script_test.go:188:
            > env GO111MODULE=on
            > go get -m rsc.io/quote@v1.5.1
            [stderr]
            go: finding rsc.io/quote v1.5.1
            go: finding rsc.io/sampler v1.3.0
            go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            > go mod vendor
            [stderr]
            go: downloading rsc.io/quote v1.5.1
            go: extracting rsc.io/quote v1.5.1
            go: downloading rsc.io/sampler v1.3.0
            go: extracting rsc.io/sampler v1.3.0
            go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            go: extracting golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            > env GOPATH=$WORK/empty
            > env GOPROXY=file:///nonexist
            > go list -mod=vendor
            [stdout]
            x
            > go list -mod=vendor -m -f '{{.Path}} {{.Version}} {{.Dir}}' all
            [stdout]
            x  $WORK/gopath/src
            golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c $WORK/gopath/src/vendor/golang.org/x/text
            rsc.io/quote v1.5.1 $WORK/gopath/src/vendor/rsc.io/quote
            rsc.io/sampler v1.3.0 $WORK/gopath/src/vendor/rsc.io/sampler
            [stderr]
            ==================
            WARNING: DATA RACE
            Read at 0x00c0001b8000 by goroutine 27:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:859 +0x572
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Previous write at 0x00c0001b8000 by goroutine 26:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:870 +0x4dc
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Goroutine 27 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ReloadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:288 +0x91
              cmd/go/internal/modload.LoadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:281 +0x38
              cmd/go/internal/modload.listModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:43 +0x47
              cmd/go/internal/modload.ListModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:20 +0x6b
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:388 +0x54e
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21

            Goroutine 26 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ReloadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:288 +0x91
              cmd/go/internal/modload.LoadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:281 +0x38
              cmd/go/internal/modload.listModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:43 +0x47
              cmd/go/internal/modload.ListModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:20 +0x6b
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:388 +0x54e
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21
            ==================
            Found 1 data race(s)
            [exit status 66]
            FAIL: testdata/script/mod_getmode_vendor.txt:9: unexpected command failure

The tests just hang after that (10m timeout), and I haven't tried to see if a longer timeout would give any different result. /cc @bcmills

[bcmills]

Argh. I've been wanting to better-encapsulate access to the build list anyway, so this will give me some properties to fix in the process.

Thanks for the report.

[gopherbot]

Change https://golang.org/cl/170597 mentions this issue: cmd/go/internal/lockedfile: add a sync.Mutex to lockedfile.Mutex

[gopherbot]

Change https://golang.org/cl/170598 mentions this issue: cmd/go/internal/modload: fix aliasing bug in (*mvsReqs).Required with -mod=vendor