Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go test --race' needs gcc for main packages on windows #27089

Open
judwhite opened this issue Aug 19, 2018 · 19 comments

Comments

Projects
None yet
7 participants
@judwhite
Copy link

commented Aug 19, 2018

What version of Go are you using (go version)?

go version go1.11rc1 windows/amd64

Does this issue reproduce with the latest release?

  • Yes, reproduces with go 1.11rc1
  • No, doesn't reproduce with go 1.10.3

What operating system and processor architecture are you using (go env)?

windows/amd64
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\jud_white\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Projects\Go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\JUD_WH~1\AppData\Local\Temp\go-build672193350=/tmp/go-build -gno-record-gcc-switches

What did you do?

$ go test -cpu 1,2,4 -count 1 --race ./hello/...
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
FAIL    git.xyz.com/{org}/{repo}/cmd/hello [build failed]

$ go test -cpu 1,2,4 -count 1 ./hello/...
ok      git.xyz.com/{org}/{repo}/cmd/hello   0.105s

It doesn't happen for non-main packages:

$ go test -cpu 1,2,4 -count 1 --race ./hellopkg/...
ok      git.xyz.com/{org}/{repo}/pkg/hellopkg      10.220s

--race by itself is sufficient to repro, you don't need to set the CPU count or disable caching.

What did you expect to see?

ok      git.xyz.com/{org}/{repo}/cmd/hello   0.XYZs

What did you see instead?

# runtime/cgo
exec: "gcc": executable file not found in %PATH%
FAIL    git.xyz.com/{org}/{repo}/cmd/hello [build failed]
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2018

It's true that the race detector requires GCC to be available. There is no workaround at present.

@judwhite

This comment has been minimized.

Copy link
Author

commented Aug 20, 2018

Does this issue reproduce with the latest release?

@ianlancetaylor Maybe the answer to this is "No", it doesn't reproduce with go 1.10.3. It's also interesting this only happens with main packages.

Specifically this prevents go test --race ./... on Windows when there are main packages and gcc isn't installed.

@judwhite

This comment has been minimized.

Copy link
Author

commented Aug 21, 2018

@dvyukov Any advice?

@iWdGo

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2018

If you set CGO_ENABLED=0 as required to build w/o gcc, gets you:
go test: -race requires cgo; enable cgo by setting CGO_ENABLED=1
It seems that the failure in 1.10 was silent.

@dvyukov

This comment has been minimized.

Copy link
Member

commented Aug 21, 2018

If this works with go 1.10, I think we need to figure out what change broke this and why. Perhaps somebody on Go team knows? Or otherwise bisect.

@iWdGo

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2018

The workaround is to set -vet=off when the name of the package is main.
I just installed rc2: go version go1.11rc2 windows/amd64
This seems to point to #26451 which changed the vet behavior on Windows in relation to -c flag.

I tested the data race detection in the same environment with relevant env:

set GOARCH=amd64
set GOOS=windows
set GOROOT=C:\Go
set GCCGO=gccgo
set CC=gcc
set CGO_ENABLED=1
set GOMOD=

The flag to be tested is -race as documented even if --race is usable.

As far as I remember, gcc is not installed on my laptop. It is maybe available because of Google Cloud SDK which includes Python but I doubt so as I cannot reach it any way that I tried. The documentation https://golang.org/doc/articles/race_detector.html#Supported_Systems does not mention a prerequisite for Windows.

I ran the race detector on the example adapted for go test: https://play.golang.org/p/ZTZcFHchMiI
The test is simply:

package main

import "testing"

func TestFwrace(t *testing.T) {
	fwrace()
}

The results are (GOPATH in place of its value):

GOPATH\CloudSDK\sandbox\src>go test -race ./racedetect
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
FAIL    GOPATH/CloudSDK/sandbox/src/racedetect [build failed]

GOPATH\CloudSDK\sandbox\src>go test -race -vet=off ./racedetect
949.4737ms
==================
WARNING: DATA RACE
Read at 0x00c000076020 by goroutine 8:
  GOPATH/CloudSDK/sandbox/src/racedetect.fwrace.func1()
      GOPATH/CloudSDK/sandbox/src/racedetect/fwrace.go:14 +0xe4

Previous write at 0x00c000076020 by goroutine 6:
  GOPATH/CloudSDK/sandbox/src/racedetect.fwrace()
      GOPATH/CloudSDK/sandbox/src/racedetect/fwrace.go:12 +0x16e
  GOPATH/CloudSDK/sandbox/src/racedetect.TestFwrace()
      GOPATH/CloudSDK/sandbox/src/racedetect/fwrace_test.go:27 +0x36
  testing.tRunner()
      C:/Go/src/testing/testing.go:827 +0x169

Goroutine 8 (running) created at:
  time.goFunc()
      C:/Go/src/time/sleep.go:172 +0x58

Goroutine 6 (running) created at:
  testing.(*T).Run()
      C:/Go/src/testing/testing.go:878 +0x657
  testing.runTests.func1()
      C:/Go/src/testing/testing.go:1119 +0xaf
  testing.tRunner()
      C:/Go/src/testing/testing.go:827 +0x169
  testing.runTests()
      C:/Go/src/testing/testing.go:1117 +0x4f5
  testing.(*M).Run()
      C:/Go/src/testing/testing.go:1034 +0x2f5
  main.main()
      _testmain.go:42 +0x228
==================
1.0324072s
1.7000447s
1.9359093s
2.2237464s
2.7734292s
3.4070538s
3.7388592s
3.9227717s
4.4034788s
--- FAIL: TestFwrace (5.00s)
    testing.go:771: race detected during execution of test
FAIL
FAIL    _/C_/Users/Costa/Documents/Google/CloudSDK/sandbox/src/racedetect       6.012s

(edited for formatting)

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Aug 25, 2018

No, doesn't reproduce with go 1.10.3

@judwhite that is not what I see here with go 1.10.4

c:\Users\Alex\dev\go1.10\src>go version
go version go1.10.4 windows/amd64

c:\Users\Alex\dev\go1.10\src>go test --race strings
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
FAIL    strings [build failed]

c:\Users\Alex\dev\go1.10\src>

Alex

@judwhite

This comment has been minimized.

Copy link
Author

commented Aug 25, 2018

D:\go\src\git.dell.com\premier\motionci>go version
go version go1.10.3 windows/amd64

D:\go\src\git.dell.com\premier\motionci>go test ./cmd/motionci-server
ok      git.dell.com/premier/motionci/cmd/motionci-server       2.873s

D:\go\src\git.dell.com\premier\motionci>go test -race ./cmd/motionci-server
ok      git.dell.com/premier/motionci/cmd/motionci-server       4.105s

D:\go\src\git.dell.com\premier\motionci>gcc
'gcc' is not recognized as an internal or external command,
operable program or batch file.

D:\go\src\git.dell.com\premier\motionci>go version
go version go1.11 windows/amd64

D:\go\src\git.dell.com\premier\motionci>go test ./cmd/motionci-server
ok      git.dell.com/premier/motionci/cmd/motionci-server       2.925s

D:\go\src\git.dell.com\premier\motionci>go test -race ./cmd/motionci-server
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
FAIL    git.dell.com/premier/motionci/cmd/motionci-server [build failed]

I've been using the race detector in CI for quite a while, this is the first Go version (1.10.4 and 1.11) where this issue has been encountered. I'll use the -vet off workaround for now, thanks @iWdGo for looking into it.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Aug 26, 2018

@judwhite as you can see from my previous message, my experience is different.

So, if you want me to investigate this, you have to provide more details. I need step by step instructions of how to reproduce this. Please, assume, I have brand new Windows computer with nothing installed on it.

Thank you.

Alex

@iWdGo

This comment has been minimized.

Copy link
Contributor

commented Aug 26, 2018

This issue seems to have commonalities with others that I have. There is maybe a slight difference of behavior between Windows 10 Pro and Windows 10 Home (my version).

For instance, make.bat has been modified by these pull requests.
https://go-review.googlesource.com/131397 and https://go-review.googlesource.com/124936
The described issues never occurred on my computer.

Ema93sh added a commit to cloudfoundry/cli that referenced this issue Nov 14, 2018

Remove -race from windows units temporarily
Because of golang/go#27089, passing "-race" to
ginkgo causes the windows build to fail. We are simply removing this
flag until the golang issue is resolved. (Units on other operating
systems still check for race conditions.)

[#160911621]

Signed-off-by: Will Murphy <wmurphy@pivotal.io>

XenoPhex added a commit to cloudfoundry/cli that referenced this issue Nov 14, 2018

Remove -race from windows units temporarily on V7 side.
Because of golang/go#27089, passing "-race" to
ginkgo causes the windows build to fail. We are simply removing this
flag until the golang issue is resolved. (Units on other operating
systems still check for race conditions.)

This change is to parallel fe63a26.

[#160911621]

@bradfitz bradfitz added the OS-Windows label Nov 21, 2018

@femot

This comment has been minimized.

Copy link

commented Jan 23, 2019

I made a minimal, self-contained example: https://github.com/femot/race-no-gcc

fail/fail.go:

package main

func main() {}

Results in output:

PS D:\gopath\src\github.com\femot\race-no-gcc> go test -race ./fail
# runtime/cgo
exec: "gcc": executable file not found in %PATH%

This is true, I do not have gcc or any other C compiler on my machine.

But we can "fix" it:

PS D:\gopath\src\github.com\femot\race-no-gcc> go test -race -vet=off ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]

Let us try without a main package..
success/success.go:

package success

func main() {}
PS D:\gopath\src\github.com\femot\race-no-gcc> go test -race ./success
?       github.com/femot/race-no-gcc/success    [no test files]

Works just fine!

I am willing to try out things and provide additional information, if required.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Jan 27, 2019

I am willing to try out things and provide additional information, if required.

@femot I agree, go test --race command requires gcc, for both go1.10 and and go1.11. I don't need help reproducing this.

But, according to @judwhite, go1.10 does not requires gcc. I need steps that demonstrate go1.10 does not requires gcc. Do you know how to reproduce that?

Thank you.

Alex

@femot

This comment has been minimized.

Copy link

commented Jan 30, 2019

But, according to @judwhite, go1.10 does not requires gcc. I need steps that demonstrate go1.10 does not requires gcc. Do you know how to reproduce that?

Just tested it with 1.10 and 1.10.8 on a relatively fresh Windows 10 VM (no Go or other compilers/programming environments previously installed).

  • Installed Git (2.20.1.windows.1)
  • Install 1.10
  • Run the following:
PS C:\Users\aczeczka> go version
go version go1.10 windows/amd64
PS C:\Users\aczeczka> go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\aczeczka\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\aczeczka\go
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\aczeczka\AppData\Local\Temp\go-build255366898=/tmp/go-build -gno-record-gcc-switches
PS C:\Users\aczeczka> go get github.com/femot/race-no-gcc
package github.com/femot/race-no-gcc: no Go files in C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc
PS C:\Users\aczeczka> cd .\go\src\github.com/femot/race-no-gcc
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./success
?       github.com/femot/race-no-gcc/success    [no test files]
  • Install 1.10.8
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go version
go version go1.10.8 windows/amd64
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./success
?       github.com/femot/race-no-gcc/success    [no test files]

This is the behaviour I expected/am used to: being able to go test -race in main packages without any C compiler installed.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Feb 6, 2019

Just tested it with 1.10 and 1.10.8 ...
...
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./fail
? github.com/femot/race-no-gcc/fail [no test files]
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./success
? github.com/femot/race-no-gcc/success [no test files]

Your example doesn't have any tests. I suspect your go test -race ... command will succeed with latest version of Go too.

Alex

@femot

This comment has been minimized.

Copy link

commented Feb 6, 2019

As demonstrated previously, no tests are required to reproduce this behaviour with Go 1.11. This means go test -race will fail and complain about gcc, while there aren't even any test files present.

I updated Go on the machine where I did the tests with 1.10 and 1.10.8 and ran the commands again:

PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go version
go version go1.11.5 windows/amd64
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./success
?       github.com/femot/race-no-gcc/success    [no test files]
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./fail
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race -vet=off ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]

There were no other changes to any files, installed programs, or the system.

To me it looks like the -race option triggers go vet to be run and something about main packages requires gcc. This happens before any checks for tests files.

This is why the example I provided contains no tests files. The existence of test files makes no difference and is not needed to trigger this behaviour.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Feb 7, 2019

and ran the commands again:

That is not what I see here

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>gcc
'gcc' is not recognized as an internal or external command,
operable program or batch file.

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>go version
go version devel +8b7cf898af Fri Jan 25 04:57:20 2019 +0000 windows/amd64

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>go test -race ./success
?       github.com/femot/race-no-gcc/success    [no test files]

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>go test -race ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>go test -race -vet=off ./fail
?       github.com/femot/race-no-gcc/fail       [no test files]

c:\Users\Alex\dev\src\github.com\femot\race-no-gcc>

Alex

@femot

This comment has been minimized.

Copy link

commented Feb 7, 2019

Ok this is weird. I have two different installations here that are able to reproduce it. I also tried 1.12beta2 after seeing that you tested on devel.

Can you post the output of go env?

In my team of 4 people, one had gcc installed, one installed gcc, because of this and I just confirmed with the last one that he also has the same behaviour:

λ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\stefanm\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=c:\dev\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\stefanm\AppData\Local\Temp\go-build413539134=/tmp/go-build -gno-record-gcc-switches
 
 C:\tools\cmder 
λ go version
go version go1.11.5 windows/amd64
 
 C:\tools\cmder 
λ go get github.com/femot/race-no-gcc
package github.com/femot/race-no-gcc: no Go files in c:\dev\go\src\github.com\femot\race-no-gcc
 
 C:\tools\cmder 
λ cd $env:gopath
The system cannot find the path specified.
 
 C:\tools\cmder 
λ cd..
 
 C:\tools 
λ cd..
 
 C:\ 
λ cd dev\go\
 
 C:\dev\go 
λ cd src\github.com\femot\race-no-gcc\
 
 race-no-gcc   master 
λ go test -race ./fail
# runtime/cgo
exec: "gcc": executable file not found in %PATH%
@alexbrainman

This comment has been minimized.

Copy link
Member

commented Feb 8, 2019

Can you post the output of go env?

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Alex\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=c:\users\alex\dev
set GOPROXY=
set GORACE=
set GOROOT=c:\users\alex\dev\go
set GOTMPDIR=
set GOTOOLDIR=c:\users\alex\dev\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Alex\AppData\Local\Temp\go-build808343958=/tmp/go-build -gno-record-gcc-switches

Alex

@femot

This comment has been minimized.

Copy link

commented Feb 15, 2019

I'm out of ideas. The only thing that looks different is that you tried it on devel, so I tried it on 1.12rc1 to see if it was fixed between 1.11 and 1.12, but it still reproduced.

PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go version
go version go1.12beta2 windows/amd64
PS C:\Users\aczeczka\go\src\github.com\femot\race-no-gcc> go test -race ./fail
# runtime/cgo
exec: "gcc": executable file not found in %PATH%

alastairs added a commit to CloudHub360/ch360.go that referenced this issue Jun 27, 2019

Fix GoLang test runs on Windows
Because of golang/go#27089, we need to disable go vet support in
go test on Windows. It's a handy tool, though, so we should leave it
enabled on Linux so it runs on the build machines, etc.

alastairs added a commit to CloudHub360/ch360.go that referenced this issue Jun 27, 2019

Fix GoLang test runs on Windows
Because of golang/go#27089, we need to disable go vet support in
go test on Windows. It's a handy tool, though, so we should leave it
enabled on Linux so it runs on the build machines, etc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.