Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: GO111MODULE=on go build requires gcc #26988

Closed
nqv opened this issue Aug 14, 2018 · 20 comments

Comments

Projects
None yet
@nqv
Copy link
Contributor

commented Aug 14, 2018

What version of Go are you using (go version)?

go version go1.11rc1 linux/amd64

Does this issue reproduce with the latest release?

Not sure

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/nqv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/nqv/go"
GOPROXY=""
GORACE=""
GOROOT="/home/nqv/Downloads/go"
GOTMPDIR=""
GOTOOLDIR="/home/nqv/Downloads/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build126805613=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Build a minimal http server failed with GO111MODULE=on go build

$ cd ~/go/src/httpd
$ cat httpd.go
package main

import "net/http"

func main() {
	http.ListenAndServe(":8000", nil)
}
$ GO111MODULE=on go mod init
go: creating new go.mod: module httpd
$ GO111MODULE=on go build
# net
exec: "gcc": executable file not found in $PATH

What did you expect to see?

GO111MODULE=on go build produces binary.

What did you see instead?

Build failed due to missing gcc.
Either of these commands works fine though:

$ go build
$ GO111MODULE=on CGO_ENABLED=0 go build
@mvdan

This comment has been minimized.

Copy link
Member

commented Aug 14, 2018

It's surprising that GO111MODULE=on go build is trying to build the net package, or any other part of the standard library. How did you install Go? Does go build std do nothing, like it should?

cc @myitcv

@myitcv

This comment has been minimized.

Copy link
Member

commented Aug 14, 2018

Edit: fixed bad copy-paste.

Reproduced:

docker run --rm -i -t ubuntu:16.04 bash
apt-get -y update < /dev/null > /dev/null
apt-get -y install wget < /dev/null > /dev/null
cd /tmp
wget -q https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
tar -zxf go1.11rc1.linux-amd64.tar.gz
export PATH=/tmp/go/bin:$PATH
cd
mkdir -p go/src/httpd
cd go/src/httpd
cat <<EOD > httpd.go
package main

import "net/http"

func main() {
http.ListenAndServe(":8000", nil)
}
EOD
GO111MODULE=on go mod init
GO111MODULE=on go build

exits non-zero with:

# net
exec: "gcc": executable file not found in $PATH

whereas:

go build

just works.

/cc @rsc @bcmills @ianlancetaylor

@nqv

This comment has been minimized.

Copy link
Contributor Author

commented Aug 14, 2018

I installed go1.11 from archive

$ cd ~/Downloads
$ curl -L -O https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
$ tar xf go1.11rc1.linux-amd64.tar.gz

FYI, go build -v outputs

$ go build -v
httpd

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/poly1305
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/curve25519
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/transform
golang_org/x/text/unicode/bidi
golang_org/x/net/http2/hpack
net
# net
exec: "gcc": executable file not found in $PATH
golang_org/x/text/unicode/norm
golang_org/x/text/secure/bidirule
golang_org/x/net/idna

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on CGO_ENABLED=0 go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/curve25519
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/poly1305
golang_org/x/text/transform
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/unicode/bidi
golang_org/x/text/unicode/norm
net
golang_org/x/net/http2/hpack
golang_org/x/text/secure/bidirule
golang_org/x/net/idna
golang_org/x/net/http/httpproxy
net/textproto
crypto/x509
golang_org/x/net/http/httpguts
crypto/tls
net/http/httptrace
net/http
httpd
@rsc

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2018

It's not that surprising, and it's fine for Go 1.11.

@pda

This comment has been minimized.

Copy link

commented Aug 28, 2018

I find it surprising. It's a pain when there's no gcc.
Here's a minimal reproduction:

$ docker run --rm -w /notgo golang:1.11-alpine sh -c 'go mod init x && go build std'
go: creating new go.mod: module x
# net
exec: "gcc": executable file not found in $PATH

It looks like setting CGO_ENABLED=0 avoids the error, but I'm not sure what the implications are…

$ docker run --rm -w /x golang:1.11-alpine sh -c 'go mod init x && CGO_ENABLED=0 go build std'
go: creating new go.mod: module x
@tylercloke

This comment has been minimized.

Copy link

commented Aug 29, 2018

@rsc Can you expand a bit? It was certainly a surprise for me :) I started seeing this after upgrade to 1.11. Feels like a breaking change to the end-user if my build worked before upgrade and doesn't work now. Just want to understand what changed in more detail and what proper remediation is. Thanks!

@thepudds

This comment has been minimized.

Copy link

commented Sep 1, 2018

Regarding this issue, Russ commented:

Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.

Note this issue is specific to opting in to modules (and related to the fact that Go 1.11 ships non-module based packages because the modules system overall is still opt-in only in Go 1.11).

This particular issue should not be affecting non-module based builds as far as I understand it.

@tylercloke

This comment has been minimized.

Copy link

commented Sep 4, 2018

@thepudds Thanks! Can you link me to the original context for that comment? Also, would we expect this to affect builds that pull in deps that might use modules? We did not opt into modules and were seeing issues on 1.11.

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 4, 2018

We did not opt into modules and were seeing issues on 1.11.

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

(You can check whether module mode is active by running go env: if you're in module mode, it will have a non-empty GOMOD line.)

@tylercloke

This comment has been minimized.

Copy link

commented Sep 4, 2018

There are two issues marked as dups of this that make no mention of modules. I can also repro without modules enabled:

#27303
#27285

I'll open another issue when I have time if you want but what I'm seeing looks pretty similar to the above.

@mfridman

This comment has been minimized.

Copy link

commented Sep 24, 2018

Noticed a similar issue in our pipeline when switching from 1.10.x to 1.11.0. Specifically with the 1.11.0-alpine3.7 docker image.

We explicitly set GO111MODULE=off and were seeing similar error as reported above.

exec: "gcc": executable file not found in $PATH

The solution, as mentioned before, was to explicitly set CGO_ENABLED=0. I suspect many folks will come across this particular issue when using lightweight containers such as alpine.

@thepudds

This comment has been minimized.

Copy link

commented Sep 24, 2018

@mfridman if you look at the symptoms in #27285, do those line up with what you are seeing? #27285 has an initial CL.

@mvdan

This comment has been minimized.

Copy link
Member

commented Oct 3, 2018

We ran into the same issue when upgrading from 1.10.3-alpine to 1.11.1-alpine - errors due to gcc not being installed.

To clarify, this is when building a package inside GOPATH and without setting any Go env vars other than GOPATH and GOCACHE:

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/go/src/brank.as/brankas-api/.go_cache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build759719652=/tmp/go-build -gno-record-gcc-switches"
$ go test -p 8 -parallel 8 -coverprofile=coverage.out -count=1 -v ./...
# runtime/cgo
exec: "gcc": executable file not found in $PATH

For now I'll just globally set CGO_ENABLED=0, but I presume there's something to fix here since this worked in 1.10.

@thepudds

This comment has been minimized.

Copy link

commented Oct 6, 2018

Hi @mvdan , given you are seeing that behavior outside of modules (including an empty GOMOD reported by go env), what would you think about opening a new issue that covers the problem you saw in your immediately prior comment?

I think that was the suggestion from @bcmills several comments back:

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

As far as I am following things, it sounds like there might be at least three distinct behaviors:

  1. This original issue reported here in #26988, which might be surprising to a user but sounds like it is currently "expected" behavior with 1.11 due to the opt in nature of modules and because "the pre-built packages are non-module builds and can’t be reused".

  2. Issue #27285, which sounds like it is related to the -i flag (and which has one pending CL, but sounds like the solution might be to make -i a no-op in module mode)

  3. The issue you are reporting, which sounds like it happened when modules are disabled (and some chance might be the same thing that @mfridman and @tylercloke reported, but that might be easiest to discuss and determine in a new/separate issue. Edit: @mvdan has now opened #28065 for this issue 3).

@mvdan

This comment has been minimized.

Copy link
Member

commented Oct 6, 2018

Agreed, will open a new issue. I posted here first, in case I was missing something obvious or misunderstanding how GO11MODULE affected this issue.

@thepudds

This comment has been minimized.

Copy link

commented Oct 6, 2018

Hi @mvdan, makes sense that you posted here first. Overall, not 100% clear what's going on with the more recent reports here, and some chance there might be multiple underlying causes that might initially seem to have similar symptoms...

rarkins added a commit to renovatebot/docker-go that referenced this issue Oct 16, 2018

@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 24, 2018

This is expected: we can only ship one pre-compiled copy of the standard library, and the pre-compiled copy we ship is for non-modules (GOPATH) mode. If we need standard library packages in module mode then we may need to rebuild them if there are any changes needed. In this case the slightly different effect of the vendor folder means that changes are needed.

One way to fix this for Go 1.12 would be to rename the top-level vendor folder to something different (internal/...) and fix the imports to match. We should probably do that.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 24, 2018

Note the fix is to change the import path of the vendor folder.

@gopherbot

This comment has been minimized.

Copy link

commented Nov 15, 2018

Change https://golang.org/cl/147443 mentions this issue: vendor/golang_org: move to internal/golang_org

@gopherbot gopherbot closed this in 2012227 Nov 29, 2018

@bcmills

This comment has been minimized.

Copy link
Member

commented Nov 29, 2018

CL 147443 fixed the known cause of these symptoms. Please open a new issue if you continue to observe them with a go command built after commit 2012227.

blgm added a commit to pivotal-cf/cf-rabbitmq-smoke-tests-release that referenced this issue Dec 4, 2018

Disable Go modules and cgo
When the smoke tests release was updated to Go v1.11.2, the BOSH release
pipelines all passed, but pipelines that consumed the final release
failed because `gcc` was not installed.  This is a very similar issue to
golang/go#26988.  Experiments showed that
adding `CGO_ENABLED=0 GO111MODULE=off` to disable `cgo` and Go moudles
allowed the smoke tests to pass.  These variables should be removed when
we recieved an updated version of Go (after 1.11.2).

[#159658309]

@wolffcm wolffcm referenced this issue May 7, 2019

Closed

chore: update ragel version used by CI #1245

0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.