Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: build fails requiring C compiler for runtime/cgo #47215

Closed
lyderic opened this issue Jul 15, 2021 · 14 comments
Closed

cmd/go: build fails requiring C compiler for runtime/cgo #47215

lyderic opened this issue Jul 15, 2021 · 14 comments
Assignees
Milestone

Comments

@lyderic
Copy link

@lyderic lyderic commented Jul 15, 2021

What version of Go are you using (go version)?

I am running go 1.16.5, but I am also testing go 1.17rc1:

$ go1.17rc1 version
go version go1.17rc1 linux/amd64
$ go version
go version go1.16.5 linux/amd64

Does this issue reproduce with the latest release?

It's ok with the latest stable release (1.16.5), not with 1.17rc1

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="linux"
I am running Archlinux, fully up to date as of July 15 2021, 10:15 BST. I am running in an LXD container (LXD 4.16)

go env Output
$ go1.17rc1 env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/unix/.cache/go-build"
GOENV="/home/unix/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/unix/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/unix/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/unix/sdk/go1.17rc1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/unix/sdk/go1.17rc1/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17rc1"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build640958217=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This is the full script of what I ran:

I [unix@go117 foo]$ go1.17rc1 mod init foo
go: creating new go.mod: module foo
I [unix@go117 foo]$ ed main.go
main.go: No such file or directory
a
package main

import ( 
  . "github.com/lyderic/tools"
)

func main(){
  Cyanln("Hello, World!")
}
.
wq
98
I [unix@go117 foo]$ go1.17rc1 mod tidy
go: finding module for package github.com/lyderic/tools
go: downloading github.com/lyderic/tools v0.3.4
go: found github.com/lyderic/tools in github.com/lyderic/tools v0.3.4
go: downloading github.com/fatih/color v1.10.0
go: downloading github.com/sendgrid/sendgrid-go v3.10.0+incompatible
go: downloading github.com/wayneashleyberry/terminal-dimensions v1.0.0
go: downloading github.com/mattn/go-colorable v0.1.8
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading github.com/sendgrid/rest v2.6.4+incompatible
go: downloading github.com/stretchr/testify v1.7.0
go: downloading golang.org/x/sys v0.0.0-20210423082822-04245dca01da
go: downloading golang.org/x/net v0.0.0-20210614182718-04defd469f4e
go: downloading github.com/davecgh/go-spew v1.1.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
I [unix@go117 foo]$ go1.17rc1 build
\# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
I [unix@go117 foo]$ go build
I [unix@go117 foo]$ ./foo
Hello, World!

What did you expect to see?

With go 1.16.5, the above program compiles with no error. I use only one external lib: github.com/lyderic/tools

What did you see instead?

With go 1.17rc1, I get the following error:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

Note: if I install gcc, the error (obviously) goes away. However, the program compiles with no gcc installed if I compile with go 1.16.5, so there might be a bug.

Thanks
L.

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Jul 15, 2021

How did you install 1.16.5 and 1.17rc1 ?

@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

I installed go 1.16.5 as follows:

$ sudo pacman -S go

Then I installed go 1.17rc1 as follows:

$ go get golang.org/dl/go1.17rc1
$ go1.17rc1 download

Then I set up my PATH to include ${HOME}/go/bin

@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

To rule out any Archlinux / Pacman effect, I did a manual install of go 1.16.6 :

$ curl -LOs https://golang.org/dl/go1.16.6.linux-amd64.tar.gz
$ sudo tar xzf go1.16.6.linux-amd64.tar.gz -C /usr/local
$ export PATH=${PATH}:/usr/local/go/bin
$ go version
go version go1.16.6 linux/amd64
$ go get golang.org/dl/go1.17rc1
go: downloading golang.org/dl v0.0.0-20210713194856-38ddc79c2163
$ ~/go/bin/go1.17rc1 download
Downloaded   0.0% (     3273 / 134784935 bytes) ...
[...]
Downloaded  91.8% (123780176 / 134784935 bytes) ...
Downloaded  93.4% (125844544 / 134784935 bytes) ...
Downloaded 100.0% (134740992 / 134784935 bytes) ...
Downloaded 100.0% (134784935 / 134784935 bytes)
Unpacking /home/unix/sdk/go1.17rc1/go1.17rc1.linux-amd64.tar.gz ...
Success. You may now run 'go1.17rc1'
$ export PATH=${PATH}:${HOME}/go/bin
$ go1.17rc1 version
go version go1.17rc1 linux/amd64

It didn't fix the problem. The most notable thing is that the simple example program I use compiles fine with 1.16 but not with go 1.17. So there is a regression issue, I guess.

Many thanks

L.

@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

I tried the above manual install on a fresh ubuntu 20.04 LXC container: same error:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

However:

$ go version
go version go1.16.6 linux/amd64
$ go build
$ ./foo 
Hello, World!
@seankhliao seankhliao changed the title go 1.17rc1: "gcc" not found (works with go 1.16.5) cmd/go: build fails requiring C compiler for runtime/cgo Jul 15, 2021
@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Jul 15, 2021

It looks like go doesn't skip runtime/cgo when a C compiler isn't available and CGO_ENABLED isn't explicitly disabled

[root@1ba9e2afedf2 x]# go1.16.5 build -x runtime/cgo
WORK=/tmp/go-build3995079167
[root@1ba9e2afedf2 x]# go1.17rc1 build -x runtime/cgo
WORK=/tmp/go-build3635095243
mkdir -p $WORK/b001/
cd /root/sdk/go1.17rc1/src/runtime/cgo
TERM='dumb' CGO_LDFLAGS='"-g" "-O2" "-lpthread"' /root/sdk/go1.17rc1/pkg/tool/linux_amd64/cgo -objdir $WORK/b001/ -importpath runtime/cgo -import_runtime_cgo=false -import_syscall=false -- -I $WORK/b001/ -g -O2 -Wall -Werror ./cgo.go
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

cc @bcmills @jayconrod @matloob

@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

Yes indeed. If I disable CGO_ENABLED, it works:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
$ CGO_ENABLED=0 go1.17rc1 build
$ ./foo
Hello, World!
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 15, 2021

Closing as a duplicate of #33840. #37158, #27303, #26988 are also related.

We need a much better diagnostic here, and we should reconsider whether CGO_ENABLED should default to 1 when no C compiler is installed.

@jayconrod jayconrod closed this Jul 15, 2021
@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

The issue is that compilation works independently of how CGO_ENABLED is set when the same program is compiled with 1.16. So it looks like a regression problem to me...

$ CGO_ENABLED=0 go build
$ CGO_ENABLED=1 go build
$ CGO_ENABLED=0 go1.17rc1 build
$ CGO_ENABLED=1 go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 15, 2021

@lyderic I expect this is because of the build cache. If you run go clean -cache then CGO_ENABLED=1 go build, does it still succeed with Go 1.16 and gcc not installed?

@lyderic
Copy link
Author

@lyderic lyderic commented Jul 15, 2021

Yes it does:

$ go clean -cache
$ CGO_ENABLED=1 go build
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 15, 2021

Interesting. Reopening to investigate this more. I wonder if we changed how the cache key is calculated?

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 16, 2021

We did change how the cache key is calculated.

I've opened #47251 with an explanation and possible solution.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 19, 2021

Ok, so here's an idea for 1.18 (if we don't do #47257 or #47251 instead).

What if we bake the C compiler ID used to build the standard library into cmd/go itself, and use that as the default ID for the default CC if exec.LookPath fails to locate a matching binary?

I guess that would allow CGO_ENABLED=1 builds to succeed, but only under very narrow circumstances. They would succeed if (and only if) both:

  1. the user doesn't set any other flags that would invalidate cache entries, and
  2. the user's code doesn't depend on any third-party packages with cgo build constraints, which would presumably fail to build anyway if no C compiler is present.

But that seems too fragile to me. I prefer #47251 (don't default to CGO_ENABLED=1 without a C compiler) and/or #47257 (don't ship precompiled .a files; always require a C compiler for builds that depend on cgo code).

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 29, 2021

Verified that the workaround in CL 335409 fixes this.

Discussion continues in #47257. If that's approved and implemented, it would still be an error to build a package that needs cgo (like runtime/cgo) if no C compiler is installed. However, net and os/user would not need cgo.

@jayconrod jayconrod closed this Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants