build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

jefferai · 2015-10-01T15:52:34Z

I've been trying to figure out how to make my programs build statically by default without specifying -a -installsuffix cgo -ldflags '-extldflags "-static" -s for every project. I've read through #9344, and supposedly the change to make dynamic builds the default is only for released versions of Go. Accordingly, I tried the following:

Install Go onto my system from a package manager
Check out go1.5.1 from Git
Set GOROOT to point to the checked-out version of Go, and GOROOT_BOOTSTRAP to the system version of Go
cd src; ./all.bash -- successful
rm -rf $GOPATH/pkg
Start a new shell with GOPATH, GOROOT, and PATH all set appropriately. I can verify that I'm using the self-compiled version of Go (which itself was built dynamically), but binaries produced with my self-built Go are still being built dynamically without passing in -a -installsuffix cgo -ldflags '-extldflags "-static" -s

Is this a bug, or did I misinterpret the requirements in #9344?

Thanks!

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2015-10-01T17:24:05Z

I can't tell if this is a bug report or not. We use the issue tracker for actual bugs and proposed changes. Please ask questions on the golang-nuts mailing list or on https://forum.golangbridge.org/. Thanks.

I can't see what you are referring to in #9344. In any case, when you check out go1.5.1, you are getting a released version of Go. To get a non-released version of Go, you need to check out something that is not a release. But of course those are not supported.

If you build your own version of Go with CGO_ENABLED=0 set in the environment, you should get a version of Go that builds statically by default.

jefferai · 2015-10-01T18:07:20Z

@ianlancetaylor I couldn't tell if it was a bug either, because I wasn't sure if the workflow that was described was expected to work, or not. After reading what you said, I do think it's at least a documentation issue.

#9344 suggests that the reason for dynamic linking in released versions is because they are installed to system locations that normal users don't have access to change. This is confusing, because this should be a toggle flipped by packagers when building binary packages for release, but instead is (apparently) something enforced in the code in Git. IOW, it's not clear that "released version" means "tagged in Git" as opposed to "downloadable release from the web site".

jefferai · 2015-10-01T19:26:06Z

@ianlancetaylor I built Go from go1.5.1 tag with CGO_ENABLED=0 set (which appears to be the default anyways), and after setting that as GOROOT it still builds dynamic programs by default, unless I also set CGO_ENABLED=0 before building the program.

ianlancetaylor · 2015-10-01T20:15:48Z

I'm still not sure what you're referring to in #9344, but it doesn't much matter. What matters is the documentation.

Never set GOROOT. If it is in your environment, take it out. If you build a release, run bin/go from that release.

CGO_ENABLED=0 is not the default for a native build.

Setting CGO_ENABLED when building the Go sources is documented in src/make.bash.

jefferai · 2015-10-01T21:40:22Z

#9344 is the exact issue I've been having problems with. There are several mentions of released builds, for instance, #9344 (comment) and #9344 (comment)

It's not clear where CGO_ENABLED is toggled to 1 in a native build. Regardless, with CGO_ENABLED=0 exported and running all.bash, the built go binary is not static. This does work for my programs, though.

ianlancetaylor · 2015-10-02T00:03:29Z

The comments you link to from #9344 are both out of date, as it happens. In Go 1.5 the -a option to go build does rebuild everything, including the standard library, whether you are running a release build or not.

I just tried running

CGO_ENABLED=0 bash make.bash

on a clean checkout of Go tip. It built a statically linked program version of ../bin/go.

But, you are correct: when, using that version of go, I run go env CGO_ENABLED, it prints 1, and it will link packages like os/user dynamically. I did not expect that. I'm not sure whether it is a bug or not.

jefferai · 2015-10-02T00:30:49Z

OK -- I didn't realize the behavior changed again in 1.5.

I'll keep an eye on this report. Thanks for looking into it.

rsc · 2015-12-05T05:07:47Z

I am not 100% sure the requested behavior is what we want. I'm curious whether it changed or if it has always been the way it is now. I suspect the latter.

rsc · 2015-12-17T03:30:05Z

Looked some more. I think it has always been this way, at least back to Go 1.2: CGO_ENABLED controls now, not later. It does not bake in a default value during make.bash. Still not sure whether we want to change it.

jefferai · 2015-12-17T17:00:37Z

@rsc I appreciate you looking. As you can see from the initial issue I referenced (and others I can point to) usage of CGO_ENABLED vs. other methods of ensuring that a built binary is static (when not cross-compiling) has been a point of confusion, and I think a lot of that has been due to the changes to e.g. -a in 1.4 and more changes in 1.5.

Feel free to close this, unless you want to keep it open as a documentation enhancement request.

rsc · 2015-12-17T17:14:47Z

For now it's open because we may still want to change it.

rsc · 2016-01-06T19:58:11Z

I think this should probably force the default but it's too late for Go 1.6.

bradfitz · 2016-05-05T17:10:29Z

Didn't happen for 1.7 either.

rsc · 2016-10-17T17:31:39Z

OK, let's try this and see what breaks. Will send CL.

gopherbot · 2016-10-17T17:35:15Z

CL https://golang.org/cl/31141 mentions this issue.

The build was forcing CGO_ENABLED=1, which is not necessary. But as of Go 1.8, not only it's not necessary, it's also harmful, as the value of CGO_ENABLED used during Go's build gets recorded and used by default [1] when building any Go code, which prevents the compiler from disabling cgo automatically when cross-compiling. [2] [1] See golang/go#12808 [2] See golang/go#18360

Take advantage of the new multi-stage build feature in Docker to build the application binary in a separate container to the final build container. The final build container is now 21MB in size versus 752MB, i.e. 2.79% of the previous size. From quick testing, the integration tests now take 1 minute 12 seconds versus 1 minute 47 seconds. To avoid complications arising from dynamic linking, I've set `CGO_ENABLED=0` to make Go compile the binary statically: golang/go#12808 (comment) Use the `before_install` step in `.travis.yml` to use the latest available Docker version, since the version supported by default in Travis (version 17.03.1-ce, build c6d412e, at the time of writing) does not yet support multi-stage builds: https://docs.travis-ci.com/user/docker/#Installing-a-newer-Docker-version Multi-stage builds were introduced by version 17.06 CE: https://blog.docker.com/2017/07/multi-stage-builds/ See: https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds https://blog.alexellis.io/mutli-stage-docker-builds/

rakyll changed the title ~~Building a program with non-release version of Go is dynamic by default~~ cmd/go: building a program with non-release version of Go is dynamic by default Oct 1, 2015

ianlancetaylor changed the title ~~cmd/go: building a program with non-release version of Go is dynamic by default~~ build: setting CGO_ENABLED when building does not affect built go tool Oct 2, 2015

ianlancetaylor added this to the Go1.6 milestone Oct 2, 2015

rsc changed the title ~~build: setting CGO_ENABLED when building does not affect built go tool~~ build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go Dec 5, 2015

rsc modified the milestones: Go1.7Early, Go1.6 Jan 6, 2016

bradfitz modified the milestones: Go1.8Early, Go1.7Early May 5, 2016

timfallmk mentioned this issue May 25, 2016

Terraform Light Docker 0.6.16 Fails to run hashicorp/docker-hub-images#7

Closed

adrianlzt mentioned this issue May 30, 2016

Static compiling ceph/go-ceph#23

Closed

quentinmit added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 26, 2016

quentinmit assigned rsc Sep 26, 2016

rsc added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Oct 17, 2016

gopherbot closed this as completed in 0ba3c60 Oct 17, 2016

This was referenced Dec 17, 2016

cmd/go: 1.8beta2 cross compilation fails without CGO_ENABLED=0 #18360

Closed

go: Fix cross-compilation. Homebrew/homebrew-core#7962

Closed

mattbostock mentioned this issue Aug 21, 2017

Convert Dockerfile to multi-stage builds mattbostock/timbala#74

Merged

golang locked and limited conversation to collaborators Oct 17, 2017

gopherbot added the FrozenDueToAge label Oct 17, 2017

rsc removed their assignment Jun 23, 2022

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 1, 2015

jefferai commented Oct 1, 2015

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 1, 2015

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 2, 2015

jefferai commented Oct 2, 2015

rsc commented Dec 5, 2015

rsc commented Dec 17, 2015

jefferai commented Dec 17, 2015

rsc commented Dec 17, 2015 via email

rsc commented Jan 6, 2016

bradfitz commented May 5, 2016

rsc commented Oct 17, 2016

gopherbot commented Oct 17, 2016

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

Comments

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 1, 2015

jefferai commented Oct 1, 2015

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 1, 2015

jefferai commented Oct 1, 2015

ianlancetaylor commented Oct 2, 2015

jefferai commented Oct 2, 2015

rsc commented Dec 5, 2015

rsc commented Dec 17, 2015

jefferai commented Dec 17, 2015

rsc commented Dec 17, 2015 via email

rsc commented Jan 6, 2016

bradfitz commented May 5, 2016

rsc commented Oct 17, 2016

gopherbot commented Oct 17, 2016