Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

Closed
jefferai opened this issue Oct 1, 2015 · 15 comments

Comments

Projects
None yet
6 participants
@jefferai
Copy link

commented Oct 1, 2015

I've been trying to figure out how to make my programs build statically by default without specifying -a -installsuffix cgo -ldflags '-extldflags "-static" -s for every project. I've read through #9344, and supposedly the change to make dynamic builds the default is only for released versions of Go. Accordingly, I tried the following:

  1. Install Go onto my system from a package manager
  2. Check out go1.5.1 from Git
  3. Set GOROOT to point to the checked-out version of Go, and GOROOT_BOOTSTRAP to the system version of Go
  4. cd src; ./all.bash -- successful
  5. rm -rf $GOPATH/pkg
  6. Start a new shell with GOPATH, GOROOT, and PATH all set appropriately. I can verify that I'm using the self-compiled version of Go (which itself was built dynamically), but binaries produced with my self-built Go are still being built dynamically without passing in -a -installsuffix cgo -ldflags '-extldflags "-static" -s

Is this a bug, or did I misinterpret the requirements in #9344?

Thanks!

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2015

I can't tell if this is a bug report or not. We use the issue tracker for actual bugs and proposed changes. Please ask questions on the golang-nuts mailing list or on https://forum.golangbridge.org/. Thanks.

I can't see what you are referring to in #9344. In any case, when you check out go1.5.1, you are getting a released version of Go. To get a non-released version of Go, you need to check out something that is not a release. But of course those are not supported.

If you build your own version of Go with CGO_ENABLED=0 set in the environment, you should get a version of Go that builds statically by default.

@jefferai

This comment has been minimized.

Copy link
Author

commented Oct 1, 2015

@ianlancetaylor I couldn't tell if it was a bug either, because I wasn't sure if the workflow that was described was expected to work, or not. After reading what you said, I do think it's at least a documentation issue.

#9344 suggests that the reason for dynamic linking in released versions is because they are installed to system locations that normal users don't have access to change. This is confusing, because this should be a toggle flipped by packagers when building binary packages for release, but instead is (apparently) something enforced in the code in Git. IOW, it's not clear that "released version" means "tagged in Git" as opposed to "downloadable release from the web site".

@jefferai

This comment has been minimized.

Copy link
Author

commented Oct 1, 2015

@ianlancetaylor I built Go from go1.5.1 tag with CGO_ENABLED=0 set (which appears to be the default anyways), and after setting that as GOROOT it still builds dynamic programs by default, unless I also set CGO_ENABLED=0 before building the program.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2015

I'm still not sure what you're referring to in #9344, but it doesn't much matter. What matters is the documentation.

Never set GOROOT. If it is in your environment, take it out. If you build a release, run bin/go from that release.

CGO_ENABLED=0 is not the default for a native build.

Setting CGO_ENABLED when building the Go sources is documented in src/make.bash.

@jefferai

This comment has been minimized.

Copy link
Author

commented Oct 1, 2015

#9344 is the exact issue I've been having problems with. There are several mentions of released builds, for instance, #9344 (comment) and #9344 (comment)

It's not clear where CGO_ENABLED is toggled to 1 in a native build. Regardless, with CGO_ENABLED=0 exported and running all.bash, the built go binary is not static. This does work for my programs, though.

@rakyll rakyll changed the title Building a program with non-release version of Go is dynamic by default cmd/go: building a program with non-release version of Go is dynamic by default Oct 1, 2015

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 2, 2015

The comments you link to from #9344 are both out of date, as it happens. In Go 1.5 the -a option to go build does rebuild everything, including the standard library, whether you are running a release build or not.

I just tried running

CGO_ENABLED=0 bash make.bash

on a clean checkout of Go tip. It built a statically linked program version of ../bin/go.

But, you are correct: when, using that version of go, I run go env CGO_ENABLED, it prints 1, and it will link packages like os/user dynamically. I did not expect that. I'm not sure whether it is a bug or not.

@ianlancetaylor ianlancetaylor changed the title cmd/go: building a program with non-release version of Go is dynamic by default build: setting CGO_ENABLED when building does not affect built go tool Oct 2, 2015

@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Oct 2, 2015

@jefferai

This comment has been minimized.

Copy link
Author

commented Oct 2, 2015

OK -- I didn't realize the behavior changed again in 1.5.

I'll keep an eye on this report. Thanks for looking into it.

@rsc rsc changed the title build: setting CGO_ENABLED when building does not affect built go tool build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go Dec 5, 2015

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 5, 2015

I am not 100% sure the requested behavior is what we want. I'm curious whether it changed or if it has always been the way it is now. I suspect the latter.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2015

Looked some more. I think it has always been this way, at least back to Go 1.2: CGO_ENABLED controls now, not later. It does not bake in a default value during make.bash. Still not sure whether we want to change it.

@jefferai

This comment has been minimized.

Copy link
Author

commented Dec 17, 2015

@rsc I appreciate you looking. As you can see from the initial issue I referenced (and others I can point to) usage of CGO_ENABLED vs. other methods of ensuring that a built binary is static (when not cross-compiling) has been a point of confusion, and I think a lot of that has been due to the changes to e.g. -a in 1.4 and more changes in 1.5.

Feel free to close this, unless you want to keep it open as a documentation enhancement request.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2015

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 6, 2016

I think this should probably force the default but it's too late for Go 1.6.

@rsc rsc modified the milestones: Go1.7Early, Go1.6 Jan 6, 2016

@bradfitz bradfitz modified the milestones: Go1.8Early, Go1.7Early May 5, 2016

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 5, 2016

Didn't happen for 1.7 either.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 17, 2016

OK, let's try this and see what breaks. Will send CL.

@rsc rsc added NeedsFix and removed NeedsDecision labels Oct 17, 2016

@gopherbot

This comment has been minimized.

Copy link

commented Oct 17, 2016

CL https://golang.org/cl/31141 mentions this issue.

@gopherbot gopherbot closed this in 0ba3c60 Oct 17, 2016

BrewTestBot added a commit to BrewTestBot/homebrew-core that referenced this issue Dec 17, 2016

go: Fix cross-compilation.
The build was forcing CGO_ENABLED=1, which is not necessary.  But as of
Go 1.8, not only it's not necessary, it's also harmful, as the value of
CGO_ENABLED used during Go's build gets recorded and used by default [1]
when building any Go code, which prevents the compiler from disabling cgo
automatically when cross-compiling. [2]

[1] See golang/go#12808
[2] See golang/go#18360

mattbostock added a commit to mattbostock/timbala that referenced this issue Aug 21, 2017

Convert Dockerfile to multi-stage builds
Take advantage of the new multi-stage build feature in Docker to build
the application binary in a separate container to the final build
container.

The final build container is now 21MB in size versus 752MB, i.e.
2.79% of the previous size.

From quick testing, the integration tests now take 1 minute 12 seconds
versus 1 minute 47 seconds.

To avoid complications arising from dynamic linking, I've set
`CGO_ENABLED=0` to make Go compile the binary statically:
golang/go#12808 (comment)

Use the `before_install` step in `.travis.yml` to use the latest
available Docker version, since the version supported by default in
Travis (version 17.03.1-ce, build c6d412e, at the time of writing) does
not yet support multi-stage builds:
https://docs.travis-ci.com/user/docker/#Installing-a-newer-Docker-version

Multi-stage builds were introduced by version 17.06 CE:
https://blog.docker.com/2017/07/multi-stage-builds/

See:
https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
https://blog.alexellis.io/mutli-stage-docker-builds/

mattbostock added a commit to mattbostock/timbala that referenced this issue Aug 21, 2017

Convert Dockerfile to multi-stage builds
Take advantage of the new multi-stage build feature in Docker to build
the application binary in a separate container to the final build
container.

The final build container is now 21MB in size versus 752MB, i.e.
2.79% of the previous size.

From quick testing, the integration tests now take 1 minute 12 seconds
versus 1 minute 47 seconds.

To avoid complications arising from dynamic linking, I've set
`CGO_ENABLED=0` to make Go compile the binary statically:
golang/go#12808 (comment)

Use the `before_install` step in `.travis.yml` to use the latest
available Docker version, since the version supported by default in
Travis (version 17.03.1-ce, build c6d412e, at the time of writing) does
not yet support multi-stage builds:
https://docs.travis-ci.com/user/docker/#Installing-a-newer-Docker-version

Multi-stage builds were introduced by version 17.06 CE:
https://blog.docker.com/2017/07/multi-stage-builds/

See:
https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
https://blog.alexellis.io/mutli-stage-docker-builds/

@golang golang locked and limited conversation to collaborators Oct 17, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.