cmd/go: With Go 1.4, cannot statically link a program which imports net #9344

Closed
glyn opened this Issue Dec 16, 2014 · 26 comments

Comments

Projects
None yet
@glyn

glyn commented Dec 16, 2014

In Go 1.3.x, it was possible to statically link a program which imports the net package. This is important when building programs for cut-down environments, for example busybox.

The problem can be reproduced on Ubuntu as reported by Vasiliy Tolstov:

package main 
import "net" 
func main() { 
        _, _ = net.ResolveIPAddr("ip", "google.com") 
} 

It is possible to statically link this program in Go 1.3.x by issuing:

CGO_ENABLED=0 go build -a -x

but with Go 1.4, the same technique creates the following runtime dependencies:

linux-vdso.so.1 (0x00007fff603fe000) 
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3bd0608000) 
libc.so.6 => /lib64/libc.so.6 (0x00007f3bd0260000) 
/lib64/ld-linux-x86-64.so.2 (0x00007f3bd0825000) 
@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux Dec 16, 2014

Member

Working as intended.

In released version of go, go build/install -a std will not try to
reinstall the standard packages to avoid the common problem of permission
error when go is installed to a system directory.

Member

minux commented Dec 16, 2014

Working as intended.

In released version of go, go build/install -a std will not try to
reinstall the standard packages to avoid the common problem of permission
error when go is installed to a system directory.

@glyn

This comment has been minimized.

Show comment
Hide comment
@glyn

glyn Dec 16, 2014

So is the above static linking behaviour therefore a bug in go 1.3.x?

glyn commented Dec 16, 2014

So is the above static linking behaviour therefore a bug in go 1.3.x?

@kostix

This comment has been minimized.

Show comment
Hide comment
@kostix

kostix Dec 16, 2014

@minux, just to make it clear, do I understand correctly that using CGO_ENABLED=0 go build -a -x (in older versions of Go) actually rebuilt and reinstalled certain parts of stdlib instead of just affecting the program being built?

kostix commented Dec 16, 2014

@minux, just to make it clear, do I understand correctly that using CGO_ENABLED=0 go build -a -x (in older versions of Go) actually rebuilt and reinstalled certain parts of stdlib instead of just affecting the program being built?

@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux Dec 17, 2014

Member

To clarify, go install/build -a std won't do anything for standard packages
in a released version of Go.
See commit 8c3005c for details.

Member

minux commented Dec 17, 2014

To clarify, go install/build -a std won't do anything for standard packages
in a released version of Go.
See commit 8c3005c for details.

@glyn

This comment has been minimized.

Show comment
Hide comment
@glyn

glyn Dec 17, 2014

@minux: when you say "a released version of Go" are you referring to 1.3.x as well as 1.4??

glyn commented Dec 17, 2014

@minux: when you say "a released version of Go" are you referring to 1.3.x as well as 1.4??

@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux Dec 17, 2014

Member

This change in behavior only applies to 1.4.

Member

minux commented Dec 17, 2014

This change in behavior only applies to 1.4.

@glyn

This comment has been minimized.

Show comment
Hide comment
@glyn

glyn Dec 17, 2014

I see, thanks. Programs which import the net package and need to be statically linked cannot migrate to 1.4. Is there some reason, in principle, why such static linking cannot be supported in 1.4.x or does commit 8c3005c preclude such a solution?

glyn commented Dec 17, 2014

I see, thanks. Programs which import the net package and need to be statically linked cannot migrate to 1.4. Is there some reason, in principle, why such static linking cannot be supported in 1.4.x or does commit 8c3005c preclude such a solution?

@kostix

This comment has been minimized.

Show comment
Hide comment
@kostix

kostix Dec 17, 2014

@glyn, if I understood correctly (@minux did not address my question directly), before 1.3, when you performed the encantation being discussed, a part of the Go's stdlib has been silently rebuilt and replaced -- most of folks did not notice it because most have Go built from the source and installed somewhere under ~. Since this behaviour clearly fails with system-wide Go installs (some people do use apt-get install golang), the -a command-line option stopped affecting the Go's own files in 1.4.

The solution is just to build and install another instalce of the Go toolchain and use it via custom $GOROOT where applicable.

IOW, building fully static programs on Linux did not ever do The Right Thing. I think it might be sensible to file a wishlist bug asking to somehow make stdlib support both behaviours out of the box. Dunno if it's feasible to implement.

kostix commented Dec 17, 2014

@glyn, if I understood correctly (@minux did not address my question directly), before 1.3, when you performed the encantation being discussed, a part of the Go's stdlib has been silently rebuilt and replaced -- most of folks did not notice it because most have Go built from the source and installed somewhere under ~. Since this behaviour clearly fails with system-wide Go installs (some people do use apt-get install golang), the -a command-line option stopped affecting the Go's own files in 1.4.

The solution is just to build and install another instalce of the Go toolchain and use it via custom $GOROOT where applicable.

IOW, building fully static programs on Linux did not ever do The Right Thing. I think it might be sensible to file a wishlist bug asking to somehow make stdlib support both behaviours out of the box. Dunno if it's feasible to implement.

@kostix

This comment has been minimized.

Show comment
Hide comment
@kostix

kostix Dec 17, 2014

@glyn, I mean a custom build of the Go toolchain done with CGO_ENABLED=0 (and maybe -tags netgo, if applicable).

kostix commented Dec 17, 2014

@glyn, I mean a custom build of the Go toolchain done with CGO_ENABLED=0 (and maybe -tags netgo, if applicable).

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Dec 17, 2014

Contributor

No no no. You don't need to set GOROOT, ever. Please see my blog
http://dave.cheney.net/2014/04/20/how-to-install-multiple-versions-of-go

On Thu, Dec 18, 2014 at 6:57 AM, Konstantin Khomoutov <
notifications@github.com> wrote:

@glyn https://github.com/glyn, if I understood correctly (@minux
https://github.com/minux did not address my question directly), before
1.3, when you performed the encantation being discussed, a part of the Go's
stdlib has been silently rebuilt and replaced -- most of folks did not
notice it because most have Go built from the source and installed
somewhere under ~. Since this behaviour clearly fails with system-wide Go
installs (some people do use apt-get install golang), the -a command-line
option stopped affecting the Go's own files in 1.4.

The solution is just to build and install another instalce of the Go
toolchain and use it via custom $GOROOT where applicable.

IOW, building fully static programs on Linux did not ever do The Right
Thing. I think it might be sensible to file a wishlist bug asking to
somehow make stdlib support both behaviours out of the box. Dunno if it's
feasible to implement.


Reply to this email directly or view it on GitHub
#9344 (comment).

Contributor

davecheney commented Dec 17, 2014

No no no. You don't need to set GOROOT, ever. Please see my blog
http://dave.cheney.net/2014/04/20/how-to-install-multiple-versions-of-go

On Thu, Dec 18, 2014 at 6:57 AM, Konstantin Khomoutov <
notifications@github.com> wrote:

@glyn https://github.com/glyn, if I understood correctly (@minux
https://github.com/minux did not address my question directly), before
1.3, when you performed the encantation being discussed, a part of the Go's
stdlib has been silently rebuilt and replaced -- most of folks did not
notice it because most have Go built from the source and installed
somewhere under ~. Since this behaviour clearly fails with system-wide Go
installs (some people do use apt-get install golang), the -a command-line
option stopped affecting the Go's own files in 1.4.

The solution is just to build and install another instalce of the Go
toolchain and use it via custom $GOROOT where applicable.

IOW, building fully static programs on Linux did not ever do The Right
Thing. I think it might be sensible to file a wishlist bug asking to
somehow make stdlib support both behaviours out of the box. Dunno if it's
feasible to implement.


Reply to this email directly or view it on GitHub
#9344 (comment).

@vtolstov

This comment has been minimized.

Show comment
Hide comment
@vtolstov

vtolstov Dec 17, 2014

2014-12-17 22:57 GMT+03:00 Konstantin Khomoutov notifications@github.com:

@glyn, if I understood correctly (@minux did not address my question directly), before 1.3, when you performed the encantation being discussed, a part of the Go's stdlib has been silently rebuilt and replaced -- most of folks did not notice it because most have Go built from the source and installed somewhere under ~. Since this behaviour clearly fails with system-wide Go installs (some people do use apt-get install golang), the -a command-line option stopped affecting the Go's own files in 1.4.

This is not true. I'm use go from package system (that install it in
system-wide /usr/lib/go) and -a works fine under user rebuild all
inside /tmp and doing static linking after all deps recompiled.

Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
jabber: vase@selfip.ru

2014-12-17 22:57 GMT+03:00 Konstantin Khomoutov notifications@github.com:

@glyn, if I understood correctly (@minux did not address my question directly), before 1.3, when you performed the encantation being discussed, a part of the Go's stdlib has been silently rebuilt and replaced -- most of folks did not notice it because most have Go built from the source and installed somewhere under ~. Since this behaviour clearly fails with system-wide Go installs (some people do use apt-get install golang), the -a command-line option stopped affecting the Go's own files in 1.4.

This is not true. I'm use go from package system (that install it in
system-wide /usr/lib/go) and -a works fine under user rebuild all
inside /tmp and doing static linking after all deps recompiled.

Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
jabber: vase@selfip.ru

@iand

This comment has been minimized.

Show comment
Hide comment
@iand

iand Dec 18, 2014

@vtolstov I'm not sure how that can be. If you read the source code at 8c3005c you can see it expllicitly doesn't recompile std packages for a released version of Go.

What version are you running your test on?

iand commented Dec 18, 2014

@vtolstov I'm not sure how that can be. If you read the source code at 8c3005c you can see it expllicitly doesn't recompile std packages for a released version of Go.

What version are you running your test on?

@mikioh mikioh changed the title from With Go 1.4, cannot statically link a program which imports net to cmd/go: With Go 1.4, cannot statically link a program which imports net Dec 19, 2014

@vtolstov

This comment has been minimized.

Show comment
Hide comment
@vtolstov

vtolstov Dec 19, 2014

@iand i say for go 1.3 not for 1.4.
I'm not understand why breaks static linking from 1.3 to 1.4. If you don't do go install -a all works fine and nothing rewrited....

@iand i say for go 1.3 not for 1.4.
I'm not understand why breaks static linking from 1.3 to 1.4. If you don't do go install -a all works fine and nothing rewrited....

@iand

This comment has been minimized.

Show comment
Hide comment
@iand

iand Dec 19, 2014

@vtolstov the behaviour of go install -a was changed in 1.4 so it doesn't recompile the std library if you are using a release version of Go.

This issue looks like an unfortunate side effect of that change. There seems to be no way to recompile the standard library in a release version of Go without recompiling all of Go from source. While that's not onerous for those who regularly use the source version it will cause problems for those who want or need to use pre-packaged versions.

This discussion from golang-dev is relevant: https://groups.google.com/d/msg/golang-dev/2ZUi792oztM/Q0rg_DkF5HMJ

One solution might be to skip building the std library except when the -tags argument is used and any of the specified tags are used by the std library.

iand commented Dec 19, 2014

@vtolstov the behaviour of go install -a was changed in 1.4 so it doesn't recompile the std library if you are using a release version of Go.

This issue looks like an unfortunate side effect of that change. There seems to be no way to recompile the standard library in a release version of Go without recompiling all of Go from source. While that's not onerous for those who regularly use the source version it will cause problems for those who want or need to use pre-packaged versions.

This discussion from golang-dev is relevant: https://groups.google.com/d/msg/golang-dev/2ZUi792oztM/Q0rg_DkF5HMJ

One solution might be to skip building the std library except when the -tags argument is used and any of the specified tags are used by the std library.

@mikioh mikioh added the repo-main label Dec 20, 2014

@mikioh

This comment has been minimized.

Show comment
Hide comment
@mikioh

mikioh Dec 20, 2014

Contributor

See #9369.

Contributor

mikioh commented Dec 20, 2014

See #9369.

@glyn

This comment has been minimized.

Show comment
Hide comment
@glyn

glyn Dec 22, 2014

Thanks @mikioh, although I'm not sure why it was necessary for #9369 to be raised since it's a duplicate of this bug.

glyn commented Dec 22, 2014

Thanks @mikioh, although I'm not sure why it was necessary for #9369 to be raised since it's a duplicate of this bug.

@Zteve

This comment has been minimized.

Show comment
Hide comment
@Zteve

Zteve Dec 29, 2014

@glyn Ah, you do not understand: this issue is a bug report, but #9369 is a feature request. By definition, there are no bugs in the Go lang implementations: every behaviour is intentional.

Zteve commented Dec 29, 2014

@glyn Ah, you do not understand: this issue is a bug report, but #9369 is a feature request. By definition, there are no bugs in the Go lang implementations: every behaviour is intentional.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Dec 31, 2014

Contributor

I simply didn't notice this issue when I opened 9369. I'll leave them both open for now since there are useful comments on both.

Contributor

ianlancetaylor commented Dec 31, 2014

I simply didn't notice this issue when I opened 9369. I'll leave them both open for now since there are useful comments on both.

@glyn

This comment has been minimized.

Show comment
Hide comment
@glyn

glyn Jan 2, 2015

Fair enough.

glyn commented Jan 2, 2015

Fair enough.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Jan 14, 2015

Contributor

Work around: use -installsuffix.

CGO_ENABLED=0 go build -a -installsuffix cgo

Edited to add: while this workaround was useful for old versions of Go, it is completely unnecessary for Go 1.10 and later. If you have been using it, you can stop.

Contributor

ianlancetaylor commented Jan 14, 2015

Work around: use -installsuffix.

CGO_ENABLED=0 go build -a -installsuffix cgo

Edited to add: while this workaround was useful for old versions of Go, it is completely unnecessary for Go 1.10 and later. If you have been using it, you can stop.

thockin added a commit to thockin/kubernetes that referenced this issue Jan 21, 2015

@thockin thockin referenced this issue in kubernetes/kubernetes Jan 21, 2015

Merged

Fix static builds in go1.4 #3666

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Jan 24, 2015

Contributor

I don't think we are going to do anything--using -installsuffix fixes the problem.

Contributor

ianlancetaylor commented Jan 24, 2015

I don't think we are going to do anything--using -installsuffix fixes the problem.

@DamnWidget

This comment has been minimized.

Show comment
Hide comment
@DamnWidget

DamnWidget Jan 24, 2015

Hi.

It should be well documented, thanks for the effort.

Hi.

It should be well documented, thanks for the effort.

glyn added a commit to cloudfoundry-attic/garden-linux that referenced this issue Jan 26, 2015

Build netdog
Previously netdog had to be built with Go 1.3 to ensure it was statically
linked. With a workaround to Go bug 9344[1], this is now achievable with
Go 1.4.

[1]: golang/go#9344

[#86913170]

Signed-off-by: Chris Brown <cbrown@pivotal.io>

@mboersma mboersma referenced this issue in deis/deis Feb 25, 2015

Merged

fix(go): go 1.4 static binaries #3133

@ap0 ap0 referenced this issue in zulily/boilerplate Apr 14, 2015

Closed

Problem running images using the 'net/http' library #4

@vieux vieux referenced this issue in docker/swarm Jul 6, 2015

Merged

fix static compile with go 1.4 #1033

@dukedougal

This comment has been minimized.

Show comment
Hide comment
@dukedougal

dukedougal Oct 20, 2015

I'm really puzzled about this. The primary reason I built my app was for static linking which I thought was the default behaviour and now I find that forever more I'll need to use what seems like a kludge to compile:

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s' src/myapp/myapp.go

I'm using Go 1.5.1

I'm really puzzled about this. The primary reason I built my app was for static linking which I thought was the default behaviour and now I find that forever more I'll need to use what seems like a kludge to compile:

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s' src/myapp/myapp.go

I'm using Go 1.5.1

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Oct 20, 2015

Contributor

If you want static compilation always then I recommend installing Go from
source

env CGO_ENABLED=0 ./all.bash

That will disable cgo permanently.

On Tue, Oct 20, 2015 at 4:55 PM, dukedougal notifications@github.com
wrote:

I'm really puzzled about this. The primary reason I built my app was for
static linking which I thought was the default behaviour and now I find
that forever more I'll need to use what seems like a kludge to compile:

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s'
src/myapp/myapp.go

I'm using Go 1.5.1


Reply to this email directly or view it on GitHub
#9344 (comment).

Contributor

davecheney commented Oct 20, 2015

If you want static compilation always then I recommend installing Go from
source

env CGO_ENABLED=0 ./all.bash

That will disable cgo permanently.

On Tue, Oct 20, 2015 at 4:55 PM, dukedougal notifications@github.com
wrote:

I'm really puzzled about this. The primary reason I built my app was for
static linking which I thought was the default behaviour and now I find
that forever more I'll need to use what seems like a kludge to compile:

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s'
src/myapp/myapp.go

I'm using Go 1.5.1


Reply to this email directly or view it on GitHub
#9344 (comment).

@hongweiyi

This comment has been minimized.

Show comment
Hide comment
@hongweiyi

hongweiyi Nov 13, 2015

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s' src/myapp/myapp.go

Go 1.5.1

LGTM

CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s' src/myapp/myapp.go

Go 1.5.1

LGTM

luxas added a commit to luxas/kubernetes-on-arm that referenced this issue Mar 1, 2016

@adrianlzt adrianlzt referenced this issue in ceph/go-ceph May 30, 2016

Open

Static compiling #23

@luxas luxas referenced this issue in kubernetes/kubernetes Jul 19, 2016

Closed

Make all Kubernetes binaries statically linked #26028

benoitf referenced this issue in eclipse/che-lib Oct 4, 2016

Use go flags to compile binaries in static mode
So no need to have glibc installed for example

@golang golang locked and limited conversation to collaborators Nov 16, 2016

glycerine referenced this issue in japanoise/gomacs Sep 25, 2017

Use Glisp instead of Zygomys
While it sucks to switch to an unmaintained project, we don't lose
anything from this switch, and we gain a killer feature: gomacs can
be staticly compiled (and it is with default settings).
Glisp's api also provides more idiomatic Go, and it seems to be a tad
quicker, too.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.