New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: regression for supporting PIE on Alpine X86_64 platform #18243

Open
williamweixiao opened this Issue Dec 8, 2016 · 30 comments

Comments

Projects
None yet
@williamweixiao
Member

williamweixiao commented Dec 8, 2016

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8beta1 linux/amd64

What operating system and processor architecture are you using (go env)?

Alpine X86_64

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build585116893=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

# go build -buildmode=pie test/helloworld.go
# ./helloworld

What did you expect to see?

hello, world

What did you see instead?

sh: ./helloworld: not found

@williamweixiao

This comment has been minimized.

Member

williamweixiao commented Dec 8, 2016

I find that the root-cause is that go internal link provide a non-existed interpreter for Alpine

# readelf -l helloworld | grep interpreter
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]

if i force using external linker then the issue won't appear

# go build -buildmode=pie -ldflags="-linkmode=external" test/helloworld.go
# ./helloworld
hello, world

since go is using external linker for PIE before 1.8, i think the issue as a regression.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Dec 8, 2016

If Alpine uses a non-standard path to the dynamic linker, you will need to use go build -ldflags=-I/path/to/ld.so or, as you say, -ldflags=-linkmode=external. We don't have a way to set the default dynamic linker used in internal linking mode. Perhaps we can fix that for 1.9.

@ianlancetaylor ianlancetaylor added this to the Go1.9 milestone Dec 8, 2016

@williamweixiao

This comment has been minimized.

Member

williamweixiao commented Dec 9, 2016

yes, Alpine uses: /lib/ld-musl-x86_64.so.1 as it dynamic linker. But it seems hard to fix since Golang has no idea about target linux distros.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Dec 9, 2016

If at all possible we should avoid adding alpine as a GOOS value. But we could add an environment variable to make.bash to set the default dynamic interpreter, and/or we could add some code to cmd/dist that tries to guess the default dynamic interpreter by compiling and linking a C program and examining the resulting executable.

@williamweixiao

This comment has been minimized.

Member

williamweixiao commented Dec 9, 2016

sounds doable! it would be good if golang can remove dependence on GCC (i mean after installed) and according development libraries since Alpine is pursuing a small size OS image

@bradfitz

This comment has been minimized.

Member

bradfitz commented Dec 9, 2016

Go doesn't depend on GCC. You can choose to build static binaries, and you don't need GCC to do so.

@williamweixiao

This comment has been minimized.

Member

williamweixiao commented Dec 9, 2016

Yes, static binaries don't need GCC and I believe Go will finally remove all the dependence on GCC. Go is already trying to remove depend on GCC for X86 platform (such as internal linker for PIE) but for other platforms such as arm64, it still requires external (GCC) linker. Even for Alpine X86 which expects "PIE", Go internal linker has this bug and users have to use GCC linker.

errm added a commit to errm/aports that referenced this issue Mar 7, 2017

community/go: upgrade to 1.8
default-buildmode-pie.patch has been removed since is addressed upstream
by golang/go@53aec79

set-external-linker.patch adresses golang/go#18243
@ncopa

This comment has been minimized.

ncopa commented Mar 7, 2017

@ianlancetaylor If Alpine uses a non-standard path to the dynamic linker

Is there any standard that defines the path to the dynamic linker?

I would expect that /lib64/ld-linux-x86-64.so.2 also breaks android?

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Mar 7, 2017

@ncopa Standard dynamic linker paths are defined by the ELF ABI Supplement for the relevant processor.

I don't know what dynamic linker path Android uses, if any, but as far as I can see the Go cmd/link tool has no special support for choosing a different dynamic linker on Android, other than the previously mentioned -I option.

errm added a commit to errm/aports that referenced this issue Apr 24, 2017

community/go: upgrade to 1.8.1
default-buildmode-pie.patch has been removed since is addressed upstream
by golang/go@53aec79

set-external-linker.patch adresses golang/go#18243
@gopherbot

This comment has been minimized.

gopherbot commented Apr 25, 2017

CL https://golang.org/cl/41628 mentions this issue.

gopherbot pushed a commit that referenced this issue Apr 25, 2017

runtime: ignore TestCgoPprofPIE test failures on Alpine
Updates #19938
Updates #18243

Change-Id: Ib6e704c0a5d596bdfaa6493902d2528bec55bf16
Reviewed-on: https://go-review.googlesource.com/41628
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot

This comment has been minimized.

gopherbot commented Apr 25, 2017

CL https://golang.org/cl/41675 mentions this issue.

gopherbot pushed a commit that referenced this issue Apr 25, 2017

runtime: ignore TestCgoPprofPIE test failures on Alpine (take 2)
s/arm64/amd64/ in previous typo CL 41628

Updates #19938
Updates #18243

Change-Id: I282244ee3c94535f229a87b6246382385ff64428
Reviewed-on: https://go-review.googlesource.com/41675
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot

This comment has been minimized.

gopherbot commented Apr 25, 2017

CL https://golang.org/cl/41678 mentions this issue.

gopherbot pushed a commit that referenced this issue Apr 25, 2017

cmd/go, cmd/dist: temporarily disable race and PIE internal link test…
…s on Alpine

In an effort to at least understand the complete set of things not
working on Alpine Linux, I've been trying to get the build passing
again, even with tests disabled.

The race detector is broken on Alpine. That is #14481 (and #9918).
So disable those tests for now.

Also, internal linking with PIE doesn't work on Alpine yet.
That is #18243. So disable that test for now.

With this CL, all.bash almost passes. There's some cgo test failing
still, but there's no bug yet, so that can be a separate CL.

Change-Id: I3ffbb0e787ed54cb82f298b6bd5bf3ccfbc82622
Reviewed-on: https://go-review.googlesource.com/41678
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot pushed a commit that referenced this issue Apr 25, 2017

cmd/dist: disable internal linking tests on Alpine
Updates #18243

Change-Id: I1fe0af65dbd52c3e8e0a245e4cbbdfca100971b4
Reviewed-on: https://go-review.googlesource.com/41759
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
@gopherbot

This comment has been minimized.

gopherbot commented Apr 26, 2017

CL https://golang.org/cl/41759 mentions this issue.

@gopherbot

This comment has been minimized.

gopherbot commented Apr 26, 2017

CL https://golang.org/cl/41798 mentions this issue.

gopherbot pushed a commit to golang/build that referenced this issue Apr 26, 2017

dashboard: set GO_EXTLINK_ENABLED=1 for Alpine builder for now
Updates golang/go#18243

Change-Id: I76989c3f6b592fb2b68df86c853896f7a2ff7a25
Reviewed-on: https://go-review.googlesource.com/41798
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

errm added a commit to errm/aports that referenced this issue May 11, 2017

community/go: upgrade to 1.8.1
set-external-linker.patch adresses golang/go#18243

errm added a commit to errm/aports that referenced this issue May 12, 2017

community/go: upgrade to 1.8.1
set-external-linker.patch adresses golang/go#18243

algitbot pushed a commit to alpinelinux/aports that referenced this issue May 16, 2017

community/go: upgrade to 1.8.1
set-external-linker.patch adresses golang/go#18243

@bradfitz bradfitz modified the milestones: Go1.10, Go1.9 Jun 7, 2017

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 7, 2017

I think the right fix here is for make.bash to auto-detect the desired default dynamic linker, and pass it in as we currently handle things like DEFAULTCC. Does someone want to work on that?

@laboger

This comment has been minimized.

Contributor

laboger commented Jun 8, 2017

Not sure if this is the right place to ask this question. I understand the two basic problems with go on Alpine are to get the correct dynamic linker in the executable and to link in the right startup code to read the sysargs correctly. But I don't understand why Alpine wants to default to -buildmode=pie for all executables? It seems like in those cases where golang would build a static executable, that should be OK for Alpine, it is only when golang builds a dynamic executable that it should be building it pie-like.

@fabled

This comment has been minimized.

fabled commented Jun 12, 2017

Musl supports static PIE executables. We would like to default all executables (dynamic and static) to be PIE. This is due to the fact that ASLR works for main binary only when compiled in PIE mode.

@jessfraz

This comment has been minimized.

Contributor

jessfraz commented Jul 17, 2017

@ianlancetaylor I can take a crack at it

@jessfraz

This comment has been minimized.

Contributor

jessfraz commented Jul 17, 2017

how would you want this to change https://github.com/golang/go/blob/master/src/cmd/link/internal/amd64/obj.go#L70 a compile time variable or something?

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jul 18, 2017

@jessfraz I think it would be fine to make that a compile-time variable, along the lines of DefaultGOROOT and friends in cmd/internal/objabi/zbootstrap.go.

@jessfraz

This comment has been minimized.

Contributor

jessfraz commented Jul 19, 2017

@gopherbot

This comment has been minimized.

gopherbot commented Jul 19, 2017

CL https://golang.org/cl/50070 mentions this issue.

@fabled

This comment has been minimized.

fabled commented Jul 19, 2017

@jessfraz Please don't hard code the dynamic linker name in the script as it's architecture specific. And you might want to check for musl c-library instead of alpine linux for wider compatibility.

@jessfraz

This comment has been minimized.

Contributor

jessfraz commented Jul 19, 2017

@jessfraz

This comment has been minimized.

Contributor

jessfraz commented Jul 19, 2017

also just curious as to why alpine does not use the default interpreter path?

@fabled

This comment has been minimized.

fabled commented Jul 20, 2017

Thinking more the issue, if cross-building in Go is to be supported, you really need to know the target specific dynamic linker name. In alpine we patch it currently like: https://git.alpinelinux.org/cgit/aports/tree/community/go/set-external-linker.patch

However, it would probably make sense to add a new ld.Thearch.Musldyndl variable (or similar). And figure out whether to use musl or glibc by a build time or run-time automatically set option.

The default interpreter path is supported via compatibility package libc6-compat in Alpine (but probably not in other musl based distributions). However, the musl interpreter is preferred for natively built software. Alpine Linux uses the musl default interpreter name. To me the following reasons make sense:

  • musl is not fully ABI compatible with glibc (though, it is compatible where possible and several glibc programs run just fine on musl).
  • you can then install side-by-side glibc and musl, and programs linked against different c-library
  • musl keeps the arch subtype (e.g. hard vs. soft float, and big vs. little endian) as part of the interpreter name, which I think is not clear in all glibc variants
@fabled

This comment has been minimized.

fabled commented Jul 20, 2017

@richfelker Maybe you can share insight on the interpreter name choice?

@YiNo

This comment has been minimized.

YiNo commented Nov 22, 2017

mark this issue.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Nov 22, 2017

@deanchina, I don't understand. What do you mean by "mark", and who are you addressing?

@richfelker

This comment has been minimized.

richfelker commented Nov 22, 2017

@fabled: I'm not sure what exactly you're looking for. The particular name choices are made so as to allow arbitrary arch/ABI combinations to exist in the same root filesystem. This matters especially for machines that support both 32- and 64-bit arch variants (also ILP32-on-64 ones) or multiple ABIs, but it's also intended to support execution of foreign binaries via something like qemu-user+binfmt_misc, or even exotic things like advanced multi-arch kernels on machines with multiple non-same-ISA cpus on the same board sharing memory and process space.

As for why it's different from glibc (aside from the above considerations and glibc's naming clashing between archs), it's so you can have both musl and glibc present in the same root fs. "ld-linux" is part of glibc and can't be used with musl, so reusing the same name would have precluded having both installed.

@rsc rsc modified the milestones: Go1.10, Go1.11 Nov 22, 2017

@gopherbot gopherbot modified the milestones: Go1.11, Unplanned May 23, 2018

tklauser added a commit to tklauser/go-riscv that referenced this issue Dec 7, 2018

cmd/dist, cmd/link: allow passing default dynamic linker/loader
Add an environment variable to make.bash to allow setting the default
dynamic linker/loader. This fixes alpine builds to use
/lib/ld-musl-x86_64.so.1:

    $ ldd ../bin/go
        /lib/ld-musl-x86_64.so.1 (0x5608dfadd000)
        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x5608dfadd000)

Also re-enable the internal linker tests that were previously disabled
for alpine (CL 41759, CL 41678).

Fixes golang#18243
Updates golang#19938

This resurrects CL 50070 which was authored by Jessie Frazelle.

Co-authored-by: Jessie Frazelle <me@jessfraz.com>

Change-Id: I132b5282045a3d60c8568e3b002a7f075eac2d93
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment