cmd/go: Can't override linkmode when setting buildmode=pie

[Foxboron]

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/fox/.cache/go-build"
GOENV="/home/fox/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/fox/.go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/fox/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1989593130=/tmp/go-build -gno-record-gcc-switches"

What did you do?

The commit 6c0135d breaks RELRO as there is no way to override which linker to use when -buildmode=pie is set and the target is linux/amd64. This makes it impossible to build hardened binaries on most Linux platforms.

I'm not sure if it's intentional to force the internal linker? Shoulnd't -linkmode=external ensure we don't use the internal linker?

https://www.redhat.com/en/blog/hardening-elf-binaries-using-relocation-read-only-relro

What did you expect to see?

λ fzf » export GOFLAGS="-buildmode=pie -trimpath -ldflags=-linkmode=external"
λ fzf » export CGO_LDFLAGS="-Wl,-z,relro,-z,now"

λ fzf » CGO_ENABLED=0 go build -o fzf                                         
# github.com/junegunn/fzf
loadinternal: cannot find runtime/cgo
λ fzf » checksec --file=./fzf         
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Partial RELRO   No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   5316) Symbols	  No	0		0	./fzf

λ fzf » CGO_ENABLED=1 go build -o fzf           
λ fzf » checksec --file=./fzf                   
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Full RELRO      Canary found      NX enabled    PIE enabled     No RPATH   No RUNPATH   5396) Symbols	  No	0		2	./fzf

What did you see instead?

λ fzf » export GOFLAGS="-buildmode=pie -trimpath -ldflags=-linkmode=external"
λ fzf » export CGO_LDFLAGS="-Wl,-z,relro,-z,now"


λ fzf » CGO_ENABLED=0 go build -o fzf 
# github.com/junegunn/fzf
loadinternal: cannot find runtime/cgo
λ fzf » checksec --file=./fzf         
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Partial RELRO   No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   4594) Symbols	  No	0		0	./fzf

λ fzf » CGO_ENABLED=1 go build -o fzf 
# github.com/junegunn/fzf
loadinternal: cannot find runtime/cgo
λ fzf » checksec --file=./fzf         
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Partial RELRO   No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   4594) Symbols	  No	0		0	./fzf

[cherrymui]

How do you conclude that external linking is not used?

I think the fact that it printed loadinternal: cannot find runtime/cgo indicated that external linking is actually used, in both cases. You can check it by passing -ldflags=-v. If it includes a line of host link, it means external linking is used, and the external linker's invocation is printed.

[cherrymui]

If the program doesn't use cgo but you want to pass flags to external linker, you can pass -ldflags=-extldflags=....

[Foxboron]

λ fzf » export GOFLAGS="-buildmode=pie -trimpath -ldflags=-linkmode=external -ldflags=-v"

λ fzf » CGO_ENABLED=0 go build -o fzf                                                    
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
96932 symbols, 27963 reachable
	1 package symbols, 36428 hashed symbols, 54613 non-package symbols, 5890 external symbols
135636 liveness data

λ fzf » CGO_ENABLED=1 go build -o fzf                                         
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
96932 symbols, 27963 reachable
	1 package symbols, 36428 hashed symbols, 54613 non-package symbols, 5890 external symbols
135636 liveness data

host link is in neither outputs?

[Foxboron]

Okay, messing up the GOFLAGS env variable (It seems like it doesn't like having multiple -ldflags...)

This is with patch 6c0135d reverted

λ fzf » export GOFLAGS="-buildmode=pie -trimpath"
λ fzf » CGO_ENABLED=0 /home/fox/Git/prosjekter/Go/go/bin/go build -ldflags '-linkmode=external -v' -o fzf
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
loadinternal: cannot find runtime/cgo
host link: "gcc" "-m64" "-Wl,-z,relro" "-pie" "-o" "/tmp/go-build3834599149/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/go-link-286472386/go.o"
96924 symbols, 40660 reachable
	1 package symbols, 36428 hashed symbols, 54613 non-package symbols, 5882 external symbols
135636 liveness data

λ fzf » CGO_ENABLED=1 /home/fox/Git/prosjekter/Go/go/bin/go build -ldflags '-linkmode=external -v' -o fzf
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
host link: "gcc" "-m64" "-Wl,-z,relro" "-pie" "-o" "/tmp/go-build1452289956/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/go-link-964871768/go.o" "/tmp/go-link-964871768/000000.o" "/tmp/go-link-964871768/000001.o" "/tmp/go-link-964871768/000002.o" "/tmp/go-link-964871768/000003.o" "/tmp/go-link-964871768/000004.o" "/tmp/go-link-964871768/000005.o" "/tmp/go-link-964871768/000006.o" "/tmp/go-link-964871768/000007.o" "/tmp/go-link-964871768/000008.o" "/tmp/go-link-964871768/000009.o" "/tmp/go-link-964871768/000010.o" "/tmp/go-link-964871768/000011.o" "/tmp/go-link-964871768/000012.o" "-g" "-O2" "-lpthread"
97087 symbols, 40831 reachable
	1 package symbols, 36438 hashed symbols, 54654 non-package symbols, 5994 external symbols
135848 liveness data

This is with 1.16

λ fzf » export GOFLAGS="-buildmode=pie -trimpath"
λ fzf » CGO_ENABLED=1 go build -ldflags '-linkmode=external -v' -o fzf                                   
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
loadinternal: cannot find runtime/cgo
host link: "gcc" "-m64" "-Wl,-z,relro" "-pie" "-o" "/tmp/go-build3793652355/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/go-link-943503868/go.o"
96924 symbols, 40660 reachable
	1 package symbols, 36428 hashed symbols, 54613 non-package symbols, 5882 external symbols
135636 liveness data

λ fzf » CGO_ENABLED=0 go build -ldflags '-linkmode=external -v' -o fzf
# github.com/junegunn/fzf
HEADER = -H5 -T0x401000 -R0x1000
loadinternal: cannot find runtime/cgo
host link: "gcc" "-m64" "-Wl,-z,relro" "-pie" "-o" "/tmp/go-build3793652355/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/go-link-943503868/go.o"
96924 symbols, 40660 reachable
	1 package symbols, 36428 hashed symbols, 54613 non-package symbols, 5882 external symbols
135636 liveness data

[cherrymui]

As you posted, host like appears in both cases, and the change you reverted doesn't affect this (it changes whether to include runtime/cgo package, but doesn't affect linkmode). So -linkmode=external does work. Is there any more problem?

For the runtime/cgo package, #31544 is related. cmd/go has logic to include runtime/cgo if external linking is requested, but somehow that logic doesn't work (unrelated to the change you reverted).

[Foxboron]

Should we rename this bugreport to be about the patch altering the runtime/cgo inclusion or? It seems weird to me that the patch is unrelated. It has to trigger the mentioned bug in some weird case e.g by changing the code path?

[cherrymui]

It is "related" in that PIE used to always include runtime/cgo, and that change stops doing that. That is intentional, and I don't think we want to revert that. Now it's up to -linkmode=external to include runtime/cgo, but that somehow doesn't always work.

Why would you want to force the inclusion of runtime/cgo?

Should we rename this bugreport to be about the patch altering the runtime/cgo inclusion or?

I think we can combine it to #31544 and close this.

[Foxboron]

Right, I get the issue now. However I think the patch should be reverted since this is quite a severe regression for binary hardening for distributions and introduces faulty/buggy behavior. What would be the downside of reverting the patch?

[cherrymui]

You haven't explain why you'd want to force the inclusion of runtime/cgo. Thanks.

[cherrymui]

Have you tried -ldflags=-extldflags=... as mentioned in #44480 (comment) ? It looks to me that you might want something cgo is included so you can use CGO_LDFLAGS (I may be wrong). If it is only about flags, try that. Thanks.

[Foxboron]

I thought it was clear. Sorry. This solves binary hardening for Arch Linux as passing CGO_LDFLAGS is easier to bypass build systems then passing -ldflags through GOFLAGS as the go compiler does not merge ldflags. It was to my understanding they where equivalent but testing now everything work with -extldflags:

go build -ldflags '-linkmode=external -extldflags=-Wl,-z,relro,-z,now' -o fzf 

This works but passing the equivalent through GOFLAGS is something I have not figured out. I should point out that not all CGO_* flags have replacement flags.

[cherrymui]

Thanks for the reply. So -extldflags works as expected.

GOFLAGS may have problems for parsing flags with spaces. But command line is always available.

PIE implying cgo is never the intention. Forcing the inclusion of C has never been supported (feel free to raise a feature request if you need it). Thanks.

[Foxboron]

It still leaves out a lot to be desired though. You can't realistically pass $GOFLAGS for the options to enable RELRO as ldflags are skipped when upstreams provides their own -X options. But all of this is better suited for it's own issue?

[mvdan]

@Foxboron from memory, you might be interested in #38522.

[Foxboron]

@mvdan Ah yes. That is the crux of the flag issues.

But then I'll close this issue and contemplate reverting the commit downstream in Arch. It's annoying that intended and unintended usage of these flags is never defined and just breaks between releases without any notes in the release notes.