Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo, cmd/link: with zig as CC/CXX, relocation target not defined, fatal error, SIGSEGV when compiling net/http #44695

Closed
zikaeroh opened this issue Feb 28, 2021 · 19 comments
Labels
NeedsInvestigation
Milestone

Comments

@zikaeroh
Copy link
Contributor

zikaeroh commented Feb 28, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/amd64

Does this issue reproduce with the latest release?

Yes, on tip too.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jake/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/testproj/helloworld/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2412854241=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.16 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.16
uname -sr: Linux 5.11.2-zen1-1-zen
/usr/lib/libc.so.6: GNU C Library (GNU libc) release release version 2.33.
gdb --version: GNU gdb (GDB) 10.1

What did you do?

package main

import _ "net/http"

func main() {}

In my PATH, I have two scripts to work around #43078:

#!/bin/sh
exec zig cc $@
#!/bin/sh
exec zig c++ $@

Then, I compiled with CC="zcc" CXX="zxx" go build.

What did you expect to see?

A successful compilation.

What did you see instead?

# some.tld/helloworld
net(.text): relocation target __errno_location not defined
net(.text): relocation target getaddrinfo not defined
net(.text): relocation target freeaddrinfo not defined
net(.text): relocation target gai_strerror not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target fwrite not defined
runtime/cgo(.text): relocation target vfprintf not defined
runtime/cgo(.text): relocation target fputc not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_create not defined
runtime/cgo(.text): relocation target nanosleep not defined
runtime/cgo(.text): relocation target pthread_detach not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target fprintf not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_mutex_lock not defined
runtime/cgo(.text): relocation target pthread_cond_wait not defined
runtime/cgo(.text): relocation target pthread_mutex_unlock not defined
runtime/cgo(.text): relocation target pthread_cond_broadcast not defined
runtime/cgo(.text): relocation target malloc not defined
/usr/lib/go/pkg/tool/linux_amd64/link: too many errors
unexpected fault address 0x7f6e97cabafe
fatal error: fault
unexpected fault address 0x7f6e97df1cf0
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7f6e97cabafe pc=0x4eb5c1]

goroutine 258 [running]:
runtime.throw(0x6dcc2d, 0x5)
	/usr/lib/go/src/runtime/panic.go:1117 +0x72 fp=0xc0009f3b08 sp=0xc0009f3ad8 pc=0x436e52
runtime.sigpanic()
	/usr/lib/go/src/runtime/signal_unix.go:741 +0x268 fp=0xc0009f3b40 sp=0xc0009f3b08 pc=0x44e548
encoding/binary.littleEndian.PutUint32(...)
	/usr/lib/go/src/encoding/binary/binary.go:73
encoding/binary.(*littleEndian).PutUint32(0x8be3c0, 0x7f6e97cabafe, 0xadc, 0x246fae, 0x4233e)
	<autogenerated>:1 +0x41 fp=0xc0009f3b60 sp=0xc0009f3b40 pc=0x4eb5c1
cmd/link/internal/ld.(*relocSymState).relocsym(0xc0009f3ea0, 0x552f, 0x7f6e97cab120, 0x14ba, 0x24798c)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:534 +0x703 fp=0xc0009f3de8 sp=0xc0009f3b60 pc=0x584183
cmd/link/internal/ld.writeBlock(0xc000114000, 0xc000e2a070, 0xc00050e000, 0xc001960418, 0x187, 0x37d, 0x507120, 0x2de50, 0xc0000b80c5, 0x1, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:953 +0x1e7 fp=0xc0009f3f10 sp=0xc0009f3de8 pc=0x58a2a7
cmd/link/internal/ld.writeBlocks.func1(0xc000114000, 0xc0023e0000, 0xc000112000, 0xc000e2a070, 0xc00050e000, 0xc001960418, 0x187, 0x37d, 0x501220, 0x2de50, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:909 +0xb1 fp=0xc0009f3f78 sp=0xc0009f3f10 pc=0x609af1
runtime.goexit()
	/usr/lib/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0009f3f80 sp=0xc0009f3f78 pc=0x46c9e1
created by cmd/link/internal/ld.writeBlocks
	/usr/lib/go/src/cmd/link/internal/ld/data.go:908 +0x565

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc0025420a8)
	/usr/lib/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0025420a0)
	/usr/lib/go/src/sync/waitgroup.go:130 +0x65
cmd/link/internal/ld.asmb(0xc000114000)
	/usr/lib/go/src/cmd/link/internal/ld/asmb.go:73 +0x235
cmd/link/internal/ld.Main(0x87f340, 0x20, 0x20, 0x1, 0x7, 0x10, 0x0, 0x0, 0x6e865e, 0x1b, ...)
	/usr/lib/go/src/cmd/link/internal/ld/main.go:339 +0x1465
main.main()
	/usr/lib/go/src/cmd/link/main.go:68 +0x258

goroutine 239 [semacquire]:
sync.runtime_Semacquire(0xc0023e0008)
	/usr/lib/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0023e0000)
	/usr/lib/go/src/sync/waitgroup.go:130 +0x65
cmd/link/internal/ld.writeBlocks(0xc000114000, 0xc00010f340, 0xc000112000, 0xc00050e000, 0xc001960418, 0x187, 0x37d, 0x501220, 0x12e070, 0xc0000b80c5, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:924 +0x736
cmd/link/internal/ld.CodeblkPad(...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:834
cmd/link/internal/ld.asmb.func1(0xc000114000, 0xc00010f340, 0x401000, 0x12e070)
	/usr/lib/go/src/cmd/link/internal/ld/asmb.go:39 +0xd3
cmd/link/internal/ld.writeParallel.func1(0xc0025420a0, 0x6f43d0, 0xc000114000, 0xc00010f340, 0x401000, 0x12e070)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:988 +0x7a
created by cmd/link/internal/ld.writeParallel
	/usr/lib/go/src/cmd/link/internal/ld/data.go:986 +0x152

goroutine 257 [runnable]:
syscall.Syscall(0xb, 0x7f6e97ba4000, 0x34eaac, 0x0, 0x0, 0x0, 0x0)
	/usr/lib/go/src/syscall/asm_linux_amd64.s:20 +0x5
syscall.munmap(0x7f6e97ba4000, 0x34eaac, 0x7f6e97ef2aab, 0xc0001301c0)
	/usr/lib/go/src/syscall/zsyscall_linux_amd64.go:957 +0x4b
syscall.(*mmapper).Munmap(0x87b3a0, 0x7f6e97ba4000, 0x34eaac, 0x34eaac, 0x0, 0x0)
	/usr/lib/go/src/syscall/syscall_unix.go:94 +0x151
syscall.Munmap(...)
	/usr/lib/go/src/syscall/syscall_linux.go:1234
cmd/link/internal/ld.(*OutBuf).munmap(...)
	/usr/lib/go/src/cmd/link/internal/ld/outbuf_mmap.go:57
cmd/link/internal/ld.(*OutBuf).Close(0xc00010e070, 0x37, 0x0)
	/usr/lib/go/src/cmd/link/internal/ld/outbuf.go:117 +0x179
cmd/link/internal/ld.linknew.func1()
	/usr/lib/go/src/cmd/link/internal/ld/sym.go:62 +0x45
cmd/link/internal/ld.runAtExitFuncs(...)
	/usr/lib/go/src/cmd/link/internal/ld/util.go:23
cmd/link/internal/ld.Exit(0x2)
	/usr/lib/go/src/cmd/link/internal/ld/util.go:30 +0x3b
cmd/link/internal/ld.Exitf(0x6e1a43, 0xf, 0x0, 0x0, 0x0)
	/usr/lib/go/src/cmd/link/internal/ld/util.go:38 +0x106
cmd/link/internal/ld.afterErrorAction()
	/usr/lib/go/src/cmd/link/internal/ld/util.go:49 +0x7a
cmd/link/internal/loader.(*ErrorReporter).Errorf(0xc000114010, 0x23c7a, 0x6eaffe, 0x20, 0xc0009e9ae8, 0x1, 0x1)
	/usr/lib/go/src/cmd/link/internal/loader/loader.go:2619 +0x143
cmd/link/internal/ld.(*ErrorReporter).errorUnresolved(0xc000114010, 0xc00050e000, 0x23c7a, 0x23c85)
	/usr/lib/go/src/cmd/link/internal/ld/errors.go:63 +0x4e9
cmd/link/internal/ld.(*relocSymState).relocsym(0xc0009e9ea0, 0x23c7a, 0x7f6e97ba5400, 0x18b, 0x34d6ac)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:199 +0x338a
cmd/link/internal/ld.writeBlock(0xc000114000, 0xc000e2a000, 0xc00050e000, 0xc00195c000, 0xa0a, 0xc00, 0x401400, 0x100220, 0xc0000b80c5, 0x1, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:953 +0x1e7
cmd/link/internal/ld.writeBlocks.func1(0xc000114000, 0xc0023e0000, 0xc000112000, 0xc000e2a000, 0xc00050e000, 0xc00195c000, 0xa0a, 0xc00, 0x401000, 0x100220, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:909 +0xb1
created by cmd/link/internal/ld.writeBlocks
	/usr/lib/go/src/cmd/link/internal/ld/data.go:908 +0x565

goroutine 242 [semacquire]:
sync.runtime_Semacquire(0xc00249a018)
	/usr/lib/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc00249a010)
	/usr/lib/go/src/sync/waitgroup.go:130 +0x65
cmd/link/internal/ld.writeBlocks(0xc000114000, 0xc00010f490, 0xc000112000, 0xc00050e000, 0xc0024b2640, 0x6bd, 0x6bd, 0x649000, 0x331e0, 0x8bf2e0, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:924 +0x736
cmd/link/internal/ld.writeDatblkToOutBuf(...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:1006
cmd/link/internal/ld.datblk(0xc000114000, 0xc00010f490, 0x649000, 0x331e0)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:994 +0xad
cmd/link/internal/ld.writeParallel.func1(0xc0025420a0, 0x6f43d8, 0xc000114000, 0xc00010f490, 0x649000, 0x331e0)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:988 +0x7a
created by cmd/link/internal/ld.writeParallel
	/usr/lib/go/src/cmd/link/internal/ld/data.go:986 +0x152

goroutine 35 [running]:
	goroutine running on other thread; stack unavailable
created by cmd/link/internal/ld.writeBlocks
	/usr/lib/go/src/cmd/link/internal/ld/data.go:908 +0x565

goroutine 241 [runnable]:
cmd/link/internal/loader.(*Loader).OuterSym(...)
	/usr/lib/go/src/cmd/link/internal/loader/loader.go:1733
cmd/link/internal/loader.(*Loader).AttrSubSymbol(...)
	/usr/lib/go/src/cmd/link/internal/loader/loader.go:1157
cmd/link/internal/ld.writeBlocks(0xc000114000, 0xc00010f420, 0xc000112000, 0xc00050e000, 0xc00249e008, 0x2f84, 0x2f84, 0x530000, 0x118e68, 0x8bf2e0, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:858 +0x1c7
cmd/link/internal/ld.writeDatblkToOutBuf(...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:1006
cmd/link/internal/ld.datblk(0xc000114000, 0xc00010f420, 0x530000, 0x118e68)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:994 +0xad
cmd/link/internal/ld.writeParallel.func1(0xc0025420a0, 0x6f43d8, 0xc000114000, 0xc00010f420, 0x530000, 0x118e68)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:988 +0x7a
created by cmd/link/internal/ld.writeParallel
	/usr/lib/go/src/cmd/link/internal/ld/data.go:986 +0x152
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7f6e97df1cf0 pc=0x4eb642]

goroutine 35 [running]:
runtime.throw(0x6dcc2d, 0x5)
	/usr/lib/go/src/runtime/panic.go:1117 +0x72 fp=0xc0009e4b08 sp=0xc0009e4ad8 pc=0x436e52
runtime.sigpanic()
	/usr/lib/go/src/runtime/signal_unix.go:741 +0x268 fp=0xc0009e4b40 sp=0xc0009e4b08 pc=0x44e548
encoding/binary.littleEndian.PutUint64(...)
	/usr/lib/go/src/encoding/binary/binary.go:91
encoding/binary.(*littleEndian).PutUint64(0x8be3c0, 0x7f6e97df1cf0, 0x18, 0x100dbc, 0x4d04c0)
	<autogenerated>:1 +0x42 fp=0xc0009e4b60 sp=0xc0009e4b40 pc=0x4eb642
cmd/link/internal/ld.(*relocSymState).relocsym(0xc0009e4ea0, 0x7346, 0x7f6e97df1c60, 0xa8, 0x100e4c)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:536 +0x9f8 fp=0xc0009e4de8 sp=0xc0009e4b60 pc=0x584478
cmd/link/internal/ld.writeBlock(0xc000114000, 0xc00257c070, 0xc00050e000, 0xc0024b2640, 0x6bd, 0x6bd, 0x64dc60, 0x331e0, 0x8bf2e0, 0x200, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:953 +0x1e7 fp=0xc0009e4f10 sp=0xc0009e4de8 pc=0x58a2a7
cmd/link/internal/ld.writeBlocks.func1(0xc000114000, 0xc00249a010, 0xc000112000, 0xc00257c070, 0xc00050e000, 0xc0024b2640, 0x6bd, 0x6bd, 0x649000, 0x331e0, ...)
	/usr/lib/go/src/cmd/link/internal/ld/data.go:909 +0xb1 fp=0xc0009e4f78 sp=0xc0009e4f10 pc=0x609af1
runtime.goexit()
	/usr/lib/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0009e4f80 sp=0xc0009e4f78 pc=0x46c9e1
created by cmd/link/internal/ld.writeBlocks
	/usr/lib/go/src/cmd/link/internal/ld/data.go:908 +0x565

If I change my code to just import net only, the panic goes away, but it still fails:

# some.tld/helloworld
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target fwrite not defined
runtime/cgo(.text): relocation target vfprintf not defined
runtime/cgo(.text): relocation target fputc not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_create not defined
runtime/cgo(.text): relocation target nanosleep not defined
runtime/cgo(.text): relocation target pthread_detach not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target fprintf not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_mutex_lock not defined
runtime/cgo(.text): relocation target pthread_cond_wait not defined
runtime/cgo(.text): relocation target pthread_mutex_unlock not defined
runtime/cgo(.text): relocation target pthread_cond_broadcast not defined
runtime/cgo(.text): relocation target malloc not defined
runtime/cgo(.text): relocation target __errno_location not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target pthread_attr_init not defined
runtime/cgo(.text): relocation target pthread_attr_getstacksize not defined
/usr/lib/go/pkg/tool/linux_amd64/link: too many errors

I don't quite know the issue prefix for the title, or if this is two issues together (the relocation failing and the linker panicking/segfaulting). It's not clear to me if this is an issue with Go or with zig either; others have gotten this to work just fine (given the other thread and blog posts about it), but I have not. Compiling with GOOS=windows or GOOS=darwin with this setup works; only Linux appears to have troubles.

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Mar 1, 2021

Can you confirm how are you building for windows and darwin?
Cause if you are cross compiling you need to set CGO_ENABLED="1" or go will not use cgo and thus not use zig when compiling.
Which would explain why windows and darwin compiled fine.

@zikaeroh
Copy link
Contributor Author

zikaeroh commented Mar 1, 2021

Yes, for Windows I get:

$ CGO_ENABLED=1 CC="zcc" CXX="zxx" GOOS="windows" go build
$ ls
go.mod
helloworld.exe
main.go

Darwin actually did fail, it just wasn't clear.

$ CGO_ENABLED=1 CC="zcc" CXX="zxx" GOOS="darwin" go build 
# runtime/cgo
zig: error: argument unused during compilation: '-arch x86_64' [-Werror,-Wunused-command-line-argument]

@zikaeroh
Copy link
Contributor Author

zikaeroh commented Mar 1, 2021

But, I think that my Windows example isn't actually using cgo; I did a go build -x and grepped the output; zcc was never invoked, so my use of net/http doesn't actually bring in CGo, probably because Windows doesn't use the system resolver.

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Mar 1, 2021

There's a issue for it not working on windows. #43886

@zikaeroh
Copy link
Contributor Author

zikaeroh commented Mar 1, 2021

Yes, thanks. I had seen that, but I wanted to start with my Linux example. We can just ignore my Darwin and Windows comments 🙂

@dmitshur dmitshur changed the title cgo, link: with zig as CC/CXX, relocation target not defined, fatal error, SIGSEGV when compiling net/http cmd/cgo, cmd/link: with zig as CC/CXX, relocation target not defined, fatal error, SIGSEGV when compiling net/http Mar 2, 2021
@dmitshur
Copy link
Contributor

dmitshur commented Mar 5, 2021

CC @cherrymui, @mdempsky, @ianlancetaylor.

@dmitshur dmitshur added this to the Backlog milestone Mar 5, 2021
@dmitshur dmitshur added the NeedsInvestigation label Mar 5, 2021
@motiejus
Copy link
Contributor

motiejus commented Jun 8, 2021

Another datapoint: I can't use cgo+zig to cross-compile from amd64-linux to amd64-darwin:

EDIT: created #46644.

@andrew-daly
Copy link

andrew-daly commented Oct 26, 2021

I encountered this error myself in the past few days and ended up looking into it.

I think this is that -linkmode=internal, which ends up getting set if the binary you are compiling isn't using cgo itself, doesn't work with zig. It appears to work with external linking

The likely cause is zig cc --print-libgcc-file-name just returns libgcc.a which is useless; the internal linkmode relies on this library to lookup undefined symbols. see https://github.com/golang/go/blob/master/src/cmd/link/internal/ld/lib.go#L596

$ CGO_ENABLED=1 CC=zcc GOOS=linux go build -ldflags '-v' .
# example.com/mod
HEADER = -H5 -T0x401000 -R0x1000
zcc [-m64 --print-libgcc-file-name]
skipping libgcc file: open libgcc.a: no such file or directory
net(.text): relocation target __errno_location not defined
... (more errors)

If I set the linkmode myself it works:

$ CGO_ENABLED=1 CC=zcc GOOS=linux go build -ldflags '-v -linkmode=external' .
# example.com/mod
HEADER = -H5 -T0x401000 -R0x1000
host link: "zcc" "-m64" "-o" "/tmp/nix-shell.yJ5EkG/go-build3511243503/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/nix-shell.yJ5EkG/go-link-213700306/go.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000000.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000001.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000002.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000003.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000004.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000005.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000006.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000007.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000008.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000009.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000010.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000011.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000012.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000013.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000014.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000015.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000016.o" "/tmp/nix-shell.yJ5EkG/go-link-213700306/000017.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-no-pie"
warning: unsupported linker arg: --compress-debug-sections
warning: unsupported linker arg: zlib-gnu
151246 symbols, 41825 reachable
	1 package symbols, 58457 hashed symbols, 85520 non-package symbols, 7268 external symbols
126752 liveness data

@zikaeroh
Copy link
Contributor Author

zikaeroh commented Oct 26, 2021

Maybe I'm misunderstanding, but when I do zig cc --print-libgcc-file-name, I get:

/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/11.1.0/libgcc.a

Which is a real 2.9MB file that exists. gcc with the same flag gives the same file.

@andrew-daly
Copy link

andrew-daly commented Oct 27, 2021

That was a different problem I was seeing and unrelated to this, sorry.

@uhthomas
Copy link

uhthomas commented Nov 19, 2021

As @andrew-daly observed, this behaviour occurs when compiling binaries where cgo is enabled but C is not actually being used. For example, the following will fail to build:

package main

import _ "runtime/cgo"

func main() {}
❯ CGO_ENABLED=1 GOARCH=amd64 GOOS=linux CC="$(pwd)/zcc" CXX="$(pwd)/zxx" go build main.go
# command-line-arguments
runtime.gcdata: missing Go type information for global symbol .dynsym: size 72
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target fwrite not defined
runtime/cgo(.text): relocation target vfprintf not defined
runtime/cgo(.text): relocation target fputc not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_create not defined
runtime/cgo(.text): relocation target nanosleep not defined
runtime/cgo(.text): relocation target pthread_detach not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target fprintf not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_mutex_lock not defined
runtime/cgo(.text): relocation target pthread_cond_wait not defined
runtime/cgo(.text): relocation target pthread_mutex_unlock not defined
runtime/cgo(.text): relocation target pthread_cond_broadcast not defined
runtime/cgo(.text): relocation target malloc not defined
runtime/cgo(.text): relocation target __errno_location not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target pthread_attr_init not defined
/usr/local/Cellar/go/1.17.2/libexec/pkg/tool/darwin_amd64/link: too many errors
unexpected fault address 0x98ba1e9
fatal error: fault

However, simply importing the C package resolves the issue:

package main

import "C"

import _ "runtime/cgo"

func main() {}

Alternatively, configuring go build to use use link external will also resolve the issue. (again as @andrew-daly observed)

❯ CGO_ENABLED=1 GOARCH=amd64 GOOS=linux CC="$(pwd)/zcc" CXX="$(pwd)/zxx" go build -ldflags=-linkmode=external main.go

For context the host is macos/amd64 targeting linux/amd64.

For Bazel rules_go users, setting linkmode = "pie" on a go_binary rule will have the same effect. Alternatively, the PIE buildmode can be set globally with a flag or with a bazelrc file.

build --@io_bazel_rules_go//go/config:linkmode=pie

Should this issue be closed? It looks like we've found a resolution and appears to be more of a Zig thing, rather than Go? Please correct me if I'm wrong.

Thank you all.

@cherrymui
Copy link
Member

cherrymui commented Nov 19, 2021

@uhthomas it is unusual for user code to import runtime/cgo directly like that. I'm not familiar with Bazel, but if Bazel adds an import like that, I'd suggest we revisit that.

In the past PIE linking requires cgo, but at least on some platforms (including linux/amd64 and darwin/amd64) that has not been the case quite a few releases ago, and the Go linker can link a PIE binary just fine (if cgo is not otherwise used). So if Bazel forces it to use cgo for PIE, maybe it shouldn't now.

@uhthomas
Copy link

uhthomas commented Nov 19, 2021

Apologies, I used runtime/cgo as it's a dependency of net/http and wanted to highlight that it's not an issue with net/http directly. This means that any code which either explicitly or implicitly depends on C (i.e runtime/cgo) will fail to build as explained in my previous comment.

It looks like rules_go sets linkmode=external for any non-standard build mode regardless of whether it's needed or not as per bazelbuild/rules_go#2630.

@cherrymui
Copy link
Member

cherrymui commented Nov 19, 2021

Thanks. It looks like when external linking is used it works, but it doesn't work when it is not? Perhaps zcc produces object files that the Zig linker understands, but the Go linker doesn't? Does zcc produce standard ELF/Mach-O/PE/etc. object files, or it has its own custom object file?

@uhthomas
Copy link

uhthomas commented Nov 26, 2021

@cherrymui Sorry for the late reply, yes this is correct. Compilation with Zig CC is not possible without external linking. This may be expected behaviour, I'm not sure.

Unfortunately I am not aware of the specifics of how Zig CC works but as I understand it shouldn't behave any differently than glibc or musl.

motiejus added a commit to motiejus/zig that referenced this issue May 4, 2022
When building object files, `zig cc` will instruct lld to remove unused
sections via `--gc-sections`. This is problematic for cgo. Briefly, go
builds cgo executables as follows*:

1. build `_cgo_.o`, which links (on linux_amd64 systems) to
   `/usr/local/go/src/runtime/race/race_linux_amd64.syso`.
2. That `.syso` contains references to symbols from libc.

If the actual Go program uses at least one libc symbol, it will link
correctly. However, if Go is building a cgo executable, but without any
C code, the sections from `.syso` file will be garbage-collected,
leaving a `_cgo_.o` without any references to libc.

I assume the `gc_sections` is an optimization. If yes, then it makes
sense for the final executable, but not for the intermediate object. If
not, please correct me.

Quoting @andrewrk in [1]:

> The C source code for the temporary executable needs to have dummy
> calls to getuid, pthread_self, sleep, and every other libc function
> that cgo/race code wants to call.

I agree in this case. However, while we could potentially fix it for go,
I don't know how many other systems do that, which compilcates use of
`zig cc` for other projects. If we consider `zig cc` a drop-in clang
replacement (except for `-fsanitize=undefined`, which I tend to agree
with), then it should not be optimizing the intermediate object files.

I assume this was added as an optimization. If that's correct, let's
optimize the final executable, but not the intermediate objects.

Fixes ziglang#11398
Fixes golang/go#44695
Fixes golang/go#52690

[*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v`

[1]: golang/go#52690 (comment)
@motiejus
Copy link
Contributor

motiejus commented May 4, 2022

I created ziglang/zig#11577 for further discussion and possible resolution.

With today's zig and go 1.18 you can compile the original file as such:

CGO_ENABLED=1 CC="zig cc -Wl,--no-gc-sections" go build main.go

motiejus added a commit to motiejus/zig that referenced this issue May 4, 2022
When building object files, `zig cc` will instruct lld to remove unused
sections via `--gc-sections`. This is problematic cgo builds that don't
explicitly use C code. Briefly, go builds cgo executables as follows*:

1. build `_cgo_.o`, which links (on linux_amd64 systems) to
   `/usr/local/go/src/runtime/race/race_linux_amd64.syso`.
2. That `.syso` contains references to symbols from libc.

If the user program uses at least one libc symbol, it will link
correctly. However, if Go is building a cgo executable, but without C
code, the sections from `.syso` file will be garbage-collected, leaving
a `_cgo_.o` without any references to libc, causing the final linking
step to not link libc.

Until now, this could be worked around by `-linkmode external` flag to
`go build`. This causes Go to link the final executable using the
external linker (which implicitly links libc). However, that flag brings
in a whole different world of worms.

I assume the `gc_sections` is an optimization; I tried to re-add
`--gc-sections` to the final executable, but that didn't go well. I know
removing such an optimization may be contentious, so let's start the
discussion here. Quoting @andrewrk in [1] (it was about `--as-needed`,
but the point remains the same):

> The C source code for the temporary executable needs to have dummy
> calls to getuid, pthread_self, sleep, and every other libc function
> that cgo/race code wants to call.

I agree this is how it *should* work. However, while we could fix it for
go, I don't know how many other systems rely on that, and we'll never
know we've fixed the last one. The point is, GCC/Clang does not optimize
sections by default, and downstream tools rely on that. If we want to
consider `zig cc` a drop-in clang replacement (except for
`-fsanitize=undefined`, which I tend to agree with), then it should not
be optimizing the intermediate object files. Or at least have a very
prominent fine-print that this is happening, with ways to work around
it.

Fixes ziglang#11398
Fixes golang/go#44695
Fixes golang/go#52690

[*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v`

[1]: golang/go#52690 (comment)
motiejus added a commit to motiejus/zig that referenced this issue May 5, 2022
When building object files, `zig cc` will instruct lld to remove unused
sections via `--gc-sections`. This is problematic cgo builds that don't
explicitly use C code. Briefly, go builds cgo executables as follows*:

1. build `_cgo_.o`, which links (on linux_amd64 systems) to
   `/usr/local/go/src/runtime/race/race_linux_amd64.syso`.
2. That `.syso` contains references to symbols from libc.

If the user program uses at least one libc symbol, it will link
correctly. However, if Go is building a cgo executable, but without C
code, the sections from `.syso` file will be garbage-collected, leaving
a `_cgo_.o` without any references to libc, causing the final linking
step to not link libc.

Until now, this could be worked around by `-linkmode external` flag to
`go build`. This causes Go to link the final executable using the
external linker (which implicitly links libc). However, that flag brings
in a whole different world of worms.

I assume the `gc_sections` is an optimization; I tried to re-add
`--gc-sections` to the final executable, but that didn't go well. I know
removing such an optimization may be contentious, so let's start the
discussion here. Quoting @andrewrk in [1] (it was about `--as-needed`,
but the point remains the same):

> The C source code for the temporary executable needs to have dummy
> calls to getuid, pthread_self, sleep, and every other libc function
> that cgo/race code wants to call.

I agree this is how it *should* work. However, while we could fix it for
go, I don't know how many other systems rely on that, and we'll never
know we've fixed the last one. The point is, GCC/Clang does not optimize
sections by default, and downstream tools rely on that. If we want to
consider `zig cc` a drop-in clang replacement (except for
`-fsanitize=undefined`, which I tend to agree with), then it should not
be optimizing the intermediate object files. Or at least have a very
prominent fine-print that this is happening, with ways to work around
it.

Fixes ziglang#11398
Fixes golang/go#44695
Fixes golang/go#52690

[*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v`

[1]: golang/go#52690 (comment)
@uhthomas
Copy link

uhthomas commented May 5, 2022

FWIW @motiejus even with --no-gc-sections (bazel-zig-cc v0.7.1), I still see:

(14:16:17) SUBCOMMAND: # //some_binary [action 'GoLink some_binary', configuration: fca3f55584ff84b9284596d140596f52e5a6c0306cb5b1c9b5df1bcc917e2439, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_thomas/796dd52d4d981101108548a9446ddca6/execroot/<redacted> && \
  exec env - \
    CGO_ENABLED=1 \
    GOARCH=amd64 \
    GOOS=linux \
    GOPATH='' \
    GOROOT=bazel-out/darwin_arm64-fastbuild-ST-03006dd95b75/bin/external/io_bazel_rules_go/stdlib_ \
    GOROOT_FINAL=GOROOT \
    PATH=/private/var/tmp/_bazel_thomas/796dd52d4d981101108548a9446ddca6/external/zig_sdk/tools:/bin:/usr/bin \
  bazel-out/darwin_arm64-opt-exec-2B5CBBC6-ST-625e526ca8a8/bin/external/go_sdk/builder '-param=bazel-out/darwin_arm64-fastbuild-ST-4a519fd6d3e4/bin/some_binary-0.params' -- -extld /private/var/tmp/_bazel_thomas/796dd52d4d981101108548a9446ddca6/external/zig_sdk/tools/c++ '-buildid=redacted' -extldflags '-target x86_64-linux-musl -Wl,--no-gc-sections -s -w')
# Configuration: fca3f55584ff84b9284596d140596f52e5a6c0306cb5b1c9b5df1bcc917e2439
# Execution platform: @local_config_platform//:host
(14:16:17) ERROR: /Users/thomas/<redacted>/some_binary/BUILD.bazel:45:9: GoLink some_binary failed: (Exit 1): builder failed: error executing command bazel-out/darwin_arm64-opt-exec-2B5CBBC6-ST-625e526ca8a8/bin/external/go_sdk/builder ... (remaining 7 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox
runtime.gcdata: missing Go type information for global symbol .dynsym: size 72
net(.text): relocation target __errno_location not defined
net(.text): relocation target getnameinfo not defined
net(.text): relocation target __errno_location not defined
net(.text): relocation target getaddrinfo not defined
net(.text): relocation target freeaddrinfo not defined
net(.text): relocation target gai_strerror not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target fwrite not defined
runtime/cgo(.text): relocation target vfprintf not defined
runtime/cgo(.text): relocation target fputc not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_create not defined
runtime/cgo(.text): relocation target nanosleep not defined
runtime/cgo(.text): relocation target pthread_detach not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target strerror not defined
runtime/cgo(.text): relocation target fprintf not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target __stack_chk_fail not defined
runtime/cgo(.text): relocation target pthread_mutex_lock not defined
external/go_sdk/pkg/tool/darwin_arm64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/darwin_arm64/link: exit status 2

Using PIE seems to fix things as usual.

@motiejus
Copy link
Contributor

motiejus commented May 6, 2022

FWIW @motiejus even with --no-gc-sections (bazel-zig-cc v0.7.1), I still see:

We have updated bazel-zig-cc in our monorepo and can't see this error any more.

Please bring this up in the mailing list, so we can investigate this properly. I am very interested on why it would still be failing.

If you can reproduce it like in this test, it would give most data (and a ready test case to put to bazel-zig-cc).

@zikaeroh
Copy link
Contributor Author

zikaeroh commented May 20, 2022

Given all of the investigation that led to finding the problem happened on #52690 instead and that CL 405414 fixes this problem, I'm just going to close my issue for this as a dupe of #52690 and let that one be the canonical issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants