Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: cannot compile Go binary with external linking without cgo with v1.22.0 #65887

Closed
damcadam opened this issue Feb 22, 2024 · 14 comments
Closed
Assignees
Labels
Documentation FixPending Issues that have a fix which has not yet been reviewed or submitted. GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@damcadam
Copy link

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='~/.cache/go-build'
GOENV='~/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='~/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='~/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/snap/go/10506'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/snap/go/10506/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='~/bootboot/mykernel/go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4216832776=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I am trying to make BootBoot from the following path:
https://gitlab.com/bztsrc/bootboot/-/tree/master/mykernel/go?ref_type=heads

I am just running the make command.

$ make

What did you see happen?

I have found that this does not work with Go 1.22.0. The error message from making is shown below.

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

What did you expect to see?

Making this code used to work in Go 1.21.7. I expected to see the following output.

$ make
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
loadinternal: cannot find runtime/cgo
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-strip -s -K mmio -K fb -K bootboot -K environment -K initstack mykernel.aarch64.elf
aarch64-linux-gnu-readelf -hls mykernel.aarch64.elf >mykernel.aarch64.txt

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0, like so:

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

The output from making with CGO enabled while using Go 1.22.0 is also shown below.

$ go version
go version go1.22.0 linux/amd64
$ make
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# runtime/cgo
gcc_arm64.S: Assembler messages:
gcc_arm64.S:30: Error: no such instruction: `stp x29,x30,[sp,'
gcc_arm64.S:34: Error: too many memory references for `mov'
gcc_arm64.S:36: Error: no such instruction: `stp x19,x20,[sp,'
gcc_arm64.S:39: Error: no such instruction: `stp x21,x22,[sp,'
gcc_arm64.S:42: Error: no such instruction: `stp x23,x24,[sp,'
gcc_arm64.S:45: Error: no such instruction: `stp x25,x26,[sp,'
gcc_arm64.S:48: Error: no such instruction: `stp x27,x28,[sp,'
gcc_arm64.S:52: Error: too many memory references for `mov'
gcc_arm64.S:53: Error: too many memory references for `mov'
gcc_arm64.S:54: Error: too many memory references for `mov'
gcc_arm64.S:56: Error: no such instruction: `blr x20'
gcc_arm64.S:57: Error: no such instruction: `blr x19'
gcc_arm64.S:59: Error: no such instruction: `ldp x27,x28,[sp,'
gcc_arm64.S:62: Error: no such instruction: `ldp x25,x26,[sp,'
gcc_arm64.S:65: Error: no such instruction: `ldp x23,x24,[sp,'
gcc_arm64.S:68: Error: no such instruction: `ldp x21,x22,[sp,'
gcc_arm64.S:71: Error: no such instruction: `ldp x19,x20,[sp,'
gcc_arm64.S:74: Error: no such instruction: `ldp x29,x30,[sp],'
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

If I am no longer supposed to be able to use an external linker while compiling with CGO disabled, how can I get past this compilation error?

@Jorropo
Copy link
Member

Jorropo commented Feb 23, 2024

A bit of a blind shot but what happens if you use CC= to make it use gcc aarch64 ?
It could be not working because it's trying to use your host system (amd64) to build gcc_arm64.S. (but aren't theses supposed to be assembled by go's assembler ?)

@Jorropo
Copy link
Member

Jorropo commented Feb 23, 2024

I'm confused by GOARCH=arm64 GOARM=5 I'm pretty sure it's supposed to be GOARM64= for arm64 I don't think GOARM does anything here.

@seankhliao seankhliao changed the title Cannot compile freestanding Go binary with v1.22.0 cmd/link: cannot compile freestanding Go binary with v1.22.0 Feb 23, 2024
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Feb 23, 2024
@seankhliao
Copy link
Member

maybe more like #64875

@cherrymui cherrymui changed the title cmd/link: cannot compile freestanding Go binary with v1.22.0 cmd/go: cannot compile Go binary with external linking without cgo with v1.22.0 Feb 23, 2024
@cherrymui
Copy link
Member

External linking without cgo is fragile. The external linker will usually link in some C objects (like the startup code) anyway, and without cgo the interaction between the C code and Go code may not work correctly. So we require cgo for external linking.

To get cgo working, as @Jorropo mentioned, you'll want to set CC to the C cross compiler.

GOARM=5

It is ignored as the target is not ARM. GOARM64 does not yet exist.

-gcflags=gitlab.com/bztsrc/bootboot=-std

The -std compiler flag is for the Go standard library packages only. Please don't set this flag.

@bcmills maybe we want to mention that cgo is required for external linking in the release notes?

@bcmills
Copy link
Contributor

bcmills commented Feb 23, 2024

This is working as intended — this was the fix for #46330, prior to which the linker would report loadinternal: cannot find runtime/cgo (as @damcadam reported in the Go 1.21.7 output!) but would link the binary and exit 0 despite the error.

I agree that adding a Go 1.22 release note seems appropriate.

@damcadam
Copy link
Author

Here's the output when I run make after setting CC to the cross compiler and removing GOARM=5:

$ make
make: Warning: File 'Makefile' has modification time 244 s in the future
CGO_ENABLED=1 CC=aarch64-linux-gnu-gcc-12 GOOS=linux GOARCH=arm64 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
/snap/go/10506/pkg/tool/linux_amd64/link: running aarch64-linux-gnu-ld failed: exit status 1
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-ld: cannot find -lpthread: file format not recognized

make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

@prattmic prattmic added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Feb 23, 2024
@prattmic prattmic added this to the Backlog milestone Feb 23, 2024
@Jorropo
Copy link
Member

Jorropo commented Feb 23, 2024

@damcadam I think you are in C cross compilation sucks land.
From experience I'm 70% sure you still have your amd64 shared objects configured in the various env variables C toolchains use to find .so files and gcc is trying to link your host amd64 pthread in the arm64 binary. It will be easier to handle that here https://go.dev/wiki/Questions

@bcmills
Copy link
Contributor

bcmills commented Feb 23, 2024

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0

@damcadam, I suggest that you file an issue for that with the bootboot developers. Go does not provide a supported bare-metal port, so building a bare-metal kernel using the Go toolchain is inherently not something the Go project supports either.

See previously:

@mknyszek mknyszek removed the compiler/runtime Issues related to the Go compiler and/or runtime. label Feb 28, 2024
@damcadam
Copy link
Author

Hello, it looks like someone filed an issue with BootBoot that's having the same problem. Here's the link to that:
https://gitlab.com/bztsrc/bootboot/-/issues/87#note_1773739397
The BootBoot developers suggested checking the release log of Go 1.22.0 to see what has changed since Go 1.21.6. The person was also directed to ask on Go forums instead of BootBoot.

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 29, 2024
@gopherbot gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Feb 29, 2024
@bcmills bcmills self-assigned this Mar 5, 2024
@bcmills bcmills added the FixPending Issues that have a fix which has not yet been reviewed or submitted. label Mar 5, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/569296 mentions this issue: _content/doc: mention cgo requirements for external linking in release notes for Go 1.21 and 1.22

@bcmills
Copy link
Contributor

bcmills commented Mar 5, 2024

@damcadam, I'm sorry, but what you're asking for just isn't a supported Go build configuration. Perhaps you can change the BootBoot Go example to build using internal linking instead of external?

@damcadam
Copy link
Author

damcadam commented Mar 6, 2024

@bcmills If building with internal linking will work for Go 1.22 and future versions of Go that would be awesome! Do you have any guidance or resources I could use for using internal linking instead of external linking?

@bcmills bcmills modified the milestones: Backlog, Go1.22.2 Mar 11, 2024
@cherrymui
Copy link
Member

Passing -ldflags=-linkmode=internal will make it use internal linking mode. But it doesn't support linker script. What do you need specifically with the linker script?

@damcadam
Copy link
Author

Sorry about the late reply. I ended up using the -buildmode=c-archive flag for go build and linking all my .o files with the archive file. This works for me, and I don't have any problems. Thanks for considering the issue and for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation FixPending Issues that have a fix which has not yet been reviewed or submitted. GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

8 participants