Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: cannot run cross compiled ARM binary on QEMU #20763

Closed
DeadNumbers opened this issue Jun 23, 2017 · 7 comments

Comments

Projects
None yet
7 participants
@DeadNumbers
Copy link

commented Jun 23, 2017

What version of Go are you using (go version)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build057525304=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

I'm try compile for ARMv* and test it on QEMU (qemu version 2.9.0)

Source

$ cat hello.go
package main

import "fmt"

func main() {
	fmt.Printf("Hello ARM world!\n")
}

Build GOARCH=arm64 go build -o hello_go hello.go and run

$ qemu-aarch64 hello_go 
runtime: failed to create new OS thread (have 2 already; errno=22)
fatal error: newosproc

runtime stack:
runtime.throw(0xb8109, 0x9)
	/usr/lib/go/src/runtime/panic.go:596 +0x70
runtime.newosproc(0x1442002a000, 0x1442003a000)
	/usr/lib/go/src/runtime/os_linux.go:163 +0x168
runtime.newm(0xbe310, 0x0)
	/usr/lib/go/src/runtime/proc.go:1628 +0x110
runtime.main.func1()
	/usr/lib/go/src/runtime/proc.go:126 +0x28
runtime.systemstack(0x114a00)
	/usr/lib/go/src/runtime/asm_arm64.s:241 +0x90
runtime.mstart()
	/usr/lib/go/src/runtime/proc.go:1132

goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/lib/go/src/runtime/asm_arm64.s:190 +0x8 fp=0x14420026780 sp=0x14420026770
runtime.main()
	/usr/lib/go/src/runtime/proc.go:127 +0x58 fp=0x144200267d0 sp=0x14420026780
runtime.goexit()
	/usr/lib/go/src/runtime/asm_arm64.s:981 +0x4 fp=0x144200267d0 sp=0x144200267d0

For ARMv5 GOARCH=arm go build -o hello_go hello.go

$ qemu-arm hello_go                  
runtime: failed to create new OS thread (have 2 already; errno=22)
fatal error: newosproc

runtime stack:
runtime.throw(0xaba2e, 0x9)
	/usr/lib/go/src/runtime/panic.go:596 +0x70
runtime.newosproc(0x10426000, 0x10436000)
	/usr/lib/go/src/runtime/os_linux.go:163 +0x158
runtime.newm(0xb1c98, 0x0)
	/usr/lib/go/src/runtime/proc.go:1628 +0x10c
runtime.main.func1()
	/usr/lib/go/src/runtime/proc.go:126 +0x24
runtime.systemstack(0x103000)
	/usr/lib/go/src/runtime/asm_arm.s:264 +0x80
runtime.mstart()
	/usr/lib/go/src/runtime/proc.go:1132

goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/lib/go/src/runtime/asm_arm.s:209 +0x4 fp=0x104227bc sp=0x104227b8
runtime.main()
	/usr/lib/go/src/runtime/proc.go:127 +0x58 fp=0x104227e4 sp=0x104227bc
runtime.goexit()
	/usr/lib/go/src/runtime/asm_arm.s:1017 +0x4 fp=0x104227e4 sp=0x104227e4

I'm not sure that the error in the Go compiler, but

#include <stdio.h>

int main(void) {
    printf("Hello ARM world!\n");
}

$ aarch64-linux-gnu-gcc hello.c -static -o hello_c && qemu-aarch64 hello_c
Hello ARM world!

@odeke-em odeke-em changed the title ARM binary can't run on QEMU cmd/go: cannot run cross compiled ARM binary on QEMU Jun 23, 2017

@odeke-em

This comment has been minimized.

Copy link
Member

commented Jun 23, 2017

@odeke-em odeke-em changed the title cmd/go: cannot run cross compiled ARM binary on QEMU runtime: cannot run cross compiled ARM binary on QEMU Jun 23, 2017

@ALTree

This comment has been minimized.

Copy link
Member

commented Jun 23, 2017

qemu in user-mode emulation is not supported, and I don't think it ever worked well (at least not in recent times). See this comment #13024 (comment)

@bradfitz bradfitz added this to the Unplanned milestone Jun 23, 2017

@bradfitz

This comment has been minimized.

Copy link
Member

commented Jun 23, 2017

Working around QEMU bugs is not a priority for the Go team so I won't target this for any release, but we can keep it open for tracking. We test on real hardware. And QEMU in full system emulation mode works as well. But if somebody else wants to investigate this, it's all yours.

@ncw

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2017

Here is a patch for go which fixes the issue.

It might even be correct but I don't really know!

Maybe the issue should be re-opened?

See also ubuntu bug tracker.

diff --git a/src/runtime/os_linux.go b/src/runtime/os_linux.go
index a6efc0e3d1..64218e3f7e 100644
--- a/src/runtime/os_linux.go
+++ b/src/runtime/os_linux.go
@@ -132,7 +132,8 @@ const (
 		_CLONE_FS | /* share cwd, etc */
 		_CLONE_FILES | /* share fd table */
 		_CLONE_SIGHAND | /* share sig handler table */
-		_CLONE_THREAD /* revisit - okay for now */
+		_CLONE_THREAD | /* revisit - okay for now */
+		_CLONE_SYSVSEM
 )
 
 //go:noescape
@bradfitz

This comment has been minimized.

Copy link
Member

commented Jul 12, 2017

Sure, I'll reopen given the renewed discussion and investigation in that Ubuntu bug.

@aclements, @ianlancetaylor?

@bradfitz bradfitz reopened this Jul 12, 2017

@bradfitz bradfitz modified the milestones: Go1.10, Unplanned Jul 12, 2017

@aclements

This comment has been minimized.

Copy link
Member

commented Jul 12, 2017

Yikes. Clone and SysV semaphores. Two of my favorite things.

@ncw's patch is clearly safe and arguably even the right thing to do (though since Go never exits Go-created threads I don't think the difference is actually observable even if you tried, but I've got patches lined up that will change that). glibc passes CLONE_SYSVSEM when creating threads and it has the right semantics for threads.

I'll send this as a CL in a bit (unless you want to send it @ncw).

@gopherbot

This comment has been minimized.

Copy link

commented Jul 12, 2017

CL https://golang.org/cl/48170 mentions this issue.

@gopherbot gopherbot closed this in 2673f9e Jul 12, 2017

nghiant2710 added a commit to nghiant2710/golang-arm that referenced this issue Aug 1, 2017

Use QEMU v2.7.0-resin-rc3 when building Go binary.
There is an issue of QEMU v2.9.0 and Go so we can not use that version to build the binary (ref golang/go#20763)

@golang golang locked and limited conversation to collaborators Jul 12, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.