runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 #37238

iswade · 2020-02-15T08:26:06Z

What version of Go are you using (`go version`)?

go version go1.13.8 linux/arm64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (`go env`)?

GOARCH="arm64"
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOOS="linux"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build664418125=/tmp/go-build -gno-record-gcc-switches"

What did you do?

run go developped program

What did you expect to see?

program run normally

What did you see instead?

program run normally but the callstask seems fall into infinite loop:

Thread 30 (Thread 0xfffe1dde3f60 (LWP 610668)):
#0  runtime.futex () at /usr1/paxos/go/src/runtime/sys_linux_arm64.s:461
#1  0x0000fffe22fd7ab0 in runtime.futexsleep (addr=0xfffe23d072f8 <runtime.newmHandoff+24>, val=0, ns=-1) at /usr1/paxos/go/src/runtime/os_linux.go:44
#2  0x0000fffe22fb6db0 in runtime.notesleep (n=0xfffe23d072f8 <runtime.newmHandoff+24>) at /usr1/paxos/go/src/runtime/lock_futex.go:151
#3  0x0000fffe22fe00c0 in runtime.templateThread () at /usr1/paxos/go/src/runtime/proc.go:1906
#4  0x0000fffe22fdeb98 in runtime.mstart1 () at /usr1/paxos/go/src/runtime/proc.go:1201
#5  0x0000fffe22fdeab8 in runtime.mstart () at /usr1/paxos/go/src/runtime/proc.go:1167
#6  0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#7  0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#8  0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#9  0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#10 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#11 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#12 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#13 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#14 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
#15 0x0000fffe2350176c in crosscall1 () at gcc_arm64.S:40
.....

Application(C language) calls library1.so(Golang), and library1.so(Golang) call another library2.so(C language) with cgo.

We build the library1.so with -buildmode=c-shared, and it is used as dynamic library.

The problem only exists at linux/arm64.
The problem does not exists at linux/amd64.

The text was updated successfully, but these errors were encountered:

iswade · 2020-02-15T10:00:52Z

at linux/arm64, with the following code, the problem also exists:

package main
  
import "C" 

import (
        "fmt"
        "time"
)

func loop() {
        for i := 0; i < 100000000; i++ {
                fmt.Println("hello")
                time.Sleep(time.Second)
        }
}

func main() {
        go loop()
        time.Sleep(time.Minute * 2)
}

go build main.go

(gdb) bt
#0  runtime.futex () at /usr1/paxos/go/src/runtime/sys_linux_arm64.s:461
#1  0x0000000000426f60 in runtime.futexsleep (addr=0x574bb0 <runtime.sched+272>, val=0, ns=60000000000) at /usr1/paxos/go/src/runtime/os_linux.go:50
#2  0x000000000040a19c in runtime.notetsleep_internal (n=0x574bb0 <runtime.sched+272>, ns=60000000000, ~r2=<optimized out>) at /usr1/paxos/go/src/runtime/lock_futex.go:193
#3  0x000000000040a270 in runtime.notetsleep (n=0x574bb0 <runtime.sched+272>, ns=60000000000, ~r2=<optimized out>) at /usr1/paxos/go/src/runtime/lock_futex.go:216
#4  0x0000000000435680 in runtime.sysmon () at /usr1/paxos/go/src/runtime/proc.go:4316
#5  0x000000000042dab8 in runtime.mstart1 () at /usr1/paxos/go/src/runtime/proc.go:1201
#6  0x000000000042d9d8 in runtime.mstart () at /usr1/paxos/go/src/runtime/proc.go:1167
#7  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#8  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#9  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#10 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#11 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#12 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40

odeke-em · 2020-02-16T02:33:20Z

Thank you @iswade for this report, the reproducing code and welcome to the Go project!

I’ll kindly loop in some CGO, runtime and arm64 experts: @randall77 @cherrymui @zhangfannie @aclements @ianlancetaylor

cherrymui · 2020-02-16T04:54:09Z

crosscall1 is basically the bottom of the stack. Maybe we could do something so gdb has the correct unwind info.

It does not affect program execution.

shawndx · 2020-02-28T07:55:12Z

Hi, is anyone looking into the issue, I'd like to take a look if needed. thanks.

ALTree · 2020-02-28T07:58:00Z

@shawn-xdji I see no assignee and no gerrit change, so I assume nobody is working on this. Feel free to take it.

shawndx · 2020-03-01T11:23:57Z

@shawn-xdji I see no assignee and no gerrit change, so I assume nobody is working on this. Feel free to take it.

Thanks for confirming.

shawndx · 2020-03-27T13:53:07Z

Hi @iswade, I got a local fix, am trying to test more scenarios, is it feasible to create a sample case for the c-shared flow you mentioned? Thanks.

iswade · 2020-03-31T01:33:36Z

@shawn-xdji c-shared flow is a big project and it is not open source, sorry cannot do it. I think the sample code is enough.

at linux/arm64, with the following code, the problem also exists:

package main
  
import "C" 

import (
        "fmt"
        "time"
)

func loop() {
        for i := 0; i < 100000000; i++ {
                fmt.Println("hello")
                time.Sleep(time.Second)
        }
}

func main() {
        go loop()
        time.Sleep(time.Minute * 2)
}

go build main.go

(gdb) bt
#0  runtime.futex () at /usr1/paxos/go/src/runtime/sys_linux_arm64.s:461
#1  0x0000000000426f60 in runtime.futexsleep (addr=0x574bb0 <runtime.sched+272>, val=0, ns=60000000000) at /usr1/paxos/go/src/runtime/os_linux.go:50
#2  0x000000000040a19c in runtime.notetsleep_internal (n=0x574bb0 <runtime.sched+272>, ns=60000000000, ~r2=<optimized out>) at /usr1/paxos/go/src/runtime/lock_futex.go:193
#3  0x000000000040a270 in runtime.notetsleep (n=0x574bb0 <runtime.sched+272>, ns=60000000000, ~r2=<optimized out>) at /usr1/paxos/go/src/runtime/lock_futex.go:216
#4  0x0000000000435680 in runtime.sysmon () at /usr1/paxos/go/src/runtime/proc.go:4316
#5  0x000000000042dab8 in runtime.mstart1 () at /usr1/paxos/go/src/runtime/proc.go:1201
#6  0x000000000042d9d8 in runtime.mstart () at /usr1/paxos/go/src/runtime/proc.go:1167
#7  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#8  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#9  0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#10 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#11 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40
#12 0x0000000000488a24 in crosscall1 () at gcc_arm64.S:40

gopherbot · 2020-04-03T05:11:22Z

Change https://golang.org/cl/226999 mentions this issue: runtime: fix infinite callstack of cgo on arm64

iswade changed the title ~~runtime: go program run normally, the the callstack seems fall into infinite loop~~ runtime: go program run normally, but the callstack seems fall into infinite loop Feb 15, 2020

odeke-em changed the title ~~runtime: go program run normally, but the callstack seems fall into infinite loop~~ runtime: go program run normally, but the callstack seems fall into infinite loop with CGO on GOOS=Linux Feb 16, 2020

odeke-em changed the title ~~runtime: go program run normally, but the callstack seems fall into infinite loop with CGO on GOOS=Linux~~ runtime: callstack seems fall into infinite loop with CGO on GOOS=linux GOARCH=arm64 Feb 16, 2020

odeke-em added arch-arm64 OS-Linux NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 16, 2020

cherrymui added this to the Unplanned milestone Feb 16, 2020

odeke-em changed the title ~~runtime: callstack seems fall into infinite loop with CGO on GOOS=linux GOARCH=arm64~~ runtime: with GDB, callstack seems fall into infinite loop with CGO on GOOS=linux GOARCH=arm64 Feb 16, 2020

odeke-em changed the title ~~runtime: with GDB, callstack seems fall into infinite loop with CGO on GOOS=linux GOARCH=arm64~~ runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 Feb 16, 2020

gopherbot closed this as completed in 8e121b1 Apr 8, 2020

golang locked and limited conversation to collaborators Apr 8, 2021

gopherbot added the FrozenDueToAge label Apr 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 #37238

runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 #37238

iswade commented Feb 15, 2020 •

edited

Loading

iswade commented Feb 15, 2020

odeke-em commented Feb 16, 2020

cherrymui commented Feb 16, 2020

shawndx commented Feb 28, 2020

ALTree commented Feb 28, 2020

shawndx commented Mar 1, 2020

shawndx commented Mar 27, 2020

iswade commented Mar 31, 2020

gopherbot commented Apr 3, 2020

runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 #37238

runtime: with GDB, callstack falls into infinite loop with CGO on GOOS=linux GOARCH=arm64 #37238

Comments

iswade commented Feb 15, 2020 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

iswade commented Feb 15, 2020

odeke-em commented Feb 16, 2020

cherrymui commented Feb 16, 2020

shawndx commented Feb 28, 2020

ALTree commented Feb 28, 2020

shawndx commented Mar 1, 2020

shawndx commented Mar 27, 2020

iswade commented Mar 31, 2020

gopherbot commented Apr 3, 2020

iswade commented Feb 15, 2020 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?