Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/objdump: panic: runtime error: index out of range [1048574] when disassembling empty infinite loop #36570

Closed
bigwhite opened this issue Jan 15, 2020 · 21 comments
Milestone

Comments

@bigwhite
Copy link

@bigwhite bigwhite commented Jan 15, 2020

What version of Go are you using (go version)?

macos:
$ go version
go version go1.13.6 darwin/amd64

linux:
# go version
go version go1.13.6 linux/amd64

Does this issue reproduce with the latest release?

yes, so far go1.13.6 is the latest release.

What operating system and processor architecture are you using (go env)?

go env Output
macos:
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tonybai/Library/Caches/go-build"
GOENV="/Users/tonybai/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tonybai/Go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/Users/tonybai/.bin/go1.13.6"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/Users/tonybai/.bin/go1.13.6/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/cz/sbj5kg2d3m3c6j650z0qfm800000gn/T/go-build651515497=/tmp/go-build -gno-record-gcc-switches -fno-common"

linux:

go env

GO111MODULE="on"
GOARCH="amd64"
GOBIN="/root/.bin/go1.13.6/bin"
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/root/.bin/go1.13.6"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/root/.bin/go1.13.6/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build943673068=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a go source file:

$
cat go-scheduler-model-case3.go
package main

import (
	"fmt"
	"runtime"
	"time"
)

func add(a, b int) int {
	return a + b
}

func deadloop() {
	for {
		add(3, 5)
	}
}

func main() {
	runtime.GOMAXPROCS(1)
	go deadloop()
	for {
		time.Sleep(time.Second * 1)
		fmt.Println("I got scheduled!")
	}
}

I ran the commands below and got a panic:

on macos:
$go build -o go-scheduler-model-case3 go-scheduler-model-case3.go
$ go tool objdump -S go-scheduler-model-case3 > go-scheduler-model-case3.s
panic: runtime error: index out of range [1048574] with length 27

goroutine 1 [running]:
cmd/internal/objfile.(*FileCache).Line(0xc0004cdd18, 0xc0001561b0, 0x82, 0xfffff, 0x10da0e1, 0xc0004cdae0, 0xc00007263d, 0x1099780, 0x1099781)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:178 +0x600
cmd/internal/objfile.(*Disasm).Print.func1(0x1099781, 0x2, 0xc0001561b0, 0x82, 0xfffff, 0xc0001a9fa0, 0x15)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:232 +0xd8
cmd/internal/objfile.(*Disasm).Decode(0xc00049e000, 0x1099780, 0x1099790, 0x0, 0x0, 0x0, 0xc0004cdde8)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:283 +0x279
cmd/internal/objfile.(*Disasm).Print(0xc00049e000, 0x11f6e00, 0xc000094008, 0x0, 0x1001000, 0xffffffffffffffff, 0x1)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:227 +0x4e1
main.main()
	/usr/local/go/src/cmd/objdump/main.go:90 +0x61d

on linux:
go tool objdump -S ./ go-scheduler-model-case3>  go-scheduler-model-case3.s
panic: runtime error: index out of range [1048574] with length 27

goroutine 1 [running]:
cmd/internal/objfile.(*FileCache).Line(0xc0003d5d18, 0xc000152b60, 0x16, 0xfffff, 0x4d7601, 0xc0003d5ae0, 0xc000069c45, 0x48d150, 0x48d151)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:178 +0x600
cmd/internal/objfile.(*Disasm).Print.func1(0x48d151, 0x2, 0xc000152b60, 0x16, 0xfffff, 0xc0004960a0, 0x15)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:232 +0xd8
cmd/internal/objfile.(*Disasm).Decode(0xc00007ed00, 0x48d150, 0x48d153, 0x0, 0x0, 0x0, 0xc0003d5de8)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:283 +0x279
cmd/internal/objfile.(*Disasm).Print(0xc00007ed00, 0x5f4460, 0xc00000e020, 0x0, 0x401000, 0xffffffffffffffff, 0x1)
	/usr/local/go/src/cmd/internal/objfile/disasm.go:227 +0x4e1
main.main()
	/usr/local/go/src/cmd/objdump/main.go:90 +0x61d

What did you expect to see?

go tool objdump -S go-scheduler-model-case3 > go-scheduler-model-case3.s run ok

What did you see instead?

the panic information listed above.

@bigwhite

This comment has been minimized.

Copy link
Author

@bigwhite bigwhite commented Jan 15, 2020

go tool of Go 1.12.7 on macos runs ok. no panic occurred.

@ALTree ALTree changed the title go tool objdump crash on MacOS and linux cmd/objdump: panic: runtime error: index out of range [1048574] when disassembly empty infinite loop Jan 15, 2020
@ALTree

This comment has been minimized.

Copy link
Member

@ALTree ALTree commented Jan 15, 2020

Thanks for reporting this. It looks like this program:

package main

func main() {
	for {
	}
}

is enough to trigger a crash, both on go1.13 and on tip.

@ALTree ALTree added this to the Go1.15 milestone Jan 15, 2020
@ALTree ALTree changed the title cmd/objdump: panic: runtime error: index out of range [1048574] when disassembly empty infinite loop cmd/objdump: panic: runtime error: index out of range [1048574] when disassembling empty infinite loop Jan 15, 2020
@randall77

This comment has been minimized.

Copy link
Contributor

@randall77 randall77 commented Jan 15, 2020

If I dump the binary without -S, I get this:

TEXT main.main(SB) /Users/khr/gowork/issue36570.go
  issue36570.go:4	0x1051570		90			NOPL			
  issue36570.go:1048575	0x1051571		ebfd			JMP main.main(SB)	

Looks like the line number is bad? I don't know offhand where the disassembler gets its line info from.
This may be a case where we just protect the lookups against bad line numbers - it's definitely a weird edge case.

@dr2chase

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 15, 2020

That bad line number is intentional; it's to prevent an infinite loop stepping in the debugger.
See https://go-review.googlesource.com/c/go/+/168477

Perhaps the debuggers are smarter now. @cherrymui had an idea that we should just write "runtime.InfiniteLoop()" and for any effect-free infinite loops that we detect (like this one), replace the code with a call to that. It will of course deschedule the goroutine, and the name speaks for itself to anyone debugging their code, and we can get rid of these line number shenanigans.

Saves energy, too.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 17, 2020

#36621 seems to report this same problem, but for pprof. Is there a quick fix we can do for 1.14?

@cherrymui

This comment has been minimized.

Copy link
Contributor

@cherrymui cherrymui commented Jan 17, 2020

I guess we could just ignore the bad line numbers.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

Cherry has made an excellent argument for bogus line = 1 (in person).

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

How about making a real function in the runtime called infiniteloop and using its line number?

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

@dr2chase @cherrymui can you recap the argument for those of us unable to be in the room?

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

There are few programs with interesting code on line 1, but most programs have a line 1.

We have to play this game because typing "n" in a debugger single steps instructions until the line number changes. For a "B ." loop, that takes a long time.
So, insert a nop with a bogus line number to break up the monotony.

I tried zero for a bogus line number, one of our tools got the vapors.
I tried a large number for a bogus line number, one of our tools got the vapors.

So, I am trying 1.

Programs that are entirely on line 1 will behave poorly in a debugger, but they did that already.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

Problem with the infinite loop answer is that (1) we're not super excited about large changes right now and (2) I am not exactly sure it will work anyway for the debugger. I just tried "debugging"

package main

func call() {
	println("called!")
}

func main() {
	for { call(); }
}

and "n" was not happy-making.

The better bogus line solves the problem for the naive empty loop, and solves the crash for this bug. However, debugging the program for this bug is still not good -- it hangs (in deadloop), unless I set a breakpoint on line 1 (!).

If the debuggers could be convinced to all set a breakpoint in runtime.InfiniteLoop, that would work, also.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jan 17, 2020

Change https://golang.org/cl/215297 mentions this issue: cmd/compile: change the "bogus line" to be 1

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

we're not super excited about large changes right now

I was not proposing to actually rewrite the loop to a call to runtime.infiniteloop. I was suggesting to use the line number from runtime.infiniteloop.

FWIW, I also think that the change to rewrite an empty infinite loop to a call to runtime.infiniteloop is actually pretty small. Off the top of my head: At the end of the trim pass, look for BlockPlain blocks whose successor is themselves. Change to BlockInfiniteLoop. During codegen, per platform, implement that block as a call; since there are no args or return values, I think we can do that easily/safely. Have runtime.infiniteloop call runtime.gopark. Might have to tweak a few things in ssa check. All in all pretty small and narrowly targeted. But certainly something that can wait for 1.15.

I am not exactly sure it will work anyway for the debugger.

I'm not sure I follow, but that's not surprising--you know this stuff much better than I.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

I had not thought of the line number in a different file hack, I should see how that behaves.
Would work for step, not sure about "next".

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

If it works well enough, it'd also offer us the opportunity as of Go 1.14 to start teaching all the debuggers to put breakpoints on runtime.infiniteloop, instead of having to wait for Go 1.15.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

My first stab at this turned into a mess.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

How do you feel about

%  dlv exec ./bogo
Type 'help' for list of commands.
(dlv) b main.main
Breakpoint 1 set at 0x1056fd0 for main.main() ./bogo.go:8
(dlv) c
> main.main() ./bogo.go:8 (hits goroutine(1):1 total:1) (PC: 0x1056fd0)
Warning: debugging optimized function
     3:     func call() {
     4:          println("called!")
     5:     }
     6:     
     7:     func main() {
=>   8:          for { }
     9:     }
(dlv) n
> main.main() <infiniteloop>:1 (PC: 0x1056fd1)
Warning: debugging optimized function
(dlv) n
> main.main() ./bogo.go:8 (hits goroutine(1):2 total:2) (PC: 0x1056fd0)
...
@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

But for mysterious reasons that didn't work for ssa/debug_test.go, despite the generated line being present, like so:

  infloop.go:10		0x1057005		90			NOPL					
  <infiniteloop>:1	0x1057006		ebfd			JMP 0x1057005				

The debuggers could start looking for <infiniteloop> before it is present, though that might complicate their testing. I think they need a more general solution for this anyway (record PC every 2-to-the-N instructions into stepping, start looking for a repeat after half a second?) because the compiler certainly does not detect all single-line infinite loops, just the (very) easy ones.

@dr2chase

This comment has been minimized.

Copy link
Contributor

@dr2chase dr2chase commented Jan 17, 2020

To be fair, this a flaw in objdump, though we can certainly tweak the compiler to avoid triggering it:

package main

func main() {
//line bogo.go:9999999
	for { }
}

yields

go tool objdump -S ./bogo > bogo.s
panic: runtime error: index out of range [9999998] with length 7
@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

Will you file a new issue to make objdump and pprof more resilient in the face of nonexistent lines? We should fix that. And then we can keep this issue about not creating references to nonexistent lines in the compiler.

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Jan 17, 2020

I was curious whether there were any surprises waiting for us in 1.15 in this arena, so I whipped up CL 215339. It's not ready for prime-time, but it should be good enough to experiment with for debuggers, pprof, objdump, etc. (And if someone who knows the scheduler wants to tell me how I've broken SIGQUIT tracebacks, that'd be awesome.)

With that CL, this code:

package main

func main() {
	for {
	}
}

compiles to:

"".main STEXT size=41 args=0x0 locals=0x8
	0x0000 00000 (x.go:3)	TEXT	"".main(SB), ABIInternal, $8-0
	0x0000 00000 (x.go:3)	MOVQ	(TLS), CX
	0x0009 00009 (x.go:3)	CMPQ	SP, 16(CX)
	0x000d 00013 (x.go:3)	PCDATA	$0, $-2
	0x000d 00013 (x.go:3)	JLS	34
	0x000f 00015 (x.go:3)	PCDATA	$0, $-1
	0x000f 00015 (x.go:3)	SUBQ	$8, SP
	0x0013 00019 (x.go:3)	MOVQ	BP, (SP)
	0x0017 00023 (x.go:3)	LEAQ	(SP), BP
	0x001b 00027 (x.go:3)	PCDATA	$0, $-2
	0x001b 00027 (x.go:3)	PCDATA	$1, $-2
	0x001b 00027 (x.go:3)	FUNCDATA	$0, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x001b 00027 (x.go:3)	FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x001b 00027 (x.go:3)	FUNCDATA	$2, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x001b 00027 (x.go:4)	PCDATA	$0, $0
	0x001b 00027 (x.go:4)	PCDATA	$1, $0
	0x001b 00027 (x.go:4)	CALL	runtime.infiniteloop(SB)
	0x0020 00032 (x.go:4)	JMP	27
	0x0022 00034 (x.go:4)	NOP
	0x0022 00034 (x.go:3)	PCDATA	$1, $-1
	0x0022 00034 (x.go:3)	PCDATA	$0, $-2
	0x0022 00034 (x.go:3)	CALL	runtime.morestack_noctxt(SB)
	0x0027 00039 (x.go:3)	PCDATA	$0, $-1
	0x0027 00039 (x.go:3)	JMP	0

and, when run, behaves appropriately.

@gopherbot gopherbot closed this in c112289 Jan 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.