Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/pprof: runtime error: slice bounds out of range on "view source" #46636

Open
prattmic opened this issue Jun 7, 2021 · 7 comments
Open

cmd/pprof: runtime error: slice bounds out of range on "view source" #46636

prattmic opened this issue Jun 7, 2021 · 7 comments
Assignees
Milestone

Comments

@prattmic
Copy link
Member

@prattmic prattmic commented Jun 7, 2021

Reported at https://groups.google.com/g/golang-dev/c/MH_e2xnSWAE/m/cCPpbtb8AQAJ

What version of Go are you using (go version)?

$ go version
go version devel go1.17-8212707871 Mon Jun 7 15:19:57 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

No

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/mpratt/.cache/go-build"
GOENV="/usr/local/google/home/mpratt/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/usr/local/google/home/mpratt/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/usr/local/google/home/mpratt/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/mpratt/src/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/mpratt/src/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.17-8212707871 Mon Jun 7 15:19:57 2021 +0000"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3999737285=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/ASWkOjkKSN-

$ go tool pprof -http :8080 ./prof cpu.profile, then browse to http://localhost:8080/ui/source

What did you expect to see?

Source

What did you see instead?

2021/06/07 12:56:09 http: panic serving 10.129.52.196:49259: runtime error: slice bounds out of range [:18446744073705353216] with capacity 663772
goroutine 6 [running]:                                                                                                                       
net/http.(*conn).serve.func1()                                                                                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:1801 +0xb9                                                 
panic({0x8e0580, 0xc0000e1770})                                                                                                              
        /usr/local/google/home/mpratt/src/go/src/runtime/panic.go:1047 +0x266     
cmd/internal/objfile.(*Disasm).Decode(0xc000119920, 0xc00031a530, 0x6, {0x0, 0x1, 0xc000198fd0}, 0xd9, 0x30)
        /usr/local/google/home/mpratt/src/go/src/cmd/internal/objfile/disasm.go:276 +0x42a                                         
main.(*objTool).Disasm(0x1, {0xc00031a530, 0x0}, 0xc00036f320, 0xc000199130, 0xd0)                                    
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:183 +0x9d                                            
cmd/vendor/github.com/google/pprof/driver.(*internalObjTool).Disasm(0xc000312f50, {0xc00031a530, 0x2}, 0x0, 0x1000067049a8c18, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:273 +0x3b                               
cmd/vendor/github.com/google/pprof/internal/report.(*sourcePrinter).expandAddresses(0xc00037c140, 0xc000314520, 0x0, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:352 +0x376              
cmd/vendor/github.com/google/pprof/internal/report.newSourcePrinter(0xc000314520, {0x9f8118, 0xc000119930}, {0xc00002e0f4, 0x27})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:312 +0x5f7       
cmd/vendor/github.com/google/pprof/internal/report.PrintWebList({0x9f1320, 0xc00036eae0}, 0xc000314520, {0x9f8118, 0xc000119930}, 0x2)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:208 +0xcc
cmd/vendor/github.com/google/pprof/internal/driver.(*webInterface).source(0xc000243740, {0x9fa330, 0xc0003222a0}, 0xc0003b2200)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/webui.go:397 +0x178                      
net/http.HandlerFunc.ServeHTTP(0x88fce0, {0x9fa330, 0xc0003222a0}, 0x7)                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                                                               
cmd/vendor/github.com/google/pprof/internal/driver.defaultWebServer.func1({0x9fa330, 0xc0003222a0}, 0xc0003b2200)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/webui.go:190 +0xb7                                                                                                                                                                    
net/http.HandlerFunc.ServeHTTP(0x4d1bd3, {0x9fa330, 0xc0003222a0}, 0x72)                                                                     
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                                                               
net/http.StripPrefix.func1({0x9fa330, 0xc0003222a0}, 0xc0002b8300)                                                                           
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2089 +0x330                                                              
net/http.HandlerFunc.ServeHTTP(0xc000199b40, {0x9fa330, 0xc0003222a0}, 0x68c60f)
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                              
net/http.(*ServeMux).ServeHTTP(0x4681ca, {0x9fa330, 0xc0003222a0}, 0xc0002b8300)          
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2424 +0x149   
net/http.serverHandler.ServeHTTP({0xc000094570}, {0x9fa330, 0xc0003222a0}, 0x6)                                                      
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2865 +0x8e                                                     
net/http.(*conn).serve(0xc0000960a0, {0x9fcf80, 0xc000243d70})                                                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:1929 +0xb08                                                              
created by net/http.(*Server).Serve
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2991 +0x4e8
@prattmic
Copy link
Member Author

@prattmic prattmic commented Jun 7, 2021

Same thing without the webserver complexity:

$ go tool pprof -weblist . ./prof cpu.profile
panic: runtime error: slice bounds out of range [:18446744073705353216] with capacity 759594

goroutine 1 [running]:
cmd/internal/objfile.(*Disasm).Decode(0xc000073930, 0xc00002b7f8, 0x6, {0x0, 0x1, 0xc0001731f8}, 0xd9, 0x30)
        /usr/local/google/home/mpratt/src/go/src/cmd/internal/objfile/disasm.go:276 +0x42a
main.(*objTool).Disasm(0x1, {0xc00002b7f8, 0x0}, 0xc00020c4e0, 0xc000173358, 0xf8)
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:183 +0x9d
cmd/vendor/github.com/google/pprof/driver.(*internalObjTool).Disasm(0xc000073f50, {0xc00002b7f8, 0x2}, 0x0, 0x100004b36650fd8, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:273 +0x3b
cmd/vendor/github.com/google/pprof/internal/report.(*sourcePrinter).expandAddresses(0xc000021040, 0xc00007d220, 0x0, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:352 +0x376
cmd/vendor/github.com/google/pprof/internal/report.newSourcePrinter(0xc00007d220, {0x9f8118, 0xc000073940}, {0xc00002e0f4, 0x27})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:312 +0x5f7
cmd/vendor/github.com/google/pprof/internal/report.PrintWebList({0x9f1320, 0xc0001afad0}, 0xc00007d220, {0x9f8118, 0xc000073940}, 0x8)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:208 +0xcc
cmd/vendor/github.com/google/pprof/internal/report.printWebSource({0x9f1320, 0xc0001afad0}, 0xc0001b1540, {0x9f8118, 0xc000073940})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:125 +0x5b
cmd/vendor/github.com/google/pprof/internal/report.Generate({0x9f1320, 0xc0001afad0}, 0x0, {0x9f8118, 0xc000073940})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/report.go:115 +0x1c5
cmd/vendor/github.com/google/pprof/internal/driver.generateReport(_, {_, _, _}, {{0x0, 0x0}, 0x0, 0x0, {0x9123ec, 0x7}, ...}, ...)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/driver.go:117 +0xd4
cmd/vendor/github.com/google/pprof/internal/driver.PProf(0xc000176980)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/driver.go:53 +0x225
cmd/vendor/github.com/google/pprof/driver.PProf(0x4601db)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:33 +0x1e
main.main()
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:38 +0xc5
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jun 7, 2021

It is interesting that list doesn't panic, whereas weblist does...

@cherrymui cherrymui added this to the Go1.17 milestone Jun 7, 2021
@prattmic
Copy link
Member Author

@prattmic prattmic commented Jun 7, 2021

I don't think list invokes the disassembler, while weblist does.

The immediate problem seems to be that vendored pprof is invoking cmd/pprof.objTool.Disasm with start 0x401000, end 0x0.

Still digging into why that is.

@ALTree
Copy link
Member

@ALTree ALTree commented Jun 7, 2021

A possible culprit is 68327e1. Does reverting it fixes it?

@prattmic
Copy link
Member Author

@prattmic prattmic commented Jun 7, 2021

I believe the problem is that https://go-review.googlesource.com/c/go/+/318049/3/src/cmd/pprof/pprof.go makes ObjAddr unconditionally return 0, but https://go-review.googlesource.com/c/go/+/318049/3/src/cmd/vendor/github.com/google/pprof/internal/report/source.go passes the result of ObjAddr to Disasm as-is. I believe that ObjAddr should return addr to remain a no-op as before.

@prattmic
Copy link
Member Author

@prattmic prattmic commented Jun 7, 2021

Yup, that does it. I'll send a CL.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 7, 2021

Change https://golang.org/cl/325809 mentions this issue: cmd/pprof: make ObjAddr a no-op

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants