Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

Closed
bozaro opened this issue Apr 24, 2020 · 7 comments
Closed

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

bozaro opened this issue Apr 24, 2020 · 7 comments

Comments

@bozaro
Copy link

@bozaro bozaro commented Apr 24, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.2 linux/amd64
go version go1.13.3 linux/amd64
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bozaro/.cache/go-build"
GOENV="/home/bozaro/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB="github.com/joomcode/*"
GOOS="linux"
GOPATH="/home/bozaro/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/bozaro/sdk/go-1.13.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/bozaro/sdk/go-1.13.3/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/bozaro/github/debug/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build072895212=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Make some dumps with steps from https://github.com/golang/debug/blob/master/internal/gocore/testdata/README

With go 1.13.2 it's works as expected:

> go build -o test . && GOTRACEBACK=crash ./test || viewcore core --exe test histogram
[1]    19749 abort (core dumped)  GOTRACEBACK=crash ./test
 count size bytes type
     5  384  1920 runtime.g
     1 1152  1152 [70+2?]string
     1  896   896 runtime.m
     1  480   480 [15]internal/cpu.option
     1  112   112 unk112
     1   96    96 [12]*runtime.p
     1   96    96 hchan<bool>
     1   64    64 runtime.timer
     1   32    32 []*runtime.moduledata
     1   32    32 [4]*runtime.g
     1   16    16 [4]uint32
     1   16    16 string

But with go 1.13.3 I got panic:

[1]    23671 abort (core dumped)  GOTRACEBACK=crash ./test
panic: check failed for heap: 67108864 vs 720896

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0006d61e0)
	/home/bozaro/github/debug/internal/gocore/process.go:457 +0x1e3
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0006d60f0)
	/home/bozaro/github/debug/internal/gocore/process.go:460 +0x8f
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc000158000, 0xc000158000, 0x5717c0, 0xc0003b1040, 0xc000791a70, 0x1, 0x1)
	/home/bozaro/github/debug/internal/gocore/process.go:463 +0x1bd0
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc000158000)
	/home/bozaro/github/debug/internal/gocore/process.go:262 +0x77b
golang.org/x/debug/internal/gocore.Core(0xc00020e000, 0x4, 0x0, 0x0)
	/home/bozaro/github/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0xc00020a300, 0x8bfb3d, 0x3, 0x0)
	/home/bozaro/github/debug/cmd/viewcore/main.go:265 +0xd2
main.runHistogram(0xc50120, 0xc0001263c0, 0x0, 0x2)
	/home/bozaro/github/debug/cmd/viewcore/main.go:457 +0xdd
github.com/spf13/cobra.(*Command).execute(0xc50120, 0xc000126360, 0x2, 0x2, 0xc50120, 0xc000126360)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0xc4f7a0, 0x8bf822, 0x1, 0xc000207f00)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/bozaro/github/debug/cmd/viewcore/main.go:243 +0x104

With go 1.14.2 I got another panic:

[1]    23046 abort (core dumped)  GOTRACEBACK=crash ./test
panic: bad uint8 type runtime.mSpanStateBox

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.region.Uint8(0xc0000be9a0, 0x7fb61e18516b, 0xc000433720, 0x8c019d)
	/home/bozaro/github/debug/internal/gocore/region.go:100 +0xc6
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc0000be9a0, 0xc0000be9a0, 0x4e7280, 0xc000706550, 0xc000620300, 0x1, 0x1)
	/home/bozaro/github/debug/internal/gocore/process.go:349 +0x566
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc0000be9a0)
	/home/bozaro/github/debug/internal/gocore/process.go:262 +0x77b
golang.org/x/debug/internal/gocore.Core(0xc0000f2000, 0x4, 0x0, 0x0)
	/home/bozaro/github/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0xc0000ee300, 0x8bfb3d, 0x3, 0x0)
	/home/bozaro/github/debug/cmd/viewcore/main.go:265 +0xd2
main.runHistogram(0xc50120, 0xc00000e3c0, 0x0, 0x2)
	/home/bozaro/github/debug/cmd/viewcore/main.go:457 +0xdd
github.com/spf13/cobra.(*Command).execute(0xc50120, 0xc00000e360, 0x2, 0x2, 0xc50120, 0xc00000e360)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0xc4f7a0, 0x8bf822, 0x1, 0xc0000ebf00)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/bozaro/github/debug/cmd/viewcore/main.go:243 +0x104

I try update golang.org/x/sys vendor to v0.0.0-20200420163511-1957bb5e6d1f but behavior has not changed.

@gopherbot gopherbot added this to the Unreleased milestone Apr 24, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 25, 2020

@gopherbot
Copy link

@gopherbot gopherbot commented May 4, 2020

Change https://golang.org/cl/232158 mentions this issue: cmd/viewcore: account for partially allocated arenas

@gopherbot
Copy link

@gopherbot gopherbot commented May 5, 2020

Change https://golang.org/cl/232160 mentions this issue: cmd/viewcore: handle typedef'd mspan.state

@gopherbot
Copy link

@gopherbot gopherbot commented May 5, 2020

Change https://golang.org/cl/232161 mentions this issue: cmd/viewcore: handle 1.14 free spans

bozaro added a commit to bozaro/debug that referenced this issue May 6, 2020
Arenas used to be allocated completely (or at least, have a span
allocated to represent the unused space). Since 1.13.3, the latest
arena only has spans up to its highest allocation point.

Fixes viewcore for 1.13.3.

This change was CL 189957, backported to 1.13 at CL 198485.

Update golang/go#38638

Change-Id: Ia8cb14925cf6f4226a80e7ff4f011aa93e8b41d1
bozaro pushed a commit to bozaro/debug that referenced this issue May 6, 2020
Arenas used to be allocated completely (or at least, have a span
allocated to represent the unused space). Since 1.13.3, the latest
arena only has spans up to its highest allocation point.

Fixes viewcore for 1.13.3.

This change was CL 189957, backported to 1.13 at CL 198485.

Update golang/go#38638

Change-Id: Ia8cb14925cf6f4226a80e7ff4f011aa93e8b41d1
bozaro pushed a commit to bozaro/debug that referenced this issue May 6, 2020
mspan.state used to (go1.13) be a uint8, now (go1.14) it is a struct
with a uint8 field.

Update golang/go#38638

Change-Id: I042787a61479808d37afa0a7f8caed06b5a18567
bozaro pushed a commit to bozaro/debug that referenced this issue May 6, 2020
Free spans are now implicit. Derive the free span information
by finding all the memory in the inuse address ranges that isn't
attributed to any span.

Fixes viewcore for all versions of 1.14.
Also works for tip.

Fixes golang/go#38638

Change-Id: Ied283f59391aad8ed5b1b657d2fd6e89e9e9ae6c
@ppggff
Copy link

@ppggff ppggff commented May 7, 2020

With commits in bozaro's repo, test.go works. But my go program(with cgo) still panic:

go version go1.14.1 linux/amd64

panic: check failed for heap: 1598029824 vs 1282605056

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0037ea180)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:515 +0x1e3
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0037ea090)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:518 +0x8f
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc0011cc000, 0xc0011cc000, 0x16f1e20, 0xc0013fa550, 0xc000104000, 0x18, 0x20)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:521 +0x237e
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc0011cc000)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:262 +0x775
golang.org/x/debug/internal/gocore.Core(0xc0000f0000, 0xa, 0x0, 0x0)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0x49d8a9, 0xc0000e9ba8, 0x0, 0xc000089520)
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:265 +0xd2
main.runRoot(0xc55ee0, 0xc000092320, 0x0, 0x2)
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:287 +0x45
github.com/spf13/cobra.(*Command).execute(0xc55ee0, 0xc00009c020, 0x2, 0x2, 0xc55ee0, 0xc00009c020)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x2aa
github.com/spf13/cobra.(*Command).ExecuteC(0xc55ee0, 0x8d3ce3, 0x1, 0xc0000e9f00)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:243 +0x111
gopherbot pushed a commit to golang/debug that referenced this issue May 7, 2020
Arenas used to be allocated completely (or at least, have a span
allocated to represent the unused space). Since 1.13.3, the latest
arena only has spans up to its highest allocation point.

Fixes viewcore for 1.13.3.

This change was CL 189957, backported to 1.13 at CL 198485.

Update golang/go#38638

Change-Id: Ia8cb14925cf6f4226a80e7ff4f011aa93e8b41d1
Reviewed-on: https://go-review.googlesource.com/c/debug/+/232158
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
gopherbot pushed a commit to golang/debug that referenced this issue May 7, 2020
mspan.state used to (go1.13) be a uint8, now (go1.14) it is a struct
with a uint8 field.

Update golang/go#38638

Change-Id: I042787a61479808d37afa0a7f8caed06b5a18567
Reviewed-on: https://go-review.googlesource.com/c/debug/+/232160
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 26, 2020

@randall77 Is there anything else to do here?

@ianlancetaylor ianlancetaylor modified the milestones: Go1.15, Unreleased Jun 26, 2020
@randall77
Copy link
Contributor

@randall77 randall77 commented Jun 26, 2020

Yes, just waiting on a +2. I'll ping the CL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.