x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

bozaro · 2020-04-24T11:12:53Z

What version of Go are you using (`go version`)?

$ go version
go version go1.13.2 linux/amd64
go version go1.13.3 linux/amd64
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bozaro/.cache/go-build"
GOENV="/home/bozaro/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB="github.com/joomcode/*"
GOOS="linux"
GOPATH="/home/bozaro/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/bozaro/sdk/go-1.13.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/bozaro/sdk/go-1.13.3/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/bozaro/github/debug/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build072895212=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Make some dumps with steps from https://github.com/golang/debug/blob/master/internal/gocore/testdata/README

With go 1.13.2 it's works as expected:

> go build -o test . && GOTRACEBACK=crash ./test || viewcore core --exe test histogram
[1]    19749 abort (core dumped)  GOTRACEBACK=crash ./test
 count size bytes type
     5  384  1920 runtime.g
     1 1152  1152 [70+2?]string
     1  896   896 runtime.m
     1  480   480 [15]internal/cpu.option
     1  112   112 unk112
     1   96    96 [12]*runtime.p
     1   96    96 hchan<bool>
     1   64    64 runtime.timer
     1   32    32 []*runtime.moduledata
     1   32    32 [4]*runtime.g
     1   16    16 [4]uint32
     1   16    16 string

But with go 1.13.3 I got panic:

[1]    23671 abort (core dumped)  GOTRACEBACK=crash ./test
panic: check failed for heap: 67108864 vs 720896

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0006d61e0)
	/home/bozaro/github/debug/internal/gocore/process.go:457 +0x1e3
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0006d60f0)
	/home/bozaro/github/debug/internal/gocore/process.go:460 +0x8f
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc000158000, 0xc000158000, 0x5717c0, 0xc0003b1040, 0xc000791a70, 0x1, 0x1)
	/home/bozaro/github/debug/internal/gocore/process.go:463 +0x1bd0
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc000158000)
	/home/bozaro/github/debug/internal/gocore/process.go:262 +0x77b
golang.org/x/debug/internal/gocore.Core(0xc00020e000, 0x4, 0x0, 0x0)
	/home/bozaro/github/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0xc00020a300, 0x8bfb3d, 0x3, 0x0)
	/home/bozaro/github/debug/cmd/viewcore/main.go:265 +0xd2
main.runHistogram(0xc50120, 0xc0001263c0, 0x0, 0x2)
	/home/bozaro/github/debug/cmd/viewcore/main.go:457 +0xdd
github.com/spf13/cobra.(*Command).execute(0xc50120, 0xc000126360, 0x2, 0x2, 0xc50120, 0xc000126360)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0xc4f7a0, 0x8bf822, 0x1, 0xc000207f00)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/bozaro/github/debug/cmd/viewcore/main.go:243 +0x104

With go 1.14.2 I got another panic:

[1]    23046 abort (core dumped)  GOTRACEBACK=crash ./test
panic: bad uint8 type runtime.mSpanStateBox

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.region.Uint8(0xc0000be9a0, 0x7fb61e18516b, 0xc000433720, 0x8c019d)
	/home/bozaro/github/debug/internal/gocore/region.go:100 +0xc6
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc0000be9a0, 0xc0000be9a0, 0x4e7280, 0xc000706550, 0xc000620300, 0x1, 0x1)
	/home/bozaro/github/debug/internal/gocore/process.go:349 +0x566
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc0000be9a0)
	/home/bozaro/github/debug/internal/gocore/process.go:262 +0x77b
golang.org/x/debug/internal/gocore.Core(0xc0000f2000, 0x4, 0x0, 0x0)
	/home/bozaro/github/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0xc0000ee300, 0x8bfb3d, 0x3, 0x0)
	/home/bozaro/github/debug/cmd/viewcore/main.go:265 +0xd2
main.runHistogram(0xc50120, 0xc00000e3c0, 0x0, 0x2)
	/home/bozaro/github/debug/cmd/viewcore/main.go:457 +0xdd
github.com/spf13/cobra.(*Command).execute(0xc50120, 0xc00000e360, 0x2, 0x2, 0xc50120, 0xc00000e360)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0xc4f7a0, 0x8bf822, 0x1, 0xc0000ebf00)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/bozaro/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/bozaro/github/debug/cmd/viewcore/main.go:243 +0x104

I try update golang.org/x/sys vendor to v0.0.0-20200420163511-1957bb5e6d1f but behavior has not changed.

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2020-04-25T01:21:15Z

CC @randall77

gopherbot · 2020-05-04T21:43:02Z

Change https://golang.org/cl/232158 mentions this issue: cmd/viewcore: account for partially allocated arenas

gopherbot · 2020-05-05T00:22:12Z

Change https://golang.org/cl/232160 mentions this issue: cmd/viewcore: handle typedef'd mspan.state

gopherbot · 2020-05-05T00:22:31Z

Change https://golang.org/cl/232161 mentions this issue: cmd/viewcore: handle 1.14 free spans

Arenas used to be allocated completely (or at least, have a span allocated to represent the unused space). Since 1.13.3, the latest arena only has spans up to its highest allocation point. Fixes viewcore for 1.13.3. This change was CL 189957, backported to 1.13 at CL 198485. Update golang/go#38638 Change-Id: Ia8cb14925cf6f4226a80e7ff4f011aa93e8b41d1

mspan.state used to (go1.13) be a uint8, now (go1.14) it is a struct with a uint8 field. Update golang/go#38638 Change-Id: I042787a61479808d37afa0a7f8caed06b5a18567

Free spans are now implicit. Derive the free span information by finding all the memory in the inuse address ranges that isn't attributed to any span. Fixes viewcore for all versions of 1.14. Also works for tip. Fixes golang/go#38638 Change-Id: Ied283f59391aad8ed5b1b657d2fd6e89e9e9ae6c

ppggff · 2020-05-07T09:24:50Z

With commits in bozaro's repo, test.go works. But my go program(with cgo) still panic:

go version go1.14.1 linux/amd64

panic: check failed for heap: 1598029824 vs 1282605056

goroutine 1 [running]:
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0037ea180)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:515 +0x1e3
golang.org/x/debug/internal/gocore.(*Process).readSpans.func2(0xc0037ea090)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:518 +0x8f
golang.org/x/debug/internal/gocore.(*Process).readSpans(0xc0011cc000, 0xc0011cc000, 0x16f1e20, 0xc0013fa550, 0xc000104000, 0x18, 0x20)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:521 +0x237e
golang.org/x/debug/internal/gocore.(*Process).readHeap(0xc0011cc000)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:262 +0x775
golang.org/x/debug/internal/gocore.Core(0xc0000f0000, 0xa, 0x0, 0x0)
	/home/gpadmin/go/src/golang.org/x/debug/internal/gocore/process.go:156 +0x2c4
main.readCore(0x49d8a9, 0xc0000e9ba8, 0x0, 0xc000089520)
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:265 +0xd2
main.runRoot(0xc55ee0, 0xc000092320, 0x0, 0x2)
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:287 +0x45
github.com/spf13/cobra.(*Command).execute(0xc55ee0, 0xc00009c020, 0x2, 0x2, 0xc55ee0, 0xc00009c020)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x2aa
github.com/spf13/cobra.(*Command).ExecuteC(0xc55ee0, 0x8d3ce3, 0x1, 0xc0000e9f00)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/gpadmin/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	/home/gpadmin/go/src/golang.org/x/debug/cmd/viewcore/main.go:243 +0x111

Arenas used to be allocated completely (or at least, have a span allocated to represent the unused space). Since 1.13.3, the latest arena only has spans up to its highest allocation point. Fixes viewcore for 1.13.3. This change was CL 189957, backported to 1.13 at CL 198485. Update golang/go#38638 Change-Id: Ia8cb14925cf6f4226a80e7ff4f011aa93e8b41d1 Reviewed-on: https://go-review.googlesource.com/c/debug/+/232158 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>

mspan.state used to (go1.13) be a uint8, now (go1.14) it is a struct with a uint8 field. Update golang/go#38638 Change-Id: I042787a61479808d37afa0a7f8caed06b5a18567 Reviewed-on: https://go-review.googlesource.com/c/debug/+/232160 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>

ianlancetaylor · 2020-06-26T00:13:48Z

@randall77 Is there anything else to do here?

randall77 · 2020-06-26T01:35:33Z

Yes, just waiting on a +2. I'll ping the CL.

gopherbot added this to the Unreleased milestone Apr 24, 2020

ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 25, 2020

ianlancetaylor modified the milestones: Unreleased, Go1.15 Apr 25, 2020

bozaro pushed a commit to bozaro/debug that referenced this issue May 6, 2020

cmd/viewcore: handle typedef'd mspan.state

a1fc89a

mspan.state used to (go1.13) be a uint8, now (go1.14) it is a struct with a uint8 field. Update golang/go#38638 Change-Id: I042787a61479808d37afa0a7f8caed06b5a18567

ianlancetaylor modified the milestones: Go1.15, Unreleased Jun 26, 2020

gopherbot closed this as completed in golang/debug@1510a91 Jun 26, 2020

jordanlewis mentioned this issue May 6, 2021

x/debug/cmd/viewcore: viewcore panics: bad int32 type uint32 #45984

Open

golang locked and limited conversation to collaborators Jun 26, 2021

gopherbot added the FrozenDueToAge label Jun 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

bozaro commented Apr 24, 2020

ianlancetaylor commented Apr 25, 2020

gopherbot commented May 4, 2020

gopherbot commented May 5, 2020

gopherbot commented May 5, 2020

ppggff commented May 7, 2020

ianlancetaylor commented Jun 26, 2020

randall77 commented Jun 26, 2020

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

x/debug: viewcore tool don't support dumps with go1.13.3+ #38638

Comments

bozaro commented Apr 24, 2020

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

ianlancetaylor commented Apr 25, 2020

gopherbot commented May 4, 2020

gopherbot commented May 5, 2020

gopherbot commented May 5, 2020

ppggff commented May 7, 2020

ianlancetaylor commented Jun 26, 2020

randall77 commented Jun 26, 2020

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?