Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/pprof: memprofile malformed profile: mismatch: sample has #22036

Closed
sealcomeback opened this issue Sep 26, 2017 · 2 comments
Closed

runtime/pprof: memprofile malformed profile: mismatch: sample has #22036

sealcomeback opened this issue Sep 26, 2017 · 2 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@sealcomeback
Copy link

sealcomeback commented Sep 26, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go1.8.1

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xiaoju/go"
GORACE=""
GOROOT="/home/xiaoju/push-auto/go1.8.1"
GOTOOLDIR="/home/xiaoju/push-auto/go1.8.1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build603754697=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

var memprofile string
flag.StringVar(&memprofile, "memprofile", "memprofile", "write mem profile to this file")
        flag.Parse()
        var memfd *os.File = nil
        var err4 error
        if memprofile != "" {
                memfd, err4 = os.Create(memprofile)
                if err4 != nil {
                        fmt.Printf("create memprofileFile error, %v", err4)
                }
                defer memfd.Close()
        }
        if memfd != nil {
                go memProfileLoop(memfd)
        }
func memProfileLoop(fd *os.File) {
        checkTicker := time.NewTicker(10 * time.Second)
        checkTickerChan := checkTicker.C
        for {
                select {
                case <-checkTickerChan:
                        if fd != nil {
                                //runtime.GC()
                                pprof.WriteHeapProfile(fd)
                                fd.Sync()
                        }
                }
        }
}

What did you see instead?

go tool pprof bin/dispatch memprofile
malformed profile: mismatch: sample has: 4 values vs. 1748 types
@ianlancetaylor ianlancetaylor changed the title memprofile malformed profile: mismatch: sample has runtime/pprof: memprofile malformed profile: mismatch: sample has Sep 26, 2017
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 26, 2017
@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Sep 26, 2017
@ianlancetaylor
Copy link
Contributor

CC @matloob

@rsc
Copy link
Contributor

rsc commented Nov 29, 2017

Fun bug.

It's not valid to write multiple profiles in sequence to a single file.

Unfortunately:

  • profiles are gzipped protocol buffers
  • the concatenation of multiple gzipped files is a valid gzipping of the concatenation of their contents
  • the concatenation of multiple protocol buffer encodings is a valid encoding of the concatenation of their contents

Because of this, the very concatenated profile is not detected as an invalid profile during decoding. But it appears there are 437 profiles concatenated here, so that the profile header (as concatenated) details 1,748 header fields, while each sample record still only has 4 fields.

It's a very opaque error though. I filed google/pprof#273 upstream with a suggested fix. If they decide to fix it then we'll pick it up in the next sync (likely Go 1.11 at this point).

You can fix this in your program by calling fd.Truncate(0) before writing the profile in the loop.

@rsc rsc closed this as completed Nov 29, 2017
@golang golang locked and limited conversation to collaborators Nov 29, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants