This shows up as nonsense callgraphs in pprof, containing lots of runtime frames and the wrong counts for the functions I'm interested in. Compare this one to this one. Each is from an invocation of pprof that looks like:
go tool pprof -alloc_objects ~/tmp/59f9bbfa096fd3c595dc/main /tmp/mem.pprof
I have been unable to simplify the test case further. In particular, merely allocating a slice of bytes of the given size isn't good enough; it seems I have to open and read a file. I've also seen the problem when not setting runtime.MemProfileRate to a non-default value, but it is harder to reproduce.
go version devel +f2662f2 Tue Jun 23 03:18:52 2015 +0000 linux/amd64
The text was updated successfully, but these errors were encountered:
The heap profile reports a snapshot as of the most recent garbage collection. It omits more recent allocations, because otherwise the recent garbage in the profile can drown out the older live data. What's going on in your program is that the size of the read affects whether a garbage collection happens after the multiwriters are created. If so, they appear in the profile; if not, they don't.
If you want to see a snapshot of live data as of the writing of the profile, you need to call runtime.GC before writing out the profile. If you only care about allocation counts and not what is live, then you can also use GOGC=off. When the GC is off, the profile reports all allocations (since there's no "last GC").
So this is working as intended, if possibly mysteriously. I will update the runtime/pprof doc comment to make this clear.
Oh, to clarify: when you say "see a snapshot of live data", do you mean to say only objects that are still live? Or can I call runtime.GC just before writing the file to see historical allocation counts, even for objects that have long since been collected?
@jacobsa Each profile line gives n: m [N: M] for a particular allocation stack, where n: m is currently allocated (live) objects, and N: M is all objects. pprof --inuse_xxx uses n: m and pprof --alloc_xxx uses N: M. What I mean by "a snapshot of live data" is really "a snapshot of right now", because those are updated only at each GC.