runtime: cheaper alternative to runtime.ReadMemStats

[mkevac]

Hi.

runtime.ReadMemStats() calls stopTheWorld inside(), so it is very expensive to call it often (see https://github.com/mkevac/debugcharts). Especially in a production system.

I would like to see cheaper but maybe not so accurate alternative if it's possible.

Thanks.

[ianlancetaylor]

CC @dvyukov @aclements

[dvyukov]

What parts of MemStats are you interested in?

[mkevac]

This particular package uses Alloc, PauseNs and NumGC to graph memory usage and GC pauses.
But I am not sure that patch, if one is possible, should be tied to only my use case.

[dup2X]

hack go src code?

[heschi]

I took an execution trace from an application with a 15-30GB heap, and a call to runtime.ReadMemStats() paused it for about 40 ms. We don't want to give up monitoring memory usage and GC performance, but we also don't want to pause to do it. So some improvement in this area would be appreciated.

If it helps, a non-STW version of ReadMemStats that only provided roughly accurate numbers, or left out the more fine-grained information entirely, might still be fine for most purposes.

[aclements]

I haven't thought deeply about this, but I suspect ReadMemStats can be made dramatically more efficient without sacrificing accuracy. It may still have to stop the world very briefly to collect distributed statistics, but that would be much better than the current STW walk it does over the entire heap.

[Dieterbe]

Hello, I'm very interested as well. I use runtime.ReadMemStats in two places:

1. i run some code every 1s to 60s that checks .Sys only and dumps a memory profile once a threshold is breached (https://github.com/Dieterbe/profiletrigger/blob/master/heap/heap.go)
2. my service reports its own internal memory stats every second. currently that is the following (though subject to change in the future)

m.mem.TotalAlloc # nice to plot "allocation rate" (the derived version of this)
m.mem.Alloc # quite useful
m.mem.Sys # very useful
m.mem.NumGC
m.mem.GCCPUFraction # this one hasn't been particularly useful. not sure what to do with it. might stop looking at it
m.mem.HeapObjects # this should correlate somewhat to GC workload, haven't used it much yet
m.mem.PauseNs 

I've seen some cpu profiles where runtime.ReadMemStats shows up as >=10% (cumulatively), but also sometimes it's completely absent from the profile. I haven't figured out yet why or what workloads trigger this function to take more time.

Either way, very interested in having this one optimized 👍

[gopherbot]

CL https://golang.org/cl/34937 mentions this issue.

[gopherbot]

CL https://golang.org/cl/34936 mentions this issue.