Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
runtime/pprof: support efficient accumulation of custom event count profiles #18454
I want to be able to gather pprof-esque information about instantaneous events that have occurred over the lifetime of my program, possibly with sampling for performance reasons.
This is similar to what the heap profile does, at least when used with alloc_space and alloc_objects--it tracks memory allocations over a long period.
The existing runtime/pprof custom profile API seems ill-suited to this. (Insofar as I understand it. See #18453.) One could accomplish it by inventing a unique key for Add and never Remove anything. However, this will result in a giant, ever-growing map. It would be far more efficient to just keep a counter per pc, as many of the runtime-provided profiles do. It might be worth considering adding a different kind of custom profile geared more towards this use case.
I don't have a concrete proposal, since I haven't thought about this deeply. This issue is just intended to open discussion, particularly since pprof labels are coming for 1.9, and it might be worth considering how they interact with custom profiles--hopefully productively.
@josharian Something like
The comment text is bad but you get the idea. This would capture the kinds of things that we do for the mutex profile as well as basic counters.
I'd like to make sure we get the other pprof changes through first, but this seems like a reasonable followup.
@josharian Sorry, looks like I dropped a bunch of Github notifications about two weeks ago.
You have more context here than I do, but I don't think a new data structure is required for the API I sketched. There's nothing about "top N" in the usual profiles; it's supposed to be a representative sample of the overall behavior, not just the "weighty" behavior.
I think all that is needed is a
changed the title
proposal: runtime/pprof: support efficient accumulation custom profiles
Feb 13, 2017
@josharian Perhaps the code in x/time/rate might be useful for this: