New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: Additional Allocator Metrics in `runtime.MemStats` #11890

Open
matttproud opened this Issue Jul 27, 2015 · 6 comments

Comments

Projects
None yet
6 participants
@matttproud
Contributor

matttproud commented Jul 27, 2015

Hi, I am wondering if it would be amenable to include several additional core metrics in runtime.MemStats, namely the following measures:

  1. No. of spans pending being released by size class.

    This helps server authors understand the discrepancy between reported
    heap reservation/allocation versus process RSS.

  2. No. of live spans (with active allocations contained therein) by size class.

    Essential corollary for no. 1.

  3. Measure of sum of span occupancy by size class.

    This helps server and runtime authors understand level of span reuse
    and divine potential problems with heap fragmentation viz-a-viz no. 2.

  4. Sum of span allocations by size class (not of inner object allocations).

    This helps server authors understand the aggregate throughput of
    memory flow in realtime, a measure of efficiency that is useful to
    capture when comparing releases and automated canary tests.

  5. Sum of span releases by size class (not of inner object allocations).

    Useful corollary for no. 4.

  6. Summary statistics about age for spans by size classes: min, median, average, max.

    Throughput in no. 4 and no. 5 is useful, but this takes the level of detail to a
    deeper level.

  7. Cumulative sums of individual allocations made for each span size class.

    Useful throughput measure for individual allocations.

These would be inordinately beneficial in gross measurement of fleet memory allocator performance as well as offer server authors deeper telemetric insights into the lifecycles of their allocations. pprof is great for one-off diagnosis but not real-time analytics of the fleet.

I would be happy coming to compromise on these, especially to enhance the language and requirements as well as to possibly volunteer time in the implementation of these representations should we come to agreement.

@ianlancetaylor ianlancetaylor changed the title from Additional Allocator Metrics in `runtime.MemStats` to runtime: Additional Allocator Metrics in `runtime.MemStats` Jul 27, 2015

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jul 27, 2015

@bradfitz

This comment has been minimized.

Member

bradfitz commented Jul 28, 2015

@RLH

This comment has been minimized.

Contributor

RLH commented Jul 28, 2015

Go seem to have exposed the internal implementation concept of a span in memstats but not in any meaningful way such as indicating the size of a span. This is good since it allows future implementations freedom to change the size of a span. Go should preserve this freedom. This proposal seems to imply that one will need to know the size of a span for it to be useful. Is that true?

@matttproud

This comment has been minimized.

Contributor

matttproud commented Jul 28, 2015

Yes, that is true. The core reason why is the size class is a useful hint in indicating the size of the individual objects involved in the measurement. For instance, this is to help differentiate between low volume of large object churn versus high volume of small object churn.

As for hiding the implementation details of the runtime, let me offer a few remarks:

  1. Metrics that abstract/hide what the runtime is really doing under the hood are of specious value,
    because it forces the consumers of said metrics to study the runtime in much greater detail than
    the case of just exposing the raw internals bare. A quick analogy of this from the world of
    Java: MBean and MXBeans (example package) are the
    training wheels of internal runtime metrics; whereas anyone worth his or her salt uses HSPerfData
    example because of the level of actionable detail it provides. At
    the very worst, the Bean metric approach shoehorns implementation-specific details into
    standardized types that either do not make sense or lose needed detail. Most of this pain occurs
    at runtime when one gets a runtime exception or zero values from a bean that exists as a
    placeholder but has no relevance for the runtime in use.

    The question I have is this: what happens the moment there are alternative runtimes available for
    Go? It seems doubtful that each runtime would be able to meaningfully shoehorn its allocator
    metrics into runtime.MemStats unless they all use the same paradigms.

  2. The Go 1.X Stability Grant makes a lot of sense in general, but
    it may not be well to apply it doctrinally to the runtime package since that one is out of all of the
    packages the most likely to undergo churn behind the scenes.

  3. The domain of runtime is ultimately implementation-specific. Treating it as less than that is a
    disservice to the user.

I think there is a middleground; just not sure what it is.

@aclements

This comment has been minimized.

Member

aclements commented Jul 28, 2015

Details aside, there's been some discussion of the philosophy of MemStats over on #10323. I agree that we have to expose implementation details in order for it to be truly useful, but we've already made the mistake of exposing such details in MemStats that are no longer really relevant, but are covered under Go 1 compatibility.

@matttproud

This comment has been minimized.

Contributor

matttproud commented Jul 28, 2015

Yep, that bug had been the impetus for filing this one (just get the need
out there on public record).

Is it possible to, for instance, keep the legacy fields in the struct and
mark them as "do not use" in the field comments or let go fix assist in
auto-migrations where it makes sense? Alternatively, could we create a
proposal to weaken the stability grant around select aspects of the
runtime? Each of these seems tractable, admittedly with differing levels
of satisfaction, smell, and disgust.

@dvyukov

This comment has been minimized.

Member

dvyukov commented Dec 3, 2015

Current HeapSys/HeapAlloc/HeapInuse/HeapIdle/HeapReleased allow to answer 1, 2 and 3 questions (minus by-size-class part, which is more of an implementation detail, and when you think about RSS size classes are more-of-less irrelevant).
I don't understand what exactly you want to do with 4, 5, 6 and 7 numbers. Please elaborate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment