New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: label runtime functions based on their roles #21190

hyangah opened this Issue Jul 27, 2017 · 3 comments


None yet
4 participants

hyangah commented Jul 27, 2017

As runtime provides more features, we often observe a program's profile is dominated by runtime functions. For example, this is a cpu profile of a program from article (but compiled with go1.9). I highlighted the runtime function frames with purple color. Even for more complex program, we expect at least a quarter of the profiled CPU time to be associated with runtime.


This is useful to analyze performance issues related to runtime. However, often times, it distracts users from focusing on the application logic. On the other hand, Go runtime provides various functionalities (scheduling, GC, map, synchronization, etc), it's still difficult to answer questions like e.g.,

  • what's the CPU time spent by GC?
  • what's the CPU time spent by map operation?

without knowledge on runtime implementation.

Go 1.9 introduced profiler labels that allows users to label profile samples.
Users can do aggregation or filtering of samples based on the labels (e.g. rpc methods, users, operation mode, etc).

Similarly, I propose to add labels for runtime functions based on the roles of functions to help aggregating and analyzing the cost involving runtime more conveniently. Labels are grouping of runtime functions curated by runtime developers. Implementation-wise, it doesn't necessarily have to use the similar mechanism like the profiler label. Instead, when pprof proto is constructed, it can check the existence of the runtime functions and label based on them.

note 1: Alternative approach discussed was to name the runtime functions based on feature it offers, (e.g. all GC related functions should be prefixed with GC or gc, etc) and users use them to filter or hide using pprof options. That is not as nice as labels, and requires more efforts from users to use it right. Another approach mentioned was to filter out uninteresting frames when publishing or visualizing profiles. That is not ideal because sometimes we want the details and also the list of 'uninteresting' functions may change over time.

note 2: We can envision that pprof recognizes runtime labels and presents the aggregated boxes for each group instead of the detailed runtime call graphs unless asked to display the detailed runtime call graph.

@rakyll @randall77 @aclements


This comment has been minimized.


AlekSi commented Aug 9, 2017

That's awesome. Hopefully, someone from Go team will chime in soon.


This comment has been minimized.


bradfitz commented Aug 9, 2017

@AlekSi, @hyangah is on the Go team.


This comment has been minimized.


AlekSi commented Aug 9, 2017

… Github should add 🤦‍♂️ reaction so I could add it without writing another comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment