Join GitHub today
runtime: CPU profiles should attribute time spent in runtime.morestack to the function call that triggered it #25943
What did you do?
Details can be found in #18138
What did you expect to see?
I expected CPU profiles to attribute time spent in runtime.morestack to the function that triggered stack growth.
What did you see instead?
Time spent in runtime.morestack is measured properly, but is not tied to any particular stack or series of function calls, making it difficult to determine which code path is experiencing excessive goroutine stack growth.
Does this issue reproduce with the latest release (go1.10.3)?
Should we make the existing cpu profile to show the morestack call stack stitched with the goroutine stack that caused stack growth (that means I guess runtime.gentraceback behavior change)?
Or do we want to create a new built-in profile for stack size changes (per @josharian's comment in the issue 18138)?
Just back from a Go NYC meetup, where @richardartoul went through the backstory here, so I took a second look at this bug. I think I see at least the first problem, but I haven't actually tested it. Notes for whoever works on this next:
Profiling (on Linux) is driven by the
I think I'd start by changing that test to do check against a list of functions that switch to the system stack, not just
(All of this is for CPU profiling. @hyangah looked at memory profiling and IIRC concluded that there was a hardcoded test in there somewhere that just cut off the stack trace for runtime functions, but I don't have the link handy.)
This should now be fixed for CPU profiles. Note that due to implementation issues
Memory profiling of runtime functions is a separate issue which I haven't looked at closely. The change above probably helped, but it doesn't matter due to this policy in the protobuf writer:
which accounts all memory allocation by runtime functions to the user code that called them. This is probably to hide stuff like string-to-byte conversions, which don't show up explicitly in user code.
I'm going to close this issue since the original comment requested CPU only. We can reopen if necessary.
I agree, but since the morestack call is never returned to, putting it in the backtrace would require a level of specific hackery that I was uncomfortable with. I tried making newstack return normally in the common case but it proved more difficult than I think it's worth.
I don't think newstack is that much less clear than morestack. If it actually causes someone trouble we can revisit.