runtime: add profiling of stack segment allocation

[rsc]

See also issue #1477.

[rsc]

Comment 1:

Labels changed: added priority-later, removed priority-medium.

[rsc]

Comment 2:

Labels changed: added performance.

[rsc]

Comment 3:

Labels changed: added go1.1maybe.

[robpike]

Comment 4:

Labels changed: removed go1.1maybe.

[rsc]

Comment 5:

[The time for maybe has passed.]

[dvyukov]

Comment 6:

Issue #5544 has been merged into this issue.

[dvyukov]

Comment 7:

FTR, here is the description of the dup:
Problem description on golang-nuts:
https://groups.google.com/d/msg/golang-nuts/mWXsJJMdL0M/wioVT04ye7UJ
In short, pprof misses 7.5% of samples in newstack/oldstack (perf reports them
properly). This happens because the profiler ignores samples on g0 (where
newstack/oldstack are executed). We need to figure out a way to attribute the samples to
the running goroutine.

[dvyukov]

Comment 8:

It seems to be quite tricky to implement w/o an additional slowdown for stack switch.
What do you think about the following workaround?
There are some other cases when we discard SIGPROF, e.g. on external threads. So a user
may see a profile for Go program, while 95% of time is actually spent on external
threads. The user will try to optimize Go code, but it makes no sense.
The idea is to create fake functions like:
static void ExternalThread(void) {}
static void RuntimeScheduler(void) {}
static void StackSwitch(void) {}
Then, in SIGPROF handler if we encounter a case that we can not handle, create a fake
stack with 1 entry pointing to one of the above functions. Then users at least will get
some idea of where the time is spent, and the profile will be more consistent with e.g.
perf.

[ianlancetaylor]

Comment 9:

Suppose we just count the SIGPROF signals that we skipped and report that as part of the
profile?  "N% of signals were unknown--external thread, stack split, etc."

[minux]

Comment 10:

i think it will better if we can classify the missing SIGPROFs as it could
help the user identify potential problems with there Go code (for example,
growing stack too much) so #8 SGTM.

[dvyukov]

Comment 11:

>Suppose we just count the SIGPROF signals that we skipped and report that as part of
the profile?
External threads and scheduler are quite different.
Is there a support for such reporting? It must be supported by pprof --text and
--web/svg.
What I am proposing is roughly the same, but a bit more detailed and probably simple
(because all required support must be there).

[dvyukov]

Comment 12:

> i think it will better if we can classify the missing SIGPROFs as it could
help the user identify potential problems with there Go code
Agree.
Probably we will start with just Runtime/ExternalThreads, But it will be possible to add
new categories if needed.

[dvyukov]

Comment 13:

On a similar note, we currently completely miss GC and syscalls (because they are on
g0). Syscalls are more or less acceptable, but missing whole GC is very unpleasant.

[rsc]

Comment 14:

Labels changed: added go1.3.

[robpike]

Comment 15:

Labels changed: removed go1.3.

[rsc]

Comment 16:

Labels changed: added go1.3maybe.

[rsc]

Comment 17:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 18:

Labels changed: added repo-main.

[rsc]

We profile g0 work now, and also there are no stack segments anymore.