Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: threadcreate profile is broken #6104

Open
dvyukov opened this issue Aug 11, 2013 · 14 comments
Open

runtime: threadcreate profile is broken #6104

dvyukov opened this issue Aug 11, 2013 · 14 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Milestone

Comments

@dvyukov
Copy link
Member

dvyukov commented Aug 11, 2013

With the new scheduler threadcreate profile is broken. Most of the threads are created
by sysmon thread, which stack is not very informative.
We need to either remove the profile, or capture stack of the blocking cgo/syscall that
caused new thread creation.
@rsc
Copy link
Contributor

rsc commented Aug 13, 2013

Comment 1:

It would be nice to capture a useful stack but it may be okay to remove it
too. I can't remember ever using it.

@dvyukov
Copy link
Member Author

dvyukov commented Aug 13, 2013

Comment 2:

What is a "useful stack"?
I've attached threadcreate profile from HTTP server FTR.
Lots of threads are created by sysmon (not useful).
Timerproc created 3 threads (not useful).
Creation of random goroutines caused thread creation up to GOMAXPROCS (not useful).
I can imagine an end user trying to figure out what threadcreate profile is trying to
say...
It must be centered around blocking syscalls and locked threads. But which exactly?
Probably all, because they all can trigger thread creation. It all needs to be rethought
and probably even renamed.
Probably now it's better to just remove it.

Attachments:

  1. threadcreate.txt (5407 bytes)

@rsc
Copy link
Contributor

rsc commented Aug 13, 2013

Comment 3:

The idea was that if, for example, your program was creating 1000s of
threads because they kept getting blocked on cgo DNS lookups, you'd see the
cgo DNS call in the profile sample, because it is what blocked and
triggered the creation of a new thread.

@robpike
Copy link
Contributor

robpike commented Aug 30, 2013

Comment 4:

Not for 1.2.

Labels changed: removed go1.2maybe.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 5:

Labels changed: added go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 6:

Labels changed: removed feature.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 7:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 8:

Labels changed: added repo-main.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@pradeepsawlani

This comment was marked as off-topic.

@ianlancetaylor

This comment was marked as off-topic.

@gopherbot
Copy link

gopherbot commented Aug 7, 2019

Change https://golang.org/cl/189318 mentions this issue: runtime/pprof: Plumb labels for goroutine profiles

gopherbot pushed a commit that referenced this issue Apr 22, 2020
Goroutines are directly associated with labels. It's relatively easy to
plumb those through without creating goroutine-locals in the wild.

This is accomplished by splitting out most of the code from the public
`runtime.GoroutineProfile` into a new unexported
`runtime.goroutineProfileWithLabels`, which then has a thin wrapper
linked into the `runtime/pprof` package as
`runtime_goroutineProfileWithLabels`. (mirroring the way labels get
associated with the `g` for a goroutine in the first place)

Per-#6104, OS-thread creation profiles are a bit useless, as `M`s tend
to be created be created by a background goroutine. As such, I decided
not to add support for capturing the labels at `M`-creation-time, since
the stack-traces seem to always come out `nil` for my simple test
binaries.

This change currently provides labels for debug=0 and debug=1, as
debug=2 is currently entirely generated by the runtime package and I
don't see a clean way of getting the `labelMap` type handled properly
within the `runtime` package.

Update the comment added in cl/131275 to mention goroutine support for
labels.

Updates #23458

Change-Id: Ia4b558893d7d10156b77121cd9b70c4ccd9e1889
Reviewed-on: https://go-review.googlesource.com/c/go/+/189318
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
arp242 added a commit to arp242/go that referenced this issue Jul 24, 2022
Retrieving the stack traces hasn't worked since 2013:
golang#6104

However, this is not clear at all from the documentation, which is
confusing.

Once 6104 is fixed the documentation can be updated again, but there
doesn't seem much interest in doing so.
arp242 added a commit to arp242/go that referenced this issue Jul 24, 2022
…ate"

Retrieving the stack traces hasn't worked since 2013:
golang#6104

However, this is not clear at all from the documentation, which is
confusing.

Once 6104 is fixed the documentation can be updated again, but there
doesn't seem much interest in doing so.
SaveTheRbtz added a commit to onflow/flow-go that referenced this issue Jul 29, 2022
It is broken since 2013[1]. Verified on localnet.

[1] golang/go#6104
bors bot added a commit to onflow/flow-go that referenced this issue Jul 29, 2022
2898: [Profile] disable threadcreate profiler r=SaveTheRbtz a=SaveTheRbtz

[It has been broken since 2013](golang/go#6104). Verified on localnet and removed.

Co-authored-by: Alexey Ivanov <rbtz@dapperlabs.com>
@gopherbot
Copy link

gopherbot commented Aug 16, 2022

Change https://go.dev/cl/423116 mentions this issue: runtime: always collect M creation stack

@prattmic
Copy link
Member

prattmic commented Aug 16, 2022

I've sent https://go.dev/cl/423116 as a "fix" for this. tl;dr, the CL just stops hiding stacks from within the scheduler. The result of this profile probably isn't very meaningful to the typical user, but it can be quite handy for someone investigating strange behavior in the runtime.

@prattmic prattmic added the compiler/runtime Issues related to the Go compiler and/or runtime. label Aug 16, 2022
@prattmic
Copy link
Member

prattmic commented Aug 16, 2022

e.g., here is an example profile from the package os tests:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
Status: In Progress
Development

No branches or pull requests

7 participants