Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/pprof: background GC goroutine sample have pprof labels #50032

Closed
prattmic opened this issue Dec 7, 2021 · 3 comments
Closed

runtime/pprof: background GC goroutine sample have pprof labels #50032

prattmic opened this issue Dec 7, 2021 · 3 comments
Assignees
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@prattmic
Copy link
Member

prattmic commented Dec 7, 2021

gcBgMarkStartWorkers starts several gcBgMarkWorker goroutines. This is called when starting the first GC. If the GC is triggered by a goroutine with pprof labels, then the GC worker goroutines will inherit the pprof labels.

cc @mknyszek @bcmills

@prattmic prattmic added the NeedsFix The path to resolution is known, but the work has not been done. label Dec 7, 2021
@prattmic prattmic added this to the Backlog milestone Dec 7, 2021
@mknyszek
Copy link
Contributor

mknyszek commented Dec 7, 2021

It's worth noting that this can also happen after the first GC if GOMAXPROCS increases.

@gopherbot
Copy link

Change https://golang.org/cl/369983 mentions this issue: runtime: do not inherit labels on system goroutines

@gopherbot
Copy link

Change https://golang.org/cl/369984 mentions this issue: runtime/pprof: assert that labels never appear on unexpected samples

gopherbot pushed a commit that referenced this issue Jan 19, 2022
This makes TestLabelSystemstack much more strict, enabling it to detect
any misplacement of labels.

Unfortunately, there are several edge cases where we may not have an
obviously correct stack trace, so we generally except the runtime
package, with the exception of background goroutines that we know should
not be labeled.

For #50007
For #50032

Change-Id: I8dce7e7da04f278ce297422227901efe52782ca0
Reviewed-on: https://go-review.googlesource.com/c/go/+/369984
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
@prattmic prattmic self-assigned this Feb 17, 2022
jproberts pushed a commit to jproberts/go that referenced this issue Jun 21, 2022
GC background mark worker goroutines are created when the first GC is
triggered (or next GC after GOMAXPROCS increases). Since the GC can be
triggered from a user goroutine, those workers will inherit any pprof
labels from the user goroutine.

That isn't meaningful, so avoid it by excluding system goroutines from
inheriting labels.

Fixes golang#50032

Change-Id: Ib425ae561a3466007ff5deec86b9c51829ab5507
Reviewed-on: https://go-review.googlesource.com/c/go/+/369983
Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
jproberts pushed a commit to jproberts/go that referenced this issue Jun 21, 2022
This makes TestLabelSystemstack much more strict, enabling it to detect
any misplacement of labels.

Unfortunately, there are several edge cases where we may not have an
obviously correct stack trace, so we generally except the runtime
package, with the exception of background goroutines that we know should
not be labeled.

For golang#50007
For golang#50032

Change-Id: I8dce7e7da04f278ce297422227901efe52782ca0
Reviewed-on: https://go-review.googlesource.com/c/go/+/369984
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
@prattmic prattmic self-assigned this Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

3 participants