Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/metrics: add contention metrics #49881

Open
znkr opened this issue Nov 30, 2021 · 5 comments
Open

runtime/metrics: add contention metrics #49881

znkr opened this issue Nov 30, 2021 · 5 comments

Comments

@znkr
Copy link

@znkr znkr commented Nov 30, 2021

Right now lock contention is only possible to debug with pprof using the block profile. This is very useful once contention has been identified as the issue, but since it has to be turned on manullay it doesn't help in identifying that contention is an issue. Exporting the cumulative wait time via runtime/metrics would allow continouous monitoring of contention and help in debugging Go programs.

@prattmic
Copy link
Member

@prattmic prattmic commented Nov 30, 2021

@prattmic prattmic added this to the Go1.19 milestone Nov 30, 2021
@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 30, 2021

Interesting... Looks like we already measure time spent blocked for starvation purposes anyway. I suppose this wouldn't be too hard to expose. Is cumulative wait time the right metric? It's useful for comparison, but it doesn't really tell you much on its own.

An approximate distribution of latencies might be more useful for this, because you can correlate that with e.g. request latency, but it's slightly more expensive and it requires a bit more post-processing. Its other downside is that it's approximate -- I'm not sure if there's some special use case enabled by having a precise cumulative wait time.

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 30, 2021

To be clear I'm not opposed to cumulative wait time. If that's the convention we can move forward with that. Just want to make sure we considered our options.

@mknyszek mknyszek changed the title runtime/metrics: Contention metrics runtime/metrics: add contention metrics Nov 30, 2021
@znkr
Copy link
Author

@znkr znkr commented Nov 30, 2021

Latency distribution would be even better. Approximate times would be good enough for all use cases I can think of.

@znkr
Copy link
Author

@znkr znkr commented Jan 20, 2022

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants