Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: scavenger not as effective as in previous releases #35788

Open
mknyszek opened this issue Nov 22, 2019 · 8 comments
Open

runtime: scavenger not as effective as in previous releases #35788

mknyszek opened this issue Nov 22, 2019 · 8 comments
Assignees
Labels
Milestone

Comments

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 22, 2019

Recent page allocator changes (#35112) forced changes to scavenger as well. AFAICT these changes came with two problems.

Firstly, the scavenger maintains an address which it uses to mark which part of the address space has already been searched. Unfortunately the updates of this value are racy, and it's difficult to determine what kind of check makes sense to prevent these races. Today these races mean that the scavenger can miss updates and end up not working for a whole GC cycle (since it only runs down the heap once), or it could end up iterating over address space that a partially concurrent scavenge (e.g. from heap-growth) had already looked at. The affect on application performance is a higher average RSS than in Go 1.13.

Secondly, the scavenger is awoken on each "pacing" update, but today that only means an update to the goal. Because the scavenger is now self-paced, this wake-up is mostly errant, and is generally not a good indicator that there's new work to be done. What this means for application performance is that it might be scavenging memory further down the heap than it should (violating some principles of the original scavenge design) and thus causing undue page faults, resulting in worse CPU performance than in Go 1.13.

I suggest that we fix these for Go 1.14 (fixes are already implemented), but they need not block the beta.

@mknyszek mknyszek added this to the Go1.14 milestone Nov 22, 2019
@mknyszek mknyszek self-assigned this Nov 22, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 22, 2019

Change https://golang.org/cl/207998 mentions this issue: runtime: wake scavenger and update address on sweep done

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 22, 2019

Change https://golang.org/cl/208378 mentions this issue: runtime: remove scavAddr in favor of address ranges

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Dec 5, 2019

@mknyszek What is left to do here for 1.14?

@mknyszek

This comment has been minimized.

Copy link
Contributor Author

@mknyszek mknyszek commented Dec 5, 2019

All the patches for this are out for review and I'm iterating on them.

@mknyszek

This comment has been minimized.

Copy link
Contributor Author

@mknyszek mknyszek commented Dec 5, 2019

Sorry, that sounds like they haven't been reviewed yet. By "iterating on them" I mean that they're being iterated on via review passes.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Dec 11, 2019

Change https://golang.org/cl/210877 mentions this issue: runtime: add scavenge waker goroutine

@aclements

This comment has been minimized.

Copy link
Member

@aclements aclements commented Dec 19, 2019

We've decided this seems a bit too high-risk for 1.14 without enough reward. In practice, these races seem quite rare, and if they do happen, they're likely to only delay scavenging by one GC cycle. Bumping to 1.15. If we get evidence that this is causing real memory pressure problems in 1.14, we have the patches ready.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 10, 2020

Change https://golang.org/cl/218997 mentions this issue: runtime: avoid re-scanning scavenged and untouched memory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.