Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: improve mcentral scalability #37487

Open
mknyszek opened this issue Feb 26, 2020 · 7 comments
Open

runtime: improve mcentral scalability #37487

mknyszek opened this issue Feb 26, 2020 · 7 comments
Assignees
Labels
Milestone

Comments

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Feb 26, 2020

After #35112 landed we saw some notable improvements in allocator scalability, but performance still plateaus at around 20-24 cores. The cause is mcentral, which has become the new scalability bottleneck. The reason it's a bottleneck is because each per-size-class mcentral is a pair of linked lists protected by lock. The lock covers all iteration and operations on the mcentral. While a single addition or removal from the linked list isn't a source of great contention, caching a span from an mcentral involves iteration which is a source of significant contention this lock.

Furthermore, the code around mcentral is fairly confusing. The main source of this confusion is span ownership: when should a span be in an mcentral? Currently a span may be owned simultaneously by:

  1. An mcentral.
  2. A concurrent sweeper.
  3. mheap_.sweepSpans.
  4. An mcache.

This makes reasoning about the span lifecycle and ownership tricky. With some refactoring, I think we can achieve scalability and also change the span ownership model to limit the number of simultaneous owners. @aclements suggested before that we could repurpose the gcSweepBuf, a data structure built for fast concurrent access, to replace the linked lists in the mcentral. We can take this idea further and also use these data structures for sweeping, instead of having a separate mheap_.sweepSpans. This means that markrootSpans will need a different mechanism for finding spans with specials, but we can use a bitmap for that similar to the page reclaimer.

By unifying the sweep queue with mcentral we can also make it so that concurrent sweepers take complete ownership of the span, which makes reasoning about (*mspan).sweep much easier as well. Finally, since we don't need to acquire a lock, there's nothing wrong with an mcache taking complete ownership of a span.

There remains one place where multiple span ownership would still exist and that's with the page reclaimer, which will probably never be able to take ownership of a span. But that's OK, since it only ever sweeps spans which will be freed to the heap, so all the other mechanisms can just ignore spans which are picked up by the page reclaimer (identified by their sweepgen value).

@mknyszek mknyszek added this to the Go1.15 milestone Feb 26, 2020
@mknyszek mknyszek self-assigned this Feb 26, 2020
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221179 mentions this issue: runtime: add spanSet data structure

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221178 mentions this issue: runtime: add bitmap-based markrootSpans implementation

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221182 mentions this issue: runtime: add new mcentral implementation

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221181 mentions this issue: runtime: implement the spanSet data structure

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221180 mentions this issue: runtime: manage a pool of spanSetBlocks and free them eagerly

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221183 mentions this issue: runtime: clean up old markrootSpans

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 26, 2020

Change https://golang.org/cl/221184 mentions this issue: runtime: clean up old mcentral code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.