Concurrent sweep #1681

schani · 2015-04-02T23:49:21Z

The purpose of this is to reduce pause times of major collections by making sweep completely concurrent.

The first phase of sweeping is iterating through the block list, freeing blocks without live objects, and designating the others for lazy sweeping. This phase happened while the world was stopped. This makes it concurrent. The changes include the introduction of a very simple thread pool abstraction (currently supporting only a single thread) that unifies concurrent marking, jobs when scanning roots, and concurrent sweeping.

These are benchmarking results on Linux/AMD64:

"default-sgen" is master. "sgen-concurrent-sweep" is this branch with concurrent sweep enabled, "sgen-no-concurrent-sweep" is this branch with concurrent sweep disabled.

I don't know why binarytree is slower with this branch, concurrent sweep disabled. I ran it on my OSX machine and got these results:

master

       25.07 real        23.57 user         1.39 sys
       25.01 real        23.55 user         1.40 sys
       25.17 real        23.72 user         1.39 sys
       25.24 real        23.77 user         1.39 sys
       25.23 real        23.68 user         1.43 sys

       avg: 25.144

no-concurrent-sweep

       24.63 real        22.99 user         1.49 sys
       24.46 real        22.91 user         1.49 sys
       24.43 real        22.94 user         1.43 sys
       24.60 real        23.02 user         1.50 sys
       24.34 real        22.83 user         1.44 sys

       avg: 24.492

concurrent-sweep

       24.16 real        23.25 user         1.46 sys
       24.26 real        23.28 user         1.51 sys
       24.13 real        23.19 user         1.49 sys
       24.06 real        23.09 user         1.53 sys
       24.16 real        23.28 user         1.44 sys

       avg: 24.154

Here are some pause time graphs. This branch on the left (with concurrent sweep enabled), master on the right.

graph4:

health:

binarytree:

Both the concurrent sweep thread as well as the nursery collector will need access to the block array. Until we've made that lock-free, we're simply using a lock.

The nursery collector requires that sweeping has finished, and instead of waiting it will cooperate with the sweep thread to finish more quickly. The sweep thread will traverse the block array from high indexes to low ones while the nursery collector will go from low to high. They will contend only very briefly when they meet somewhere in the middle.

… held.

…read.

New blocks need to be swept if they're allocated during a non-concurrent major collection or while a concurrent major collection is running.

It's updated from nursery collections and from the sweep thread concurrently.

Don't use the difference to the last collection, but just calculate the maximum heap size and trigger a collection when it's reached.

And since we always wait for the sweep now we can do iterations over the blocks without taking the lock.

We cannot start threads during stop-the-world pauses (some thread APIs, including pthreads, can deadlock), and we want to do marking and sweeping with the same threads. This new thread pool facility will allow that.

So we only have one worker thread that's used both for marking and, later, for sweeping, too.

akoeplinger · 2015-04-03T20:43:56Z

@schani please note that there's a crash during the System.dll testsuite on jenkins: Can't iterate blocks while the world is running or sweep is in progress.

schani · 2015-04-03T21:15:36Z

@akoeplinger Fixed.

evincarofautumn · 2015-04-03T23:44:49Z

I didn’t observe an appreciable difference in performance between master and this branch on binarytree, on Linux nor OS X. Here is a cursory run of perf on Linux anyway:

master:

 Performance counter stats for './master/bin/mono-sgen /home/jon/benchmarker/tests/shootout/binarytree.exe 19':

      29506.661239 task-clock (msec)         #    0.999 CPUs utilized
             8,450 context-switches          #    0.286 K/sec
                11 cpu-migrations            #    0.000 K/sec
           459,294 page-faults               #    0.016 M/sec
   115,067,872,588 cycles                    #    3.900 GHz                     [100.00%]
   213,770,451,534 instructions              #    1.86  insns per cycle         [100.00%]
    27,582,284,387 branches                  #  934.782 M/sec                   [100.00%]
       116,143,327 branch-misses             #    0.42% of all branches         [100.00%]

      29.523146559 seconds time elapsed

concurrent-sweep:

 Performance counter stats for './concurrent-sweep/bin/mono-sgen /home/jon/benchmarker/tests/shootout/binarytree.exe 19':

      30812.231183 task-clock (msec)         #    1.039 CPUs utilized
             9,469 context-switches          #    0.307 K/sec
                26 cpu-migrations            #    0.001 K/sec
           282,797 page-faults               #    0.009 M/sec
   119,664,947,463 cycles                    #    3.884 GHz                     [100.00%]
   214,789,375,480 instructions              #    1.79  insns per cycle         [100.00%]
    28,030,875,077 branches                  #  909.732 M/sec                   [100.00%]
       232,322,906 branch-misses             #    0.83% of all branches         [100.00%]

      29.665200704 seconds time elapsed

concurrent-sweep has more context-switches and cpu-migrations (naturally), as well as more branch misses. I was somewhat surprised to see it also had fewer page-faults.

akoeplinger · 2015-04-03T23:47:59Z

@schani looks like it still fails with the same assert during System.GC.GetTotalMemory (at least on i386, the amd64 build failed due to an already started xvfb...).

kumpera · 2015-04-04T21:38:18Z

@evincarofautumn a more useful metric for this would be total pause time and not wallclock time.

schani · 2015-04-07T20:37:13Z

@akoeplinger It seems that wasn't the latest commit. It works now.

@kumpera On pause times see my charts above.

akoeplinger · 2015-04-07T21:04:57Z

@schani I pulled and build this PR to make sure it's not a Jenkins error and I see crashes locally during the System testsuite too (Ubuntu 14.04/amd64).

schani · 2015-04-07T23:21:26Z

@akoeplinger Could you post a log of that? It works for me on Ubuntu 14.10/amd64.

akoeplinger · 2015-04-07T23:58:23Z

@schani did a few test runs: http://pastebin.com/Dtyh790k

schani · 2015-04-08T00:30:38Z

@akoeplinger I can't reproduce this, neither on Linux nor on OSX. Can you confirm that this does not occur for you with master? Are you using any specific options? Could you post a full log?

akoeplinger · 2015-04-08T00:51:34Z

@schani you're right, I just saw it happening on master on Jenkins as well and on my copy. Looks like I somehow didn't properly clean up my working dir when I tested on master, sorry!

schani added 30 commits April 2, 2015 16:41

[sgen] Split sweeping into three parts: start, block loop, finish.

1fbaf72

[sgen] Function for querying whether world is stopped.

207f2a7

[sgen] Guard allocated block array with a mutex.

788b784

Both the concurrent sweep thread as well as the nursery collector will need access to the block array. Until we've made that lock-free, we're simply using a lock.

[sgen] Make slots free list lock-free.

0caf65b

[sgen] Sweep state instead of have_swept.

e6699ba

[sgen] Make memory governor ready for concurrent sweep.

bfdfbe1

[sgen] On forced major collections, free blocks.

8e8efe3

[sgen] Replace swept with more detailed block state.

7412b44

[sgen] Concurrent sweep

fdfaade

[sgen] Card clearing and moving to shadow cards don't require sweep.

3b1059b

[sgen] Binary protocol entries for block alloc/free/change state.

60078ae

[sgen] Factor out function that checks individual blocks for sweeping.

9cabf9a

[sgen] Set the block state to CHECKING with the allocated blocks lock…

b5682e3

… held.

[sgen] Fix M&S heap consistency checks.

079804f

[sgen] Make function for block checking wait for another potential th…

24262e0

…read.

[sgen] Comments.

440ca36

[sgen] Fix initial block state.

7627baa

New blocks need to be swept if they're allocated during a non-concurrent major collection or while a concurrent major collection is running.

[sgen] Allow card table scanning to do block checking work.

fcdb3a9

[sgen] Remove dead parameter to sweep_block().

0a4d98c

[sgen] Don't iterate over block array while it's being compacted.

37068c1

[sgen] Write binary protocol for card scan before scanning the card.

98d93ad

[sgen] num_major_sections must be updated atomically.

216713e

It's updated from nursery collections and from the sweep thread concurrently.

[sgen] Count major sections before/after sweep correctly.

f38b8ae

[sgen] Simplify allowance logic.

b36a81c

Don't use the difference to the last collection, but just calculate the maximum heap size and trigger a collection when it's reached.

[sgen] Whenever we wait for sweep, help the sweeping thread.

dc9f7f0

And since we always wait for the sweep now we can do iterations over the blocks without taking the lock.

[sgen] Assert world is stopped and not sweeping when iterating blocks.

b5b493d

[sgen] Remove dead macros.

c24e166

[sgen] Just one iteration over the blocks at start of major collection.

578e66c

[sgen] Clean up FIXMEs.

68ba43a

schani added 17 commits April 2, 2015 16:41

[sgen] Rename a function to better reflect what it does.

75cfa06

[sgen] Don't repeat code and don't load multiple times.

c4b9b5c

[sgen] Comments.

1639bef

[sgen] Only do a debug block iteration on a higher assert level.

ec2572d

[sgen] Don't unnecessarily compute object size index.

55a2ff8

[sgen] Fix a comment.

8333c2f

[sgen] Implement a simple thread pool and do concurrent sweep with it.

6fb229a

We cannot start threads during stop-the-world pauses (some thread APIs, including pthreads, can deadlock), and we want to do marking and sweeping with the same threads. This new thread pool facility will allow that.

[sgen] Wait for the sweep job properly.

183dc29

[sgen] Workers use thread pool.

83dfb12

So we only have one worker thread that's used both for marking and, later, for sweeping, too.

[sgen] Have the worker state just be an integer.

01930f5

[sgen] Remove some dead code and data.

9bcdbf2

[sgen] Reinstate old M&S section reserve.

c1385d2

[sgen] Don't CAS when setting block state.

74065cb

[sgen] Mark a function inline.

d62f8e2

[sgen] Increase assert levels.

8278dd3

[sgen] Remove a debug printf.

a4e8ad3

[sgen] Format block iteration macros.

ae4c1ef

[sgen] Don't assert in GC.GetTotalMemory.

1a43231

schani merged commit 1a43231 into mono:master Apr 9, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Concurrent sweep #1681

Concurrent sweep #1681

schani commented Apr 2, 2015

akoeplinger commented Apr 3, 2015

schani commented Apr 3, 2015

evincarofautumn commented Apr 3, 2015

akoeplinger commented Apr 3, 2015

kumpera commented Apr 4, 2015

schani commented Apr 7, 2015

akoeplinger commented Apr 7, 2015

schani commented Apr 7, 2015

akoeplinger commented Apr 7, 2015

schani commented Apr 8, 2015

akoeplinger commented Apr 8, 2015

Concurrent sweep #1681

Concurrent sweep #1681

Conversation

schani commented Apr 2, 2015

akoeplinger commented Apr 3, 2015

schani commented Apr 3, 2015

evincarofautumn commented Apr 3, 2015

akoeplinger commented Apr 3, 2015

kumpera commented Apr 4, 2015

schani commented Apr 7, 2015

akoeplinger commented Apr 7, 2015

schani commented Apr 7, 2015

akoeplinger commented Apr 7, 2015

schani commented Apr 8, 2015

akoeplinger commented Apr 8, 2015