gc: rename module and remove (de)committed memory distinction #496
Conversation
In the future this file will hold more OS-specific functions needed by the GC, and this file looks like a good place to put them, so all OS-specific functions are grouped in a single module. Also, having the prefix gc on every file, when all this is inside a gc package, doesn't seem to make a lot of sense, thus it will be removed progressively.
|
I've done some performance check in Linux with the tests in |
The distinction is only useful for Windows, Posix has no standard way to do this, even when apparently you can emulate it in Linux using PROT_NONE while mmap()ping (but Linux don't commit memory until you write on it, so is pointless to use it in Linux). This distinction makes the code a little more complex, because is spread in several parts of the GC, and it doesn't have an evident positive impact (having memory mapped but not committed is a very transient state anyway, except on extremely rare situations, the GC will commit all the memory eventually, as you keep allocating, it could take more or less time, but eventually you'll have all your memory committed just before the first time the collector runs).
|
(fixed unittest) |
|
So we never have many pools with uncommited memory? |
|
No, just one pool, maybe you can have another pool with some uncommitted space if an allocation failed because it can't find a big enough chunk free/uncommitted in the current pools, but normally everything should be committed eventually in the GC, otherwise you have a serious fragmentation problem, which is a problem by itself. |
|
Then let's do this, I will review this in detail when I'm back home. |
|
Remember, before pulling this it would be good if somebody can run at least the benckmarks in |
|
I've run testgc3 from the test-suite on win32 with avgtime: exe with patch: Total time (ms): 77225.8 exe without patch: Total time (ms): 83800.4 Repeating the runs produced Total time (ms): 77955.7 and 80084.3, so measurements vary quite a bit, but a net improvement seems to exist. I could not try the parser tests as they are currently blocked by bug 9044/6461. |
| ncommitted += tocommit; | ||
|
|
||
| return tocommit > n; | ||
| searchStart = npages; |
There was a problem hiding this comment.
I'm not very familiar with this searchStart optimization, so if anybody can verify this bit of code is still correct, it will be appreciated!
|
And nice program this --- avgtime.huge_single.d.2.063.log 2013-05-31 12:37:28.674837751 +0200
+++ avgtime.huge_single.d.git.log 2013-05-31 12:42:33.120600823 +0200
-Avg time : 3.7239
-Std dev. : 0.413316
+Avg time : 4.3469
+Std dev. : 0.522962
--- avgtime.rand_large.d.2.063.log 2013-05-31 12:37:42.631101841 +0200
+++ avgtime.rand_large.d.git.log 2013-05-31 12:42:47.028864209 +0200
-Avg time : 1201.35
-Std dev. : 69.8966
+Avg time : 1195.81
+Std dev. : 56.8985
--- avgtime.rand_small.d.2.063.log 2013-05-31 12:37:55.899352924 +0200
+++ avgtime.rand_small.d.git.log 2013-05-31 12:43:00.193113502 +0200
-Avg time : 1137.72
-Std dev. : 25.0925
+Avg time : 1122.44
+Std dev. : 18.6431
--- avgtime.tree1.d.2.063.log 2013-05-31 12:42:11.508191567 +0200
+++ avgtime.tree1.d.git.log 2013-05-31 12:47:16.073960787 +0200
-Avg time : 23182
-Std dev. : 181.616
+Avg time : 23187.8
+Std dev. : 183.165
--- avgtime.tree2.d.2.063.log 2013-05-31 12:42:16.972295037 +0200
+++ avgtime.tree2.d.git.log 2013-05-31 12:47:21.506063722 +0200
-Avg time : 419.456
-Std dev. : 4.29148
+Avg time : 416.213
+Std dev. : 1.55321The only case where the unpatched version is worse is import std.stdio, core.memory;
void main(string[] args) {
enum mul = 1000;
auto ptr = GC.malloc(mul * 1_048_576, GC.BlkAttr.NO_SCAN);
GC.collect();
}I have no idea why there is any difference, because all that memory should be even committed, but seems to be a pretty unrealistic case. This is how I generated these diffs: |
|
I noticed my desktop had very high memory usage while doing this tests, so I freed some memory and repeated them, also adding --- avgtime.huge_single.d.2.063.log 2013-05-31 13:24:25.585416424 +0200
+++ avgtime.huge_single.d.git.log 2013-05-31 13:18:27.846272607 +0200
-Avg time : 0.958
-Std dev. : 0.0843955
+Avg time : 0.9093
+Std dev. : 0.031493
--- avgtime.rand_large.d.2.063.log 2013-05-31 13:24:39.345690778 +0200
+++ avgtime.rand_large.d.git.log 2013-05-31 13:18:41.366543012 +0200
-Avg time : 1168.21
-Std dev. : 50.7714
+Avg time : 1152.12
+Std dev. : 49.1627
--- avgtime.rand_small.d.2.063.log 2013-05-31 13:24:51.925941565 +0200
+++ avgtime.rand_small.d.git.log 2013-05-31 13:18:53.966794978 +0200
-Avg time : 1063.46
-Std dev. : 21.5651
+Avg time : 1067.1
+Std dev. : 13.2134
--- avgtime.tree1.d.2.063.log 2013-05-31 13:29:35.739592805 +0200
+++ avgtime.tree1.d.git.log 2013-05-31 13:23:37.724461937 +0200
-Avg time : 25720.7
-Std dev. : 129.007
+Avg time : 25715.2
+Std dev. : 149.692
--- avgtime.tree2.d.2.063.log 2013-05-31 13:29:43.647750086 +0200
+++ avgtime.tree2.d.git.log 2013-05-31 13:23:45.608619195 +0200
-Avg time : 613.644
-Std dev. : 7.81523
+Avg time : 611.554
+Std dev. : 4.12177Now the only test where the new version is worse is |
|
Looks OK to me. |
gc: rename module and remove (de)committed memory distinction
|
Indeed, and part of the plan is to clean the mess a little bit. Thanks. |
The distinction is only useful for Windows, Posix has no standard way to
do this, even when apparently you can emulate it in Linux using
PROT_NONE while mmap()ping (but Linux don't commit memory until you
write on it, so is pointless to use it in Linux).
This distinction makes the code a little more complex, because is spread
in several parts of the GC, and it doesn't have an evident positive
impact (having memory mapped but not committed is a very transient state
anyway, except on extremely rare situations, the GC will commit all the
memory eventually, as you keep allocating, it could take more or less
time, but eventually you'll have all your memory committed just before
the first time the collector runs).
About the rename: