Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Adjust external allocated memory less often #429
While I think the V8 implementation could/should do a better job of handling requests to adjust external memory without over-reacting to each and every request, in the meantime the
This fix was inspired by marudor/libxmljs2#22, which addressed nodejs/node#30995. I would note that the current implementation in this PR under-reports external memory usage, since the reporting lags behind the actual usage by up to ~10MB. I'm definitely open to switching things around a bit so that
Here's another project that benefitted from adjusting external allocated memory less often: mapnik/node-mapnik#136
It would be ideal to ship Meteor 1.9 with an official version of
Apparently calling v8::Isolate::AdjustAmountOfExternalAllocatedMemory frequently results in lots of wasted CPU cycles on garbage collection, per discussion here: meteor/meteor#10527 (comment) This fix was inspired by marudor/libxmljs2#22, which seems to have addressed nodejs/node#30995. Another project that benefitted from adjusting external allocated memory less often: mapnik/node-mapnik#136
Thanks to the allocation of the Coroutine stack (see set_stack_size), each Fiber has a size of approximately 1MB on a 64-bit machine, so using a threshold of 1MB gives no improvement over adjusting the memory every single time a fiber starts or exits. Instead (with this commit), we adjust memory whenever the gap exceeds 10x the size of a Fiber.
Doesn't v8 already have some threshold logic?
It seems like v8 won't actually send a notification to the GC unless more than 32mb has been allocated. Each fiber is a 1mb so that would be a lot of fibers.
Also did you experiment with just removing
It looks like V8 has imposed a threshold since v6.1.58, which was first present in Node v9.0.0, which updated V8 to v6.1.534.36. The threshold was only 1MB until v6.1.94, when it increased to 32MB (its current value)… but that change should also have been included in the Node 9 changes (and thus Node 12.14.0 too). Hmm.
Without doing a lot more digging, I don't have a good answer as to why the V8 thresholding was inadequate, though I would note that this area of the code is fairly complicated, with some significant known bugs like getting the subtraction backwards. Perhaps there are still unknown bugs that cause the memory pressure to be checked too often? It's either that, or multiple Meteor developers are misreading their CPU charts in the same way, which seems unlikely.
In other words, I would be more than happy for
Is there a particular experiment you would like to see to validate this removal, other than running a patched version of
@laverdet Ok, from the Meteor perspective, simply not calling
Would you be willing to release this (much simpler) change as
Besides your argument about V8 not having much opportunity to collect
This includes laverdet/node-fibers#429, fixing the CPU spikes reported and discussed here: #10527 (comment) Using an official fibers release rather than a GitHub URL is preferable because it doesn't require building fibers from source when deploying a Meteor app to production, and also doesn't rely on GitHub being operational, though of course it does rely on other networked services like npm.
I did notice in the Galaxy logs that
However, this was also happening for
@filipenevola If this ever becomes a problem, I think we just need to update our base Ubuntu image here: https://github.com/meteor/galaxy-images/blob/master/galaxy-app/Dockerfile#L1 (let me know if you need my help with that).