Skip to content
This repository

Parallelization of install scripts causing crash on VPS #1920

justmoon opened this Issue December 22, 2011 · 1 comment

2 participants

Stefan Thomas Luke Arduini
Stefan Thomas

I was trying to npm install -g bitcoinjs on a MediaTemple (ve) Server, but ran into random errors like (paraphrasing) "could not allocate virtual memory" and "could not start thread". Most of the time one of the node-waf processes just fails without error message. While the error seems to be somewhat random, the installation failed reliably every time I tried it.

Eventually I tried installing some of the dependencies that contain C++ code first, one-by-one and then installing bitcoinjs last which worked fine.

Here is the log for first trying to install it in one go, then individually:

What I believe is happening is that NPM tries to compile all of the dependencies at once. However gcc is pretty heavy (node-waf may even start multiple threads itself, I'm not sure), so what ends up happening is that memory or simply the allowed number of threads is exhausted.

Note that the installation works fine on machines with a regular kernel that doesn't have those hard resource limits some VPSes have. I still suspect that parallelizing large numbers of node-waf processes or more generally install scripts is a bad idea there as well, slowing down the installation rather than speeding it up.

Luke Arduini
luk- commented June 17, 2013

Closing this since it's so old, if it's still an issue with node-gyp we can reopen.

Luke Arduini luk- closed this June 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.