ERROR: spawn EMFILE (80 file limit?)

[dennishall]

Error: spawn EMFILE
at errnoException (child_process.js:481:11)
at ChildProcess.spawn (child_process.js:444:11)
at child_process.js:342:9
at gm._spawn (/Users/myusername/node_modules/gm/lib/command.js:156:16)
at gm._exec (/Users/myusername/node_modules/gm/lib/command.js:137:17)
at gm.identify (/Users/myusername/node_modules/gm/lib/getters.js:74:10)
at gm.size (/Users/myusername/node_modules/gm/lib/getters.js:33:12)

.. attempted to walk a directory, getting image sizes for each image encountered.

Node: 0.6.12
OS: mac osx 10.6.8

I tried directly calling gm(<filename>).size(function(e, v){ ... }); on the last file in the list before the error, and it worked fine.

I created an array literal containing the 81 image filenames logged to the console leading up to the error and iterating over just them. Same result. But as soon as I removed any one of them from the array, everything was ok.

[aheckmann]

are you running these all in parallel? you may be hitting the file handle limit.

try $ sudo ulimit -n 4096

[dennishall]

They are in parallel. Is there a gm(<filename>).sizeSync() function I could use instead?

After a tiny bit of research, I found something like that also (try increasing the filehandle limit). It didn't make any difference.

I was just about to post a followup to my issue report.

I ended up 'chunking' the result set so that gm/javascript/filesystem had time to release the handles.

// assume an array of filenames, `fileList`
var chunkSize = 50; // 100 was too many. Choked every time at 80 or 81.
(function loop(i){

  var filename = fileList[i];
  //console.log('processing... ', filename);

  gm(filename).size(function(err, value){
    // note : value may be undefined
    if(value){
      console.log(value, filename);
    } else {
      console.log('(error)', err);
    }
  });

  i++;
  if(i == fileList.length) return; // we're done.
  if(i%chunksize == 0){
    setTimeout(function(){ loop(i); }, 50);
  } else {
    loop(i);
  }
})(0);

[aheckmann]

graphicsmagick is pretty hungry i guess.

[aheckmann]

i was able to run a few hundred before running out.

[dennishall]

We shouldn't have to chunk .. I still consider this an issue

[aheckmann]

graphicsmagick is hungry. theres nothing gm can do about that

[cyrus-za]

For those of you who stumble upon this issue while searching for reasons that you getting the node EMFILE error, after searching all over the place and not finding a valid reason, I found the problem in my code.

I was calling gm with a stream. Although I was handling the error with readStream.on('error', ..) it was too late to call next() or respond with a status code, as gm has already been called and although the client gets a response, the server never closes the gm instance, so the next time there's a error it will spawn another gm instance and eventually run into the EMFILE error.

That being said, I used readStream.on('readable') and only run my gm code in there.

I know it's a bit off topic, but I am sure people with the issue will stumble upon this page.

[lucaswxp]

I'm still experiencing this issue. How to I close the gm opened file descriptor manually?

Here is my code:

var series = [];
for(var i in files){ var file = files[i];
  (function(file){
    series.push(function(resolve){
        gm(file)
          .resize(null, 200)
          .quality(80)
          .write(file, function (err) {
            if (err) console.log(err);

            resolve();
          });
      });
  })(file);
}

async.series(series);

[barbasa]

@cyrus-za can you give us an example of code on how you did it ? I'm having the same issue but I haven't managed to fix it :/

Thanks!

[JonSchapiro]

I think the point is to simulate the calls to gm "synchronously" where as the normal calls are async and spawn a new process each time.

[JonSchapiro]

@dennishall thank you so much for this. worked like a charm! was going to do everything synchronously with callbacks but this is much better!