Skip to content
This repository

EMFILE installing a package #3259

Closed
juliangruber opened this Issue March 17, 2013 · 55 comments
Julian Gruber
$ node -v
v0.10.0
$ npm -v
1.2.14
$ sudo npm install -g component
npm http GET https://registry.npmjs.org/component
npm http 200 https://registry.npmjs.org/component
npm http GET https://registry.npmjs.org/component/-/component-0.13.0.tgz
npm http 200 https://registry.npmjs.org/component/-/component-0.13.0.tgz
npm http GET https://registry.npmjs.org/superagent/0.10.0
npm http GET https://registry.npmjs.org/component-builder/0.6.3
npm http GET https://registry.npmjs.org/string-to-js/0.0.1
npm http GET https://registry.npmjs.org/jog/0.4.0
npm http GET https://registry.npmjs.org/batch/0.2.1
npm http GET https://registry.npmjs.org/win-fork/1.0.0
npm http GET https://registry.npmjs.org/archy/0.0.2
npm http GET https://registry.npmjs.org/debug
npm http GET https://registry.npmjs.org/commander/1.0.5
npm http GET https://registry.npmjs.org/mkdirp/0.3.4
npm http 200 https://registry.npmjs.org/string-to-js/0.0.1
npm http GET https://registry.npmjs.org/string-to-js/-/string-to-js-0.0.1.tgz
npm http 200 https://registry.npmjs.org/component-builder/0.6.3
npm http GET https://registry.npmjs.org/component-builder/-/component-builder-0.6.3.tgz
npm http 200 https://registry.npmjs.org/jog/0.4.0
npm http GET https://registry.npmjs.org/jog/-/jog-0.4.0.tgz
npm http 200 https://registry.npmjs.org/win-fork/1.0.0
npm http GET https://registry.npmjs.org/win-fork/-/win-fork-1.0.0.tgz
npm http 200 https://registry.npmjs.org/superagent/0.10.0
npm http GET https://registry.npmjs.org/superagent/-/superagent-0.10.0.tgz
npm http 200 https://registry.npmjs.org/batch/0.2.1
npm http GET https://registry.npmjs.org/batch/-/batch-0.2.1.tgz
npm http 200 https://registry.npmjs.org/archy/0.0.2
npm http GET https://registry.npmjs.org/archy/-/archy-0.0.2.tgz
npm http 200 https://registry.npmjs.org/mkdirp/0.3.4
npm http 200 https://registry.npmjs.org/string-to-js/-/string-to-js-0.0.1.tgz
npm http GET https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.4.tgz
npm http 200 https://registry.npmjs.org/component-builder/-/component-builder-0.6.3.tgz
npm http 200 https://registry.npmjs.org/debug
npm http GET https://registry.npmjs.org/debug/-/debug-0.7.2.tgz
npm http 200 https://registry.npmjs.org/commander/1.0.5
npm http GET https://registry.npmjs.org/commander/-/commander-1.0.5.tgz
npm http 200 https://registry.npmjs.org/jog/-/jog-0.4.0.tgz
npm http 200 https://registry.npmjs.org/win-fork/-/win-fork-1.0.0.tgz
npm http 200 https://registry.npmjs.org/superagent/-/superagent-0.10.0.tgz
npm http 200 https://registry.npmjs.org/batch/-/batch-0.2.1.tgz
npm http 200 https://registry.npmjs.org/archy/-/archy-0.0.2.tgz
npm http 200 https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.4.tgz
npm http 200 https://registry.npmjs.org/debug/-/debug-0.7.2.tgz
npm http 200 https://registry.npmjs.org/commander/-/commander-1.0.5.tgz
npm http GET https://registry.npmjs.org/fs-extra
npm http GET https://registry.npmjs.org/component-require/0.2.0
npm http GET https://registry.npmjs.org/redis/0.7.1
npm http GET https://registry.npmjs.org/ms/0.1.0
npm http GET https://registry.npmjs.org/better-assert
npm http GET https://registry.npmjs.org/commander/0.5.2
npm http GET https://registry.npmjs.org/keypress
npm http GET https://registry.npmjs.org/qs/0.5.2
npm http GET https://registry.npmjs.org/formidable/1.0.9
npm http GET https://registry.npmjs.org/mime/1.2.5
npm http GET https://registry.npmjs.org/emitter-component/0.0.6
npm http GET https://registry.npmjs.org/methods/0.0.1
npm http GET https://registry.npmjs.org/cookiejar/1.3.0
npm http 200 https://registry.npmjs.org/ms/0.1.0
npm http GET https://registry.npmjs.org/ms/-/ms-0.1.0.tgz
npm http 200 https://registry.npmjs.org/redis/0.7.1
npm http GET https://registry.npmjs.org/redis/-/redis-0.7.1.tgz
npm http 200 https://registry.npmjs.org/better-assert
npm http GET https://registry.npmjs.org/better-assert/-/better-assert-0.1.0.tgz
npm http 200 https://registry.npmjs.org/component-require/0.2.0
npm http GET https://registry.npmjs.org/component-require/-/component-require-0.2.0.tgz
npm http 200 https://registry.npmjs.org/commander/0.5.2
npm http GET https://registry.npmjs.org/commander/-/commander-0.5.2.tgz
npm http 200 https://registry.npmjs.org/keypress
npm http GET https://registry.npmjs.org/keypress/-/keypress-0.1.0.tgz
npm http 200 https://registry.npmjs.org/qs/0.5.2
npm http GET https://registry.npmjs.org/qs/-/qs-0.5.2.tgz
npm http 200 https://registry.npmjs.org/mime/1.2.5
npm http GET https://registry.npmjs.org/mime/-/mime-1.2.5.tgz
npm http 200 https://registry.npmjs.org/fs-extra
npm http GET https://registry.npmjs.org/fs-extra/-/fs-extra-0.5.0.tgz
npm http 200 https://registry.npmjs.org/formidable/1.0.9
npm http GET https://registry.npmjs.org/formidable/-/formidable-1.0.9.tgz
npm http 200 https://registry.npmjs.org/methods/0.0.1
npm http GET https://registry.npmjs.org/methods/-/methods-0.0.1.tgz
npm http 200 https://registry.npmjs.org/cookiejar/1.3.0
npm http GET https://registry.npmjs.org/cookiejar/-/cookiejar-1.3.0.tgz
npm http 200 https://registry.npmjs.org/emitter-component/0.0.6
npm http 200 https://registry.npmjs.org/ms/-/ms-0.1.0.tgz
npm http GET https://registry.npmjs.org/emitter-component/-/emitter-component-0.0.6.tgz
npm http 200 https://registry.npmjs.org/redis/-/redis-0.7.1.tgz
npm http 200 https://registry.npmjs.org/better-assert/-/better-assert-0.1.0.tgz
npm http 200 https://registry.npmjs.org/component-require/-/component-require-0.2.0.tgz
npm http 200 https://registry.npmjs.org/commander/-/commander-0.5.2.tgz
npm http 200 https://registry.npmjs.org/keypress/-/keypress-0.1.0.tgz
npm http 200 https://registry.npmjs.org/qs/-/qs-0.5.2.tgz
npm http 200 https://registry.npmjs.org/mime/-/mime-1.2.5.tgz
npm http 200 https://registry.npmjs.org/fs-extra/-/fs-extra-0.5.0.tgz
npm http 200 https://registry.npmjs.org/formidable/-/formidable-1.0.9.tgz
npm http 200 https://registry.npmjs.org/methods/-/methods-0.0.1.tgz
npm WARN package.json methods@0.0.1 No README.md file found!
npm http GET https://registry.npmjs.org/callsite
npm http GET https://registry.npmjs.org/ncp
npm http GET https://registry.npmjs.org/jsonfile
npm http 200 https://registry.npmjs.org/cookiejar/-/cookiejar-1.3.0.tgz
npm http GET https://registry.npmjs.org/rimraf
npm http 200 https://registry.npmjs.org/emitter-component/-/emitter-component-0.0.6.tgz
npm http 200 https://registry.npmjs.org/jsonfile
npm http GET https://registry.npmjs.org/jsonfile/-/jsonfile-0.0.1.tgz
npm http 200 https://registry.npmjs.org/rimraf
npm http GET https://registry.npmjs.org/rimraf/-/rimraf-2.1.4.tgz
npm http 200 https://registry.npmjs.org/callsite
npm http GET https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz
npm http 200 https://registry.npmjs.org/ncp
npm http GET https://registry.npmjs.org/ncp/-/ncp-0.2.7.tgz
npm http 200 https://registry.npmjs.org/jsonfile/-/jsonfile-0.0.1.tgz
npm http 200 https://registry.npmjs.org/rimraf/-/rimraf-2.1.4.tgz
npm http 200 https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz
npm http 200 https://registry.npmjs.org/ncp/-/ncp-0.2.7.tgz
npm http GET https://registry.npmjs.org/graceful-fs
npm http 200 https://registry.npmjs.org/graceful-fs
npm http GET https://registry.npmjs.org/graceful-fs/-/graceful-fs-1.2.0.tgz
npm http 200 https://registry.npmjs.org/graceful-fs/-/graceful-fs-1.2.0.tgz
/usr/local/bin/component-info -> /usr/local/lib/node_modules/component/bin/component-info
/usr/local/bin/component -> /usr/local/lib/node_modules/component/bin/component
/usr/local/bin/component-ls -> /usr/local/lib/node_modules/component/bin/component-ls
/usr/local/bin/component-build -> /usr/local/lib/node_modules/component/bin/component-build
/usr/local/bin/component-convert -> /usr/local/lib/node_modules/component/bin/component-convert
/usr/local/bin/component-install -> /usr/local/lib/node_modules/component/bin/component-install
/usr/local/bin/component-help -> /usr/local/lib/node_modules/component/bin/component-help
/usr/local/bin/component-search -> /usr/local/lib/node_modules/component/bin/component-search
/usr/local/bin/component-create -> /usr/local/lib/node_modules/component/bin/component-create
/usr/local/bin/component-changes -> /usr/local/lib/node_modules/component/bin/component-changes
/usr/local/bin/component-docs -> /usr/local/lib/node_modules/component/bin/component-docs
/usr/local/bin/component-wiki -> /usr/local/lib/node_modules/component/bin/component-wiki
npm ERR! Error: EMFILE, too many open files
npm ERR!     at new Glob (/usr/local/lib/node_modules/npm/node_modules/glob/glob.js:104:21)
npm ERR!     at glob (/usr/local/lib/node_modules/npm/node_modules/glob/glob.js:57:11)
npm ERR!     at gypfile (/usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:150:17)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:137:40
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:54:35
npm ERR!     at Array.forEach (native)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:54:11
npm ERR!     at Array.forEach (native)
npm ERR!     at asyncMap (/usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:53:8)
npm ERR!     at extras (/usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:136:17)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 12.2.1
npm ERR! command "node" "/usr/local/bin/npm" "i" "-g" "component"

/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:247
  log.error("cwd", process.cwd())
                           ^
Error: EMFILE, too many open files
    at process.errorHandler (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:247:28)
    at process.EventEmitter.emit (events.js:117:20)
    at process._fatalException (node.js:272:26)
Farid Nouri Neshat

I got this too on a installing a repo with a lot of dependencies:

npm ERR! fetch failed https://registry.npmjs.org/connect/-/connect-2.4.6.tgz
npm ERR! Error: EMFILE, open '/Users/faridneshat/.npm/f9af8e28-lob-whatev-glob-whatev-0-1-8-tgz.lock'
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 10.8.0
npm ERR! command "/Users/faridneshat/nvm/v0.10.1/bin/node" "/Users/faridneshat/nvm/v0.10.1/bin/npm" "install"
npm ERR! Error: EMFILE, too many open files
npm ERR!     at errorHandler (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/utils/error-handler.js:247:28)
npm ERR!     at cb (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:79:20)
npm ERR!     at /Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:153:26
npm ERR!     at cb (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:475:40)
npm ERR!     at cb (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:706:5
npm ERR!     at /Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:805:24
npm ERR!     at /Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/node_modules/slide/lib/chain.js:16:22
npm ERR!     at /Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/install.js:967:14
npm ERR!     at cb (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 10.8.0
npm ERR! command "/Users/faridneshat/nvm/v0.10.1/bin/node" "/Users/faridneshat/nvm/v0.10.1/bin/npm" "install"

/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/utils/error-handler.js:247
  log.error("cwd", process.cwd())
                           ^
Error: EMFILE, too many open files
    at process.errorHandler (/Users/faridneshat/nvm/v0.10.1/lib/node_modules/npm/lib/utils/error-handler.js:247:28)
    at process.EventEmitter.emit (events.js:117:20)
    at process._fatalException (node.js:272:26)

The limit(from ulimit) was 256. Increasing it to 2048 fixed it.

Nathan Zadoks
Collaborator

Should we be handling this in npm? Or should we consider too low a ulimit user error?

Julian Gruber

npm shouldn't require a custom ulimit. That's what @isaacs made graceful-fs for.

Nathan Zadoks
Collaborator

Okay, brilliant. Pullreq welcome.

Julian Gruber

it's used already :D that's the weird thing.

Kevin Dente

I'm seeing this one a ton too, all of a sudden. Never saw it until recently. ulimit works around the problem.

Joshua Gross

I'm getting this today with unlimited ulimit...

Isaac Z. Schlueter
Owner

@JoshuaGross What does ulimit -a report?

Isaac Z. Schlueter
Owner

This is a bug in graceful-fs. It's catching EMFILEs from fs.open, but not from fs.readdir, which also uses a file descriptor. Patch coming soon.

Aside, I'm not sure if I'd call it a bug in npm, per se, but it's certainly reading a lot more files and dirs than it really ought to have to in order to do an ls, so that's not very efficient. Probably a ** in a glob somewhere making it traverse the entire tree.

In the meantime, you can do sudo ulimit -n 2560 (or "unlimited" or some other big number) which seems to make this go away in the short term.

Joshua Gross
Nick Nisi

I'm getting this too, trying to install bower, component, grunt-cli, etc. globally.

Nick Nisi

Everything seems to be resolved after I reinstalled node 0.10.3

Max Desyatov

reproducible with node 0.10.5 on OS X. node was installed from MacPorts if that matters

Nick Nisi

I'm still having the issue. I have 0.10.3 installed on OSX 10.8 with nvm, but the nvm part doesn't seem to matter. I get this every time I install something globally. I get around it by running nvm use 0.10.3, which seems to allow me to install the next thing globally.

Justin Thomas
base698 commented May 07, 2013

Still have the issue in 0.10.5 Mac default ulimit only 256

Jean-Claude Wippler
jcw commented May 10, 2013

Got this error here too, npm 1.2.18, node 0.10.5, macosx 10.8.3, ulimit default 256.

Edit: the command was "npm ls -g" - but "npm update -g" works fine.

Eric Capps

+1

Pierce Moore

Just to chime in, this is still an issue. npm install -g grunt-cli throws numerous errors about missing keys and EMFILE, even navigating to a local directory and running npm install throws the EMFILE error.

Matthias Lienau
atufkas commented May 27, 2013

+1 Same with me. OSX 10.8, node 0.10.7 (via Installer), npm 1.2.23. Issue "EMFILE, too many open files" now occurs in global npm update. Latest installs were yo, bower and grunt-cli. I know that's probably several issues but: Overall npm install/update errors and warnings (unmet dependencies, that "no repository field" thing) heavily increased since upgrading from node 0.8.x and installing latest npm with that.

Kenan Sulayman

Stop posting "me too" comments. OSX's 256 max-filedescriptors in default are ought to be erroneous. Try sudo ulimit -n 4096 & ulimit -n 4096 (userland) and you'll get temporarily around that. By the way, running npm install et al. using sudo will often get you even further to the goal.

Mariusz Nowak
medikoo commented May 30, 2013

@KenanSulayman you shouldn't hack your system to have NPM working, it should be resolved internally within NPM, that's possible, there are known solutions for that.

Isaac Z. Schlueter
Owner
isaacs commented May 30, 2013

In the latest npm release, this is fixed. Node v0.10.9 will be released today with that npm update. This issue is closed. In the meantime, if you can't upgrade, bump up your unlimit -n.

Isaac Z. Schlueter isaacs closed this May 30, 2013
Isaac Z. Schlueter
Owner
isaacs commented May 30, 2013

Note: Setting the ulimit -n 16 does make it die, since it ends up not having an available file descriptor to do the DNS lookup for the registry. But with ulimit -n 64, it works fine.

Eric Capps

+1 to @medikoo. Unnecessary use of sudo is bad news.

Thanks for the fix @isaacs.

hudgins
hudgins commented May 31, 2013

@isaacs This appears to still be happening (with v0.10.9 and npm 1.2.24) if you try to install a module with --dev (try 'npm install --dev tap').

Blake Embrey

Still happening to me on the latest install when installing globally.

node -v
v0.10.9
npm -v
1.2.24
Brainux
Brainux commented May 31, 2013

ulimit -n 16, 64, 128, 256, 512, 1024, 2048, I stopped here.

Still happening to me on the latest install when installing globally.

node -v
v0.10.9
npm -v
1.2.24

Brainux
Brainux commented May 31, 2013

After restarting it is ok,like @JoshuaGross said above.

Isaac Z. Schlueter
Owner
isaacs commented May 31, 2013

@Brainux Restarting what?

What are you guys installing that's causing this error? Is there any way I can reproduce it?

hudgins

@isaacs Did you try 'npm install --dev tap'? I tried rebooting my Mac and that does seem to have improved things for some reason, but it still dies eventually with errors like:

npm ERR! Error: ENOENT, utime '/Users/ahudgins/bug/node_modules/tap/node_modules/glob/node_modules/minimatch/node_modules/lru-cache/node_modules/weak/node_modules/mocha/node_modules/coffee-script/node_modules/jison/node_modules/browserify/node_modules/browser-resolve/node_modules/crypto-browserify/node_modules/tape/.travis.yml'

and ultimately:

npm ERR! Error: EMFILE, open '/Users/ahudgins/bug/node_modules/tap/node_modules/glob/node_modules/minimatch/node_modules/lru-cache/node_modules/weak/node_modules/mocha/node_modules/coffee-script/node_modules/jison/node_modules/browserify/node_modules/umd/node_modules/rfile/node_modules/callsite/Makefile'

Francisco López

I try ulimit command with 10000+ and not solve my problem, neither rebooting. I have a Mac.

Brainux

@isaacs, in my case, restart == reboot. I´m using:
Ubuntu
node -v
v0.10.9
npm -v
1.2.24

My partial .bash_history
...
npm install -g buster phantom
npm install -g phantom
npm install -g buster
apt-get update
apt-get install nodejs //nothing new
npm install -g buster
npm install -verbose -g buster
npm install -g phantom
npm uninstall -g phantom
npm install -g phantom
ulimit -n 64
npm install -g phantom
npm install -verbose -g phantom
npm update -g
ulimit -n 128
npm update -g
ulimit -n 256
npm update -g
ulimit -n 512
npm update -g
ulimit -n 1024
npm update -g
ulimit -n 2048
npm update -g
unlimit -a
ulimit -a
ulimit -n 2048
ulimit -a
npm install -g phantom
npm install -g generator-karma@~0.2.0
npm install -g
npm install -g buster
// ahahahahhhhhhhhhhhh!!!!
init 6
npm install -g phantom
npm install -g buster
:)

// one day after:
npm install -g wordpress.js //error again
npm install -g wordpress.js //error again
npm install -g -verbose wordpress.js //error again
npm install -g -verbose wordpress.js //error again
npm install -g wordpress.js // error again
:(
npm install -g wordpress.js // it´s working!!!! No "init 6" !!!
//very annoying issue
//To test:
npm uninstall -g -verbose phantom //ok
npm install -g phantom //ok
npm uninstall -g -verbose buster //ok
npm install -g buster //ok
npm uninstall -g wordpress.js //ok
npm install -g wordpress.js //ok

after some erros, node just worked by itself.

Is there a module to generate a standard report and post here?

Jean-Claude Wippler
jcw commented June 06, 2013

Node 0.10.10, npm 1.2.25, OSX 10.8.4, still getting the EMFILE with "npm install -g hubot" (npm update -g worked).

Note ulimit -n is at default 256 - I don't think the answer should be "adjust your OS settings to make npm work", the ball really is in npm's court, I'm afraid - it's the only command generating this type of error.

Kenan Sulayman

I don't understand the I don't think the answer should be "adjust your OS settings to make npm work" issue — no operating system except OSX has a default fd-max of 0x100. Why should every wheel be reinvented, just because Apple thinks that's right?

Domenic Denicola
Collaborator

@isaacs You said earlier:

This is a bug in graceful-fs. It's catching EMFILEs from fs.open, but not from fs.readdir, which also uses a file descriptor. Patch coming soon.

I didn't see a patch to graceful-fs. Perhaps that could explain what our users are seeing?

Jean-Claude Wippler
jcw commented June 06, 2013

@KenanSulayman - npm is a tool (one of many), OSX is the environment in which it runs. I see two reasons:

  1. I don't really want to let a tool dictate the global settings of my system (where would one draw the line?)
  2. solving it once within npm means that lots of people don't have to address (or even know about) this issue

Having said that, I agree that the best solution would be for Apple to make better default choices.

With apologies if my choice of words came across as too harsh.

Jozef Izso

I'm having issue with updating globally installed packge in npm:

$ node -v
v0.10.10

$ npm -v
1.2.30

$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.8.4 (12E55)
      Kernel Version: Darwin 12.4.0
$ sudo npm update -g generator-server-configs
npm WARN package.json bytes@0.2.0 No repository field.
npm WARN package.json cookie-signature@1.0.1 No repository field.
npm WARN package.json fresh@0.1.0 No repository field.
npm WARN package.json pause@0.0.1 No repository field.
npm WARN package.json dateformat@1.0.2-1.2.3 No repository field.
npm WARN package.json eventemitter2@0.4.11 No repository field.
npm WARN package.json eventemitter2@0.4.11 'repositories' (plural) Not supported.
npm WARN package.json Please pick one as the 'repository' field
npm WARN package.json cli-table@0.2.0 No repository field.
npm ERR! Error: EMFILE, open '/usr/local/lib/node_modules/yeoman-generator/node_modules/request/node_modules/forever-agent/package.json'
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 12.4.0
npm ERR! command "node" "/usr/local/bin/npm" "update" "-g" "generator-server-configs"
npm ERR! Error: EMFILE, too many open files
npm ERR!     at errorHandler (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:266:28)
npm ERR!     at /usr/local/lib/node_modules/npm/lib/update.js:27:20
npm ERR!     at /usr/local/lib/node_modules/npm/lib/outdated.js:33:30
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /usr/local/lib/node_modules/npm/lib/outdated.js:100:22
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /usr/local/lib/node_modules/npm/lib/outdated.js:96:73
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:53:40
npm ERR!     at parseJson (/usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:82:32)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 12.4.0
npm ERR! command "node" "/usr/local/bin/npm" "update" "-g" "generator-server-configs"

/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:266
  log.error("cwd", process.cwd())
                           ^
Error: EMFILE, too many open files
    at process.errorHandler (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:266:28)
    at process.EventEmitter.emit (events.js:117:20)
    at process._fatalException (node.js:272:26)
Dan Kohn dankohn referenced this issue in Medium/phantomjs June 17, 2013
Closed

Can't install on OS X #68

Michael Wales

I came across this issue last night; as others have mentioned ulimit -n 4096 has taken care of the issue for now.

Relevant terminal activity:

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

$ node -v
v0.10.12

$ npm -v
1.2.32


$ npm update -g
...
npm ERR! Error: EMFILE, too many open files
...
npm ERR! System Darwin 12.4.0
npm ERR! command "/usr/local/Cellar/node/0.10.12/bin/node" "/usr/local/bin/npm" "update" "-g"
npm ERR! cwd /Users/mike/Sites"
npm ERR! node -v v0.10.12
npm ERR! npm -v 1.2.32
npm ERR! syscall uv_cwd
npm ERR! code EMFILE
npm ERR! errno 20
...
Error: EMFILE, too many open files
    at process.errorHandler (/usr/local/Cellar/node/0.10.12/lib/node_modules/npm/lib/utils/error-handler.js:266:28)
    at process.EventEmitter.emit (events.js:117:20)
    at process._fatalException (node.js:272:26)
Gilles Ruppert

I just ran into this issue again with the following versions on OSX 10.8 with the default ulimit:

node -v # v0.10.12
npm -v # 1.2.32

Running the following command breaks it every single time:

npm update -g bower # same for npm update bower -g

Setting the ulimit to 4096 solves the issue, but as others have mentioned, this is not great. At the very least the error message could maybe mention this?

Mike Kamermans
Pomax commented July 09, 2013

can this ticket be reopened, it looks like it's definitely not fixed yet (just ran into the same thing, found this thread, there's lots of comments after the initial closing that suggests this bug persists)

Nathan Zadoks nathan7 reopened this July 09, 2013
Isaac Z. Schlueter isaacs referenced this issue from a commit July 10, 2013
Isaac Z. Schlueter graceful-fs@1.2.3
Related to #3259

Because readFile and writeFile were not wrapped, there were cases where
a lot of package.json reading would cause problems.  This wasn't an
issue with earlier versions of graceful-fs, because it monkey-patched
the builtin FS module's open() method, so that readFile and writeFile
would Just Work.

However, that caused other problems.

Note that fs.ReadStream and fs.WriteStream are still not covered in the
open fd guard.  That can be done later, but this extends EMFILE coverage
to the more commonly used methods first.
e300730
Isaac Z. Schlueter
Owner
isaacs commented July 10, 2013

Can one of you try running this and seeing if it fixes things for you? (Without the ulimit change, of course.)

npm i -g git://github.com/isaacs/npm#e300730
Gilles Ruppert

@isaacs This worked for me. (ulimit set to 256)
I double-checked with npm v1.2.32 and 1.3.2 (the bundled versions that come with 0.10.12 and 0.10.13) and both break. Installing your version works.

I tested with the command npm update -g bower.

Thanks for the fix. I hope this sorts it for everybody.

Per Ejeklint

@isaacs Nope, I tried npm i -g git://github.com/isaacs/npm#e300730 but it's the same. And I have to reinstall npm after each try.

Gilles Ruppert

@ejeklint I had to run @isaacs command with sudo since the permissions of the bundled npm are different, so I ran into that problem as well.
Which command failed on your system? That way I could try on mine to see whether it's consistent.

For the record: the new version fixed the example I posted earlier (npm update -g bower).

Per Ejeklint

@gillesruppert Yes I ran with sudo. Also tried a clean install of npm with entire /usr/local chmodded to myself.

It's npm update -g that always fails. Updating a single package works fine.

Gilles Ruppert

@ejeklint you are right: running npm update -g I get the dreaded EMFILE too many files open error with a ulimit of 256. As usual, a higher ulimit sorts it. /cc @isaacs

Jason Denizac
jden commented July 10, 2013

I am running into the same issues with EMFILE with the default ulimit of 256 on OSX / node 0.10.13 / npm 1.3.2 with a non-global dependency tree on one of my applications - was not receiving this with previous versions of node. npm ls works fine, but npm update or npm install from a clean node_modules directory fails.

Isaac Z. Schlueter
Owner
isaacs commented July 10, 2013

Yeah, I can reproduce this. Rewriting graceful-fs to go back to monkey-patching the native fs module, since that's actually hte only thing that will work here.

Isaac Z. Schlueter isaacs closed this July 11, 2013
Isaac Z. Schlueter
Owner
isaacs commented July 11, 2013

Boy this was a doozy. I think I've got it worked out. Here's what was happening:

  1. The graceful-fs module sets up a queue system so that EMFILE failures will be retried.
  2. The lockfile module opens a lock file (and keeps it open -- stupid) in exclusive mode to prevent collisions.
  3. npm outdated adds packages to the cache (again, stupid) rather than simply checking which versions are available, so it does tons of tarball fetching and unpacking.

To address this problem:

  1. graceful-fs 2.0.0 again monkey-patches the native fs module. Even though monkey patching is gross, and I'm against it in principle, it's the only way to ensure that we catch every instance of a file open operation. Also, it's much simpler and less clever now, keeping a basic queue rather than attempting to count FDs, set a max, and detect the ulimit hit.
  2. lockfile 0.4.0 doesn't keep the files open. This means that it has to not pass the fd along, but that's fine. You don't want your lockfile to prevent you from doing things that you CAN do safely!
  3. Fixing the npm outdated behavior is a little bit more involved. I just added a feature to the registry in npm/npm-registry-couchapp@e20a430 to let you fetch urls like http://registry.npmjs.org/lockfile/~0.3.2 for returnign the maxSatisfying in a range. So, that should make the creation of a new registry client a bit simpler, since fetching a range is now just like fetching a normal version.

The next release of npm will have these features. Of course, you can pull from the latest git to pull it in now. It's working for me now even with a ulimit of 32 (albeit rather slowly, sometimes to the point where the fetches time out, but usually it will eventually finish).

Per Ejeklint

Great news! Many thanks @isaacs for fixing this so fast.

Jozef Izso

Sounds great! :+1:

Gilles Ruppert

Thanks @isaacs for the fix. Installing now.

Mike Kamermans
Pomax commented July 15, 2013

Wow, that's a nice set of collaborative bugs. Great job finding that out! And props for fixing.

Tom Fuertes

If anyone's still having issues rm -rf ~/.npm helped w/ the current brew install node --HEAD

Igor Soarez Soarez referenced this issue in gruntjs/grunt-cli July 25, 2013
Closed

Installation fails #27

Mike Kamermans
Pomax commented July 29, 2013

Just so we know whether it landed with node 0.10.14/15, which tag would this have landed under? 1.3.4/5?

Tobias Bosch tbosch referenced this issue from a commit in tbosch/angular.js March 12, 2014
Tobias Bosch chore(travis): fix version for grunt-cli and clear npm cache
Related to npm/npm#3259.
49d8914
Aria Stewart aredridel referenced this issue in jedmao/codepainter April 03, 2014
Open

Unhandled EMFILE #26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.