Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

pipe(): Too many open files #1537

Closed
braddunbar opened this Issue · 10 comments

9 participants

@braddunbar

When running coffee on a directory with a large number of files (329 to be exact), I receive the following error:

pipe(): Too many open files

child_process.js:243
  var fds = this._internal.spawn(path,
                       ^
Error: Error spawning
    at ChildProcess.spawn (child_process.js:243:28)
    at child_process.js:31:15
    at child_process.js:77:15
    at child_process.js:38:27
    at /Users/brad/dev/coffee-script/lib/command.js:242:16
    at path.js:292:19

The failure seems to center on the call to mkdir -p. I've yet to find a way to fix this cleanly.

@braddunbar

After several runs, the issue manifests as

EMFILE, Too many open files '/path/to/file'

Also, this was observed on OSX.

@lancejpollard

I'm getting a similar error because of too many files open:

fs.js:632
    throw errnoException(errno, 'watch');
10:22:36 - compiled src/design.io/support.coffee
^
Error: watch EMFILE
    at errnoException (fs.js:605:11)
    at FSWatcher.start (fs.js:632:11)
    at Object.watch (fs.js:660:11)
    at Object.oncomplete (/Users/viatropos/Documents/git/personal/plugins/design.io/node_modules/coffee-script/lib/coffee-script/command.js:201:17)

This should just be written with something like the async.js library:

https://github.com/jashkenas/coffee-script/blob/master/src/command.coffee#L106

@Crash--

I'm getting the same error that @viatropos on OSX :

coffee -b -c -w -l ../
/Users/***/Sites/***/tmp/bin/nodejs/node_modules/coffee-script/lib/coffee-script/command.js:287
      if (e.code !== 'ENOENT') throw e;
                                     ^
Error: watch EMFILE
    at errnoException (fs.js:636:11)
    at FSWatcher.start (fs.js:663:11)
    at Object.watch (fs.js:691:11)

If I run this command on a smallest folder, it's work.

@braddunbar

@Crash, @viatropos -- I have been using ulimit -n to get around this issue for the time being.

It seems odd that system settings, i.e. file descriptor limit, should cause a scaling issue. I feel as though the program should wait until a file descriptor becomes available rather than throw. However, I'm still unsure about what level this type of error should be handled at.

@craSH

@braddunbar different craSH - you meant @Crash-- ;) cheers!

@TrevorBurnham
Collaborator

I just encountered the EMFILE error for the first time myself. I added a console.log to show which things were being watched, and the result was 211 lines of the form

.git/objects/00
.git/objects/01
.git/objects/02

and so on.

I've reported this as Node issue #2479. For now, you should work around it by only using coffee -w on directories that don't have more than a couple hundred items.

@TrevorBurnham
Collaborator

Ben Noordhuis has stated that there's nothing Node (or CoffeeScript) can do to work around this limit; you can modify it at the OS level using ulimit.

So, closing this issue. We should probably ignore hidden subdirectories to get around the git repo case.

@jashkenas jashkenas reopened this
@jashkenas
Owner

Reopening to ignore hidden subdirectories ;)

@michaelficarra michaelficarra closed this issue from a commit
Commit has since been removed from the repository and is no longer available.
@michaelficarra
Collaborator

Closed by 127653b.

@yyfearth yyfearth referenced this issue from a commit
@yyfearth yyfearth apply patch from original coffee-script: ignore hidden files/dir
jashkenas/coffee-script: fixes #1537: ignore hidden files/directories in watched directories
5d36e63
@bzarzuela

I'm experiencing this with version 1.4.0. In the directory I'm trying to watch, there is a symlink to another directory. Total number of files is less than 300. If I remove the symlink, the problem goes away. Using OSX Mountain Lion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.