New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipe(): Too many open files #1537

Closed
braddunbar opened this Issue Jul 22, 2011 · 10 comments

Comments

Projects
None yet
9 participants
@braddunbar

braddunbar commented Jul 22, 2011

When running coffee on a directory with a large number of files (329 to be exact), I receive the following error:

pipe(): Too many open files

child_process.js:243
  var fds = this._internal.spawn(path,
                       ^
Error: Error spawning
    at ChildProcess.spawn (child_process.js:243:28)
    at child_process.js:31:15
    at child_process.js:77:15
    at child_process.js:38:27
    at /Users/brad/dev/coffee-script/lib/command.js:242:16
    at path.js:292:19

The failure seems to center on the call to mkdir -p. I've yet to find a way to fix this cleanly.

@braddunbar

This comment has been minimized.

Show comment
Hide comment
@braddunbar

braddunbar Jul 22, 2011

After several runs, the issue manifests as

EMFILE, Too many open files '/path/to/file'

Also, this was observed on OSX.

braddunbar commented Jul 22, 2011

After several runs, the issue manifests as

EMFILE, Too many open files '/path/to/file'

Also, this was observed on OSX.

@lancejpollard

This comment has been minimized.

Show comment
Hide comment
@lancejpollard

lancejpollard Dec 27, 2011

I'm getting a similar error because of too many files open:

fs.js:632
    throw errnoException(errno, 'watch');
10:22:36 - compiled src/design.io/support.coffee
^
Error: watch EMFILE
    at errnoException (fs.js:605:11)
    at FSWatcher.start (fs.js:632:11)
    at Object.watch (fs.js:660:11)
    at Object.oncomplete (/Users/viatropos/Documents/git/personal/plugins/design.io/node_modules/coffee-script/lib/coffee-script/command.js:201:17)

This should just be written with something like the async.js library:

https://github.com/jashkenas/coffee-script/blob/master/src/command.coffee#L106

lancejpollard commented Dec 27, 2011

I'm getting a similar error because of too many files open:

fs.js:632
    throw errnoException(errno, 'watch');
10:22:36 - compiled src/design.io/support.coffee
^
Error: watch EMFILE
    at errnoException (fs.js:605:11)
    at FSWatcher.start (fs.js:632:11)
    at Object.watch (fs.js:660:11)
    at Object.oncomplete (/Users/viatropos/Documents/git/personal/plugins/design.io/node_modules/coffee-script/lib/coffee-script/command.js:201:17)

This should just be written with something like the async.js library:

https://github.com/jashkenas/coffee-script/blob/master/src/command.coffee#L106

@Crash--

This comment has been minimized.

Show comment
Hide comment
@Crash--

Crash-- Dec 28, 2011

I'm getting the same error that @viatropos on OSX :

coffee -b -c -w -l ../
/Users/***/Sites/***/tmp/bin/nodejs/node_modules/coffee-script/lib/coffee-script/command.js:287
      if (e.code !== 'ENOENT') throw e;
                                     ^
Error: watch EMFILE
    at errnoException (fs.js:636:11)
    at FSWatcher.start (fs.js:663:11)
    at Object.watch (fs.js:691:11)

If I run this command on a smallest folder, it's work.

Crash-- commented Dec 28, 2011

I'm getting the same error that @viatropos on OSX :

coffee -b -c -w -l ../
/Users/***/Sites/***/tmp/bin/nodejs/node_modules/coffee-script/lib/coffee-script/command.js:287
      if (e.code !== 'ENOENT') throw e;
                                     ^
Error: watch EMFILE
    at errnoException (fs.js:636:11)
    at FSWatcher.start (fs.js:663:11)
    at Object.watch (fs.js:691:11)

If I run this command on a smallest folder, it's work.

@braddunbar

This comment has been minimized.

Show comment
Hide comment
@braddunbar

braddunbar Dec 29, 2011

@craSH, @viatropos -- I have been using ulimit -n to get around this issue for the time being.

It seems odd that system settings, i.e. file descriptor limit, should cause a scaling issue. I feel as though the program should wait until a file descriptor becomes available rather than throw. However, I'm still unsure about what level this type of error should be handled at.

braddunbar commented Dec 29, 2011

@craSH, @viatropos -- I have been using ulimit -n to get around this issue for the time being.

It seems odd that system settings, i.e. file descriptor limit, should cause a scaling issue. I feel as though the program should wait until a file descriptor becomes available rather than throw. However, I'm still unsure about what level this type of error should be handled at.

@craSH

This comment has been minimized.

Show comment
Hide comment
@craSH

craSH Dec 29, 2011

@braddunbar different craSH - you meant @Crash-- ;) cheers!

craSH commented Dec 29, 2011

@braddunbar different craSH - you meant @Crash-- ;) cheers!

@TrevorBurnham

This comment has been minimized.

Show comment
Hide comment
@TrevorBurnham

TrevorBurnham Jan 6, 2012

Collaborator

I just encountered the EMFILE error for the first time myself. I added a console.log to show which things were being watched, and the result was 211 lines of the form

.git/objects/00
.git/objects/01
.git/objects/02

and so on.

I've reported this as Node issue #2479. For now, you should work around it by only using coffee -w on directories that don't have more than a couple hundred items.

Collaborator

TrevorBurnham commented Jan 6, 2012

I just encountered the EMFILE error for the first time myself. I added a console.log to show which things were being watched, and the result was 211 lines of the form

.git/objects/00
.git/objects/01
.git/objects/02

and so on.

I've reported this as Node issue #2479. For now, you should work around it by only using coffee -w on directories that don't have more than a couple hundred items.

@TrevorBurnham

This comment has been minimized.

Show comment
Hide comment
@TrevorBurnham

TrevorBurnham Jan 7, 2012

Collaborator

Ben Noordhuis has stated that there's nothing Node (or CoffeeScript) can do to work around this limit; you can modify it at the OS level using ulimit.

So, closing this issue. We should probably ignore hidden subdirectories to get around the git repo case.

Collaborator

TrevorBurnham commented Jan 7, 2012

Ben Noordhuis has stated that there's nothing Node (or CoffeeScript) can do to work around this limit; you can modify it at the OS level using ulimit.

So, closing this issue. We should probably ignore hidden subdirectories to get around the git repo case.

@jashkenas jashkenas reopened this Feb 21, 2012

@jashkenas

This comment has been minimized.

Show comment
Hide comment
@jashkenas

jashkenas Feb 21, 2012

Owner

Reopening to ignore hidden subdirectories ;)

Owner

jashkenas commented Feb 21, 2012

Reopening to ignore hidden subdirectories ;)

@michaelficarra

This comment has been minimized.

Show comment
Hide comment
@michaelficarra

michaelficarra Feb 22, 2012

Collaborator

Closed by 127653b.

Collaborator

michaelficarra commented Feb 22, 2012

Closed by 127653b.

@bzarzuela

This comment has been minimized.

Show comment
Hide comment
@bzarzuela

bzarzuela Jan 22, 2013

I'm experiencing this with version 1.4.0. In the directory I'm trying to watch, there is a symlink to another directory. Total number of files is less than 300. If I remove the symlink, the problem goes away. Using OSX Mountain Lion.

bzarzuela commented Jan 22, 2013

I'm experiencing this with version 1.4.0. In the directory I'm trying to watch, there is a symlink to another directory. Total number of files is less than 300. If I remove the symlink, the problem goes away. Using OSX Mountain Lion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment