Skip to content
This repository

Error: EMFILE, too many open files #38

Closed
daaain opened this Issue March 19, 2013 · 8 comments

2 participants

Daniel Demmel Krishnan Anantheswaran
Daniel Demmel

I'm working on a project with quite a good number of files (298 as picked up by Istanbul to be precise) and it doesn't seem to be able to deal with them :(

Setup is OS X 10.7.5, Node v0.10.0, Istanbul 0.1.32 and using via Guard-Jasmine 1.13.2.

fs.js:413
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^
Error: EMFILE, too many open files 'path-removed.js'
    at Object.fs.openSync (fs.js:413:18)
    at Object.fs.readFileSync (fs.js:270:15)
    at LookupStore.Store.mix.get (/usr/local/share/npm/lib/node_modules/istanbul/lib/store/fslookup.js:39:19)
    at HtmlReport.Report.mix.writeDetailPage (/usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:358:71)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:435:26
    at Object.FileWriter.writeFile (/usr/local/share/npm/lib/node_modules/istanbul/lib/util/file-writer.js:24:9)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:434:24
    at Array.forEach (native)
    at HtmlReport.Report.mix.writeFiles (/usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:428:23)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:430:22
Krishnan Anantheswaran
Owner

Could you give me the following info?

  • What is the high-level use case? What specific command are you running? Is it istanbul cover ?
  • Can you turn off reporting as part of running your tests and use the istanbul report command to produce the reports as a separate command? (see istanbul help for details). I want to make sure that it is not the test runner that is opening too many files and istanbul just happens to tip the # of open files over the limit.
  • Could you run the original command in node 0.8 and see if that makes any difference? Want to make sure it is not because of the node version.

Thanks for reporting the issue.

Daniel Demmel

My high-level use case is manually generating instrumented JS files with istanbul and then running Jasmine tests using Guard-Jasmine which will then feeds istanbul the results somehow and generates reports. I tried to read the code (relevant commit), but it's a bit too much for me to understand so early in the morning :)

Tried to run istanbul cover without Guard-Jasmine, but got stuck with how to make it run Jasmine tests, I'm not using node at all just plain client side JS. It can very well be the case that there's something with the way Guard-Jasmine is using istanbul, if you think that's the case I can open an issue there.

It's also happening with node 0.8.22.

Edit: Actually, there's one more piece to the puzzle I just realised, I do get a report on the CLI from Guard-Jasmine:

Spec coverage summary:

Statements   : 37.97% ( 6903/18178 )
Branches     : 26.18% ( 2210/8443 )
Functions    : 28.97% ( 1123/3876 )
Lines        : 38.04% ( 6899/18135 )

Which implies that the coverage bit is completely fine, it's the HTML report generation which seems to break. This also lines up with what's in the stack trace, but thought I should mention.

Krishnan Anantheswaran
Owner

This is good information - it proves that the reporter is opening too many files at the same time and it is not a side effect of the test runner.

I'm totally tied up with my day job right now so may not be able to get to a fix soon.

This is how I think you can get unblocked:

reporter.writeReport(collector, true); /* sync mode */
  • Use your fork in your package.json

Let me know if this unblocks you. I will get to a permanent fix when my current day-job deadlines are met.

Krishnan Anantheswaran
Owner

Yeah, I can reproduce it and, yeah, that fix should work.

I don't know what I was smoking when I wrote the async version of the reporter but it will take some time to untangle :)

Let me know if this is a workable solution for you for the time being.

Daniel Demmel

That worked perfectly, thanks :+1: Glad I could help with the report :)

It's quite fast as well, so this is not a bad fix :)

I don't even have a package.json (it's a pure client side project with some Ruby gems for compilation sprinkled on top), but a little bit of Googling gave me the quick and dirty way: npm install git://github.com/daaain/istanbul.git -g

Edit: Oh yeah, and by the way it works with node 0.10.0!

Krishnan Anantheswaran
Owner

Cool. I'll roll in a fix in the next week or two.

Krishnan Anantheswaran gotwarlost closed this in dbe3582 March 31, 2013
Krishnan Anantheswaran
Owner

@daaain - Fix is available in v0.1.34.

Please try it and let me know if it looks ok. Thanks!

Daniel Demmel

It works perfectly (tested with node 0.10.2), thanks a lot for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.