EMFILE, too many open files

[valotas]

I am not sure if it has to do with karma-jspm itself or karma but I'm getting the following

[23:36:35] Using gulpfile ~/Projects/xxx/gulpfile.js
[23:36:35] Starting 'test'...
[23:36:36] 'test' errored after 1.18 s
[23:36:36] Error: EMFILE, too many open files '/xxx/src/test/webapp'
    at Error (native)
    at Object.fs.readdirSync (fs.js:761:18)
    at GlobSync._readdir (/xxx/node_modules/karma/node_modules/glob/sync.js:275:41)
    at GlobSync._processReaddir (/xxx/node_modules/karma/node_modules/glob/sync.js:135:22)
    at GlobSync._process (/xxx/node_modules/karma/node_modules/glob/sync.js:130:10)
    at new GlobSync (/xxx/node_modules/karma/node_modules/glob/sync.js:46:10)
    at new Glob (/xxx/node_modules/karma/node_modules/glob/glob.js:111:12)
    at /xxx/node_modules/karma/lib/file-list.js:163:14
    at Array.map (native)
    at List._refresh (/xxx/node_modules/karma/lib/file-list.js:155:37)
    at List.refresh (/xxx/node_modules/karma/lib/file-list.js:254:27)
    at Server._start (/xxx/node_modules/karma/lib/server.js:173:12)
    at invoke (/xxx/node_modules/karma/node_modules/di/lib/injector.js:75:15)
    at Server.start (/xxx/node_modules/karma/lib/server.js:97:18)
    at Gulp.<anonymous> (/xxx/gulpfile.js:25:10)
    at module.exports (/xxx/node_modules/gulp/node_modules/orchestrator/lib/runTask.js:34:7)

My assumption is that karma can not handle the size of jspm_packages

[lgvo]

Actually I believe is not a bug at all.
You probably don't have enough limit to open files on your user.
(Maybe karma could use something like https://github.com/isaacs/node-graceful-fs to avoid that)

To test, you can change it running the command: ulimit -u 10240

You can add it at your .profile or config in the /etc/security/limits.conf (at least in linux):

For all users, add these lines:

*    soft    nofile        10240
*    hard    nofile        10240

Only for your user (valotas), add these lines:

valotas    soft    nofile        10240
valotas    hard    nofile        10240

Hope it help.

[maxwellpeterson-wf]

Yes, this is an issue with the size of your project/number of dependencies it is using and your OS settings as @lgvo has explained. StackOverflow backs this up. Thanks for the in-depth explanation!

[togakangaroo]

I am having the exact same thing happen on windows 8.1. My karma was working just fine until I upgraded to node 4.1.1 (previously was node 0.latest). Now its telling me too many open files.

[lares83]

I'm having the same issue too, on Windows 10 and node 4.1.1 and I don't know how to fix it?

[sateffen]

We're having a similar issue, but we can't explain it:

Our dev-team uses only windows 7 laptops, and our project is somewhat big (around 1500-2000 js files that get loaded). That should be no problem, and wasn't up till now.

One of my co-workers got a new laptop to replace his old, weak one, with a new, very strong one. The new and pretty strong one has windows 7 64bit system as well, but much stronger hardware, but gets this EMFILE error.

All other, even the old dev machines don't get this error. At our dev machines phantomjs keeps crushing, but that's another "problem". On our build servers (pretty weak Linux machines, but they a build process + testing may take some more minutes, don't care) everything works fine: karma, phantomjs, everything.

So there is only ONE computer, the most powerful, that has the "EMFILE" error, but all other, with the same windows 7 64bit system, don't have this problem.

We use nodejs version 4.1.1 on all tested systems (now), but the different behaviour stays the same.

[togakangaroo]

@lgvo what do you think? Seems like this is a common issue on windows now, so it can't really be tied to what you cited.

[lgvo]

@togakangaroo I do think it is just the same kind of issue.
But I don't use Windows, maybe you can try look if others libraries have the same problem when you upgrade to node 4.x.

I also didn't upgrade my projects to node 4, some backend libs aren't working properly when I tried. But I'm using the lastest iojs 3.x without any problems.

You can try use iojs too, it have almost the same features that the node 4.
I was using node 0.x before and migrate to iojs 3.x to easily upgrade to node 4.x when I find it more stable.

[togakangaroo]

@lgvo hmm, I have noticed zero issues with any other library. I need to upgrade since jspm bundle needs a newer version of node to properly bundle es6 features for production (ie I hate the idea of transforming generators, arrow functions, and const when the latest browsers already support them). Io.js is a non-starter since, not only is it a step backwards, but it would affect all other developers on my project, and all other projects using the same build server.

It is strange though, I've got another small project that has very few dependencies where this is still happening. Is there anything in karma-jspm that you know of that might cause this?

Seems like I might have to spend some time setting up a small issue repro. We can discuss then.

[lgvo]

@togakangaroo if it' s only karma-jspm it must be related to the glob dependency I think, try to use karma-jspm locally (using npm link) and update the glob library and see if it helps. About transpiling ES6 features you can watch the https://github.com/getify/es-feature-tests We are talking about that on getify/ES-Feature-Tests#9. But if you have to support more browser I don't think you can get a production solution on that maybe use the browser support only for your workflow. For backend I create some scripts that work, maybe if I get some time I make a pull request on that.

[sateffen]

Just in case: We don't use karma-jspm, we use tried plain karma and grunt-karma as runners. We have some setup with requirejs, but currently I'm not at work, so I can't tell the exact setup, but it's pretty much like here: https://karma-runner.github.io/0.8/plus/RequireJS.html

[Gu1]

Hey. I have the same problem here. EMFILE errors with karma/karma-jspm since I switched to nodejs 4.x

[togakangaroo]

Just to update everyone... I replicated this in a simple project with just karma, jspm, karma-jspm, and a single test. However, when I upgraded everything to latest, dumped node_modules and re-installed everything started working. Still investigating whether I can get my actual project back to running.

[lgvo]

Looks like that since 4.0.5 glob has support to use graceful-fs Issue.

[zckrs]

Same issue :-/

[wzhao-c]

I have same issue since upgraded to node v5

[xtreemrage]

Same issue, but that can't be the solution to edit a system file... When will this be fixed?

[m-j]

I get the same issue on Windows 8.1. I believe karama tries to "eat" all files at once and it hits system limit, therefore I believe it's an actual bug.

Has aynone tried global patching it?
https://github.com/isaacs/node-graceful-fs

Some more info. I believe problem may be due karma serving node_modules. I ran few commands through cygwin:

$ ls -R . | wc -l
  26428

$ ls -R node_modules | wc -l
  19589

$ ls -R client | wc -l
   3386

(Client is where my code and jspm_packages sit)

[valotas]

I also believe that karma tries to "eat" all the files at once. It can be a general karma problem as angular has the same problem

[rdehuyss]

+1

I'm trying to get the tests in https://github.com/AngularClass/NG6-starter/tree/jspm to work.

C:\Users\bkdevconstructiv\tmp\NG6-starter>systeminfo
...
OS Name:                   Microsoft Windows 8.1 Enterprise
OS Version:                6.3.9600 N/A Build 9600

C:\Users\bkdevconstructiv\tmp\NG6-starter>node --version
v5.4.1

C:\Users\bkdevconstructiv\tmp\NG6-starter>karma --version
Karma version: 0.12.37

My package.json:

{
  "name": "angular-jspm-starter",
  "version": "0.0.1",
  "description": "starter",
  "dependencies": {},
  "scripts": {
    "test": "karma start",
    "postinstall": "jspm install"
  },
  "keywords": [
    "angular",
    "jspm",
    "es6"
  ],
  "author": "AngularClass",
  "license": "ISC",
  "jspm": {
    "directories": {
      "baseURL": "client"
    },
    "configFile": "jspm.config.js",
    "dependencies": {
      "angular": "github:angular/bower-angular@^1.4.0",
      "angular-mocks": "npm:angular-mocks@^1.4.0",
      "angular-ui-router": "github:angular-ui/ui-router@^0.2.15",
      "capaj/systemjs-hot-reloader": "github:capaj/systemjs-hot-reloader@^0.5.0",
      "css": "github:systemjs/plugin-css@^0.1.11",
      "normalize.css": "github:necolas/normalize.css@^3.0.3",
      "text": "github:systemjs/plugin-text@^0.0.2"
    },
    "devDependencies": {
      "babel": "npm:babel-core@^5.8.24",
      "babel-runtime": "npm:babel-runtime@^5.8.24",
      "core-js": "npm:core-js@^1.1.4"
    }
  },
  "devDependencies": {
    "browser-sync": "^2.7.6",
    "chai": "^3.5.0",
    "chokidar-socket-emitter": "^0.3.0",
    "gulp": "^3.9.0",
    "gulp-html-replace": "^1.5.0",
    "gulp-ng-annotate": "^1.0.0",
    "gulp-rename": "^1.2.2",
    "gulp-template": "^3.0.0",
    "gulp-uglify": "^1.2.0",
    "jspm": "^0.16.25",
    "karma": "^0.12.36",
    "karma-chai": "^0.1.0",
    "karma-chrome-launcher": "^0.1.12",
    "karma-jspm": "^2.0.2",
    "karma-mocha": "^0.1.10",
    "karma-mocha-reporter": "^1.0.2",
    "mocha": "^2.4.4",
    "rimraf": "^2.4.3",
    "yargs": "^3.10.0"
  }
}

The error I have:

ERROR [karma]: { [Error: EMFILE: too many open files, open 'C:\Users\bkdevconstructiv\tmp\NG6-starter\client\jspm_packages\npm\util@0.10.3.js']
  errno: -4066,
  code: 'EMFILE',
  syscall: 'open',
  path: 'C:\\Users\\bkdevconstructiv\\tmp\\NG6-starter\\client\\jspm_packages\\npm\\util@0.10.3.js' }
Error: EMFILE: too many open files, open 'C:\Users\bkdevconstructiv\tmp\NG6-starter\client\jspm_packages\npm\util@0.10.3.js'
    at Error (native)

[fabioelia]

Take a look @ #115
JSPM noCache resolved the EMFILE issue

[balazsmeszegeto]

Updating karma (old version: 0.12.36, new version: 0.13.19) fixes the EMFILE issue, at least for me.

Reason, I think, because the newer version uses graceful-fs@4.1.3, instead of node-fs. graceful-fs uses a resource-pool, thus only allows to open a limited amount of files (file openings are being queued)

As a side note, I think that karma's preprocessor should act more clever. It should not send unlimited files at the same time to the preprocessor, but rather implement a smart solution. Ie. define a reasonable limit for parallel running.

[AurelienRibon]

Issue with jspm_packages fixed in pull request 126 #126

[mribichich]

Same thing here, I'm in Windows 10, with karma 0.12.x I had the EMFILE issue, and after updating to 0.13.19, it was solved

[bjanderson]

I'm having the same issue. I've read that karma-jspm@2.0.3 supposedly fixes this, or at least allows you to set noPackagesCache:true in karma.config, which doesn't work for me. I also see that the latest version is karma-jspm@2.1.0, but neither of these are available in npm. The latest valid install target is 2.0.2. How long does it usually take for npm to sync up with github?

I'm having the same problem with a project at work, so I created a project to try and figure it out.
It's basically a bare-bones foundation project that started failing as soon as I jspm installed google's material-design-icons. *Note to future: that is a work in progress, so hopefully it will work when this issue gets fixed.

For the time being, I've had to stop using karma and just run mocha straight from npm (project). But I would much rather use karma if at all possible.

[autoZhangHui]

@balazsmeszegeto
Updating karma (old version: 0.12, new version: 0.13.22) and fixes the EMFILE issue.
Thx for your advice!

[nguillaumin]

It's still happening for us with Karma 0.13.22 and karma-jspm 2.1.0 (observed on Mac & Linux)

[eyalellenbogen]

same here. windows 10. karma 0.13.22 running through gulp. Only when trying 5 karma instances simultaneously. 4 simultaneous instances pass tests fine.

[ldunca]

I have the same issue :( karma 0.13.22 and karma-jspm 2.1.0

[0Ky]

Windows Server 2012 R2 ~ Same issue. Someone please find a solution for this error.

events.js:154
      throw er; // Unhandled 'error' event
      ^
Error: EMFILE: too many open files, open 'C:\projectc1\fswrite.html'
    at Error (native)

Thanks.

[mattfricker]

I was having the same issue on Ubuntu 14.04. Tried the recommended methods above with no luck.

What ended up fixing the issue for me was deleting my jspm_packages folder and then running jspm install again. After this, my tests now run without the error.

[svi3c]

I've got the same issue. My stack trace:

[project-root]/node_modules/karma/node_modules/glob/sync.js:328
        throw er
        ^

Error: EMFILE: too many open files, scandir '[project-root]/src/app/todos'
    at Error (native)
    at Object.fs.readdirSync (fs.js:856:18)
    at GlobSync._readdir ([project-root]/node_modules/karma/node_modules/glob/sync.js:275:41)
    at GlobSync._processReaddir ([project-root]/node_modules/karma/node_modules/glob/sync.js:135:22)
    at GlobSync._process ([project-root]/node_modules/karma/node_modules/glob/sync.js:130:10)
    at new GlobSync ([project-root]/node_modules/karma/node_modules/glob/sync.js:46:10)
    at new Glob ([project-root]/node_modules/karma/node_modules/glob/glob.js:124:12)
    at [project-root]/node_modules/karma/lib/file-list.js:161:14
    at Array.map (native)
    at List._refresh ([project-root]/node_modules/karma/lib/file-list.js:153:37)
    at List.refresh ([project-root]/node_modules/karma/lib/file-list.js:252:27)
    at Server._start ([project-root]/node_modules/karma/lib/server.js:178:12)
    at invoke ([project-root]/node_modules/di/lib/injector.js:75:15)
    at Server.start ([project-root]/node_modules/karma/lib/server.js:102:18)
    at Object.<anonymous> (/home/sven/.IntelliJIdea2016.1/config/plugins/js-karma/js_reporter/karma-intellij/lib/intellijServer.js:12:13)
    at Module._compile (module.js:413:34)

What I found out:
When I move this line to the bottom of the function and replace unshift() by push(), it works for me.
So instead of having

files === [ '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/system.src.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/system-polyfills.src.js',
  '/home/sven/dev/projects/angular2-setup-poc/node_modules/karma-jspm/src/adapter.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm.test.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm.config.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/**/*',
  '/home/sven/dev/projects/angular2-setup-poc/src/app/todos/*.spec.ts',
  '/home/sven/dev/projects/angular2-setup-poc/src/**/*.ts' ]

I get

files === [ '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/system.src.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/system-polyfills.src.js',
  '/home/sven/dev/projects/angular2-setup-poc/node_modules/karma-jspm/src/adapter.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm.test.js',
  '/home/sven/dev/projects/angular2-setup-poc/jspm.config.js',
  '/home/sven/dev/projects/angular2-setup-poc/src/app/todos/*.spec.ts',
  '/home/sven/dev/projects/angular2-setup-poc/src/**/*.ts',
  '/home/sven/dev/projects/angular2-setup-poc/jspm_packages/**/*' ]

But I have no clue how this could provoke the error.
Note: As you can see, no graceful-fs is used. glob is used and this uses node-fs.
My relevant deps in the package.json:

{
  "jspm": "^0.17.0-beta.13",
  "karma": "^0.13.22",
  "karma-jspm": "2.1.0"
}

[wuliupo]

Thanks @svi3c !
After update the karma to new version, it is solved.
Run below command under your project folder (or as global, -g)

npm install karma@0.13.22
npm install karma-jspm@2.1.0
npm install jspm@0.17.0-beta.13

[cAhmad]

Thanks @svi3c you fixed the too many open files for me!

karma@0.13.22
karma-jspm@2.1.1
jspm@0.16.2

[maxwellpeterson-wf]

Fixed in #148

[akulaggarwal]

thank you @wuliupo and @svi3c. was happening for me on my mac