Fatal error: spawn EMFILE

[farmersmc]

I am building a site using the Yeoman Beta 4 setup with the latest versions of everything.

After using grunt server for a while to build my project, and this happens at least once or more daily, I get an error that kills my session and forces me to restart.

Fatal error: spawn EMFILE

I've noticed the last two times this has happened that this error occurs right after it tries running the compass:server task.

Luckily, it doesn't appear to corrupt anything or mess with my files, but it's an annoying issue that continuously pops up and forces me to restart the server.

Does anyone else experience a similar problem?

[shama]

EMFILE means it is trying to open more file descriptors than your system allows. If you're on OSX, the default is 256 and really low. You can temporarily increase it on your session with ulimit -n 10480 (assuming your own OSX).

[farmersmc]

@shama I'm not sure what is meant by file descriptors. The concept confuses me. I am on OSX where this is happening.

So a couple of questions

1. If this opening file descriptors is really low by default on OSX, why aren't there more people experiencing this problem
2. When you say "temporarily increase it", how long does temporarily need to last?

From the way you're answering, it doesn't sound like there's a good solution for this? Thanks for helping me understand.

[sindresorhus]

@farmersmc the first thing you should do when encountering a problem is try googling the error. This has been answered many times before: https://www.google.no/search?q=Fatal+error%3A+spawn+EMFILE

[farmersmc]

@sindresorhus Of course, I did this and didn't see any relevant answers to my problem, only a couple which actually had patches submitted to fix the issue in NodeJS or something else. And because I didn't see anything relevant, I had no basis to understand what was going on. Which is why I asked for help here wondering if it wasn't just me.

[farmersmc]

To both @sindresorhus and @shama, I've looked around the web for help and answers to what my problem is but most answers are too vague for me to understand or doesn't really go into what's happening. What I'm reading is that having "too many files open" seems to be what most people are saying but what does this mean? In my code editor? And what does the number 256 have to do with the amount of open files? I'm sorry for bugging you but I really don't know how to evaluate and fix this problem. I appreciate your help.

[shama]

@farmersmc There are many ways a file descriptor can be opened, read this on what a file descriptor is: http://en.wikipedia.org/wiki/File_descriptor It is technically a security "feature" as to prevent any process from opening too many and crashing your system. For some reason that limit was set to 256 on OSX.

Most of the time EMFILE is avoided by simply putting files into a queue and waiting until more slots open up. Most of the libs used in grunt do this but not always for every way a file descriptor can be opened. "watching" files uses file descriptors as well but a queue unfortunately isn't a solution here... as the user expects all files to respond and not the slice from a queue. You can poll watched files, or at a timed interval read the files. This will avoid that error but polling is slower, especially with large amounts of files.

So the simple answer is we're doing what we can to avoid this error but the easiest solution is to simply increase your limit. I recommend temporarily (only for your open terminal session) by doing ulimit -n 10480. A slightly more permanent is to put that in your bashrc and have it run automatically for you each time you open your terminal. There are other ways to totally and permanently set the limit but given the diverse nature of system environments, I don't generally provide a recommendation as it may cause system issues for some.

The other solution is to try and open less file descriptors by making sure your matching patterns (**/*) are more restrictive... For example **/*.js instead of **/* or even more restrictive depending on the application.

[farmersmc]

@shama Thank you very much for your detailed explanation! It made lots of sense and your last paragraph triggered something that, I think, helped me find the solution to this problem.

In the Yeoman generated Gruntfile.js, I noticed that the Compass watch was set to see only Sass files on the root level of the /Styles/ CSS directory. I started using a beginners Compass project that had multiple directories with other Sass files in these different levels. When I was editing these files, I noticed that LiveReload wasn't doing anything.

What I thought would solve the problem was to change this:

grunt.initConfig({
    yeoman: yeomanConfig,
    watch: {
        compass: {
            files: [
                '<%= yeoman.app %>/styles/*.{scss,sass}'
            ]
        }
    }
});

to this:

grunt.initConfig({
    yeoman: yeomanConfig,
    watch: {
        compass: {
            files: [
                '<%= yeoman.app %>/styles/*.{scss,sass}',
                '<%= yeoman.app %>/styles/**/*.{scss,sass}',
                '<%= yeoman.app %>/styles/**/**/*.{scss,sass}',
                '<%= yeoman.app %>/styles/**/**/**/*.{scss,sass}'
            ]
        }
    }
});

So you can see each level of directories I was adding thinking that was the only way to work with files on that level of folders. What I found out after asking around is that I didn't need more than one level of /**/ in order to accomplish what I wanted. Thus, I changed it to this:

grunt.initConfig({
    yeoman: yeomanConfig,
    watch: {
        compass: {
            files: [
                '<%= yeoman.app %>/styles/**/*.s{c,a}ss'
            ]
        }
    }
});

What I also had no idea about was the fact that when I make changes to Gruntfile.js or other root level files, I must always restart grunt server in order for these changes to apply.

So, considering this solution, I have gone to the Yeoman project to suggest changing the default by updating the Compass watch section. Hopefully it'll be accepted and added to the project.

[mklement0]

Thought I'd mention that the recommended value of 10480 is not allowed on OS X 10.8.4 - the max. value appears to be ulimit -n 10240. Another caveat: once you've set a limit in a given session, raising that limit later is not permitted (though lowering it is).

[wmill]

@sindresorhus just so you know, this issue is now the top post when you google that error message. So your google link is self-referential.

[arcreative]

my ulimit is unlimited, so I don't think that's the issue for me... is there a limit in grunt or possibly compass? I'm also using compass compilation and it crashes for me every minute or two.

[rubiii]

@arcreative my ulimit was set to unlimited as well, but somehow this didn’t work.
setting the ulimit via ulimit -S -n 2048 fixed it for me.

[rubiii]

ok, my ulimit was not really set to unlimited. ulimit just printed unlimited.
try ulimit -a and look for file descriptors.

[arcreative]

Per one of the referencing issues, I found that http://unix.stackexchange.com/questions/108174/how-to-persist-ulimit-settings-in-osx-mavericks works perfectly (until i run two instances of grunt watch 😀).

[ceoaliongroo]

Sometimes that change on the limits, don't resolve the issue.

so, it's possible do a double check, about what files/folder need to be watched, maybe you can omit some folders, for example:
'!<%= yeoman.app %>/bower_components/linked_library/node_modules/**/*'

if use bower link in a big projects is common to have the EMFILE error.