Error: EMFILE: too many open files (ulimit does not work on Windows 10) #2090

8ctopotamus · 2016-08-11T17:27:36Z

In the Hexo troubleshooting guide they say to try

$ ulimit -n 10000

...but this does not work on Windows.

Environment Info

Node version

v5.6.0

My site _config.yml

# Hexo Configuration
# Hexo Configuration
## Docs: https://hexo.io/docs/configuration.html
## Source: https://github.com/hexojs/hexo/

# Site
title: Cool New Site
subtitle:
description:
author:  8ctopotamus
language:
timezone:

# URL
## If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
url: http://yoursite.com
root: /
permalink: :year/:month/:day/:title/
permalink_defaults:

# Directory
source_dir: source
public_dir: public
tag_dir: tags
archive_dir: archives
category_dir: categories
code_dir: downloads/code
i18n_dir: :lang
skip_render:

# Writing
new_post_name: :title.md # File name of new posts
default_layout: post
titlecase: false # Transform title into titlecase
external_link: true # Open external links in new tab
filename_case: 0
render_drafts: false
post_asset_folder: false
relative_link: false
future: true
highlight:
  enable: true
  line_number: true
  auto_detect: false
  tab_replace:

# Category & Tag
default_category: uncategorized
category_map:
tag_map:

# Date / Time format
## Hexo uses Moment.js to parse and display date
## You can customize the date format as defined in
## http://momentjs.com/docs/#/displaying/format/
date_format: YYYY-MM-DD
time_format: HH:mm:ss

# Pagination
## Set per_page to 0 to disable pagination
per_page: 10
pagination_dir: page

# Extensions
## Plugins: https://hexo.io/plugins/
## Themes: https://hexo.io/themes/
theme: trio-theme

# Deployment
## Docs: https://hexo.io/docs/deployment.html
deploy:
  type:

My theme _config.yml

# Hexo Configuration
# Header
menu:
  VISION: /vision
  DESIGN: /design
  EXECUTION: /execution
  REALIZATION: /realization

rss: /atom.xml

# Content
excerpt_link: Read More

# Sidebar
sidebar: right
widgets:
- category
- tag
- tagcloud
- archive
- recent_posts

# display widgets at the bottom of index pages (pagination == 2)
index_widgets:
- category
- tagcloud
- archive

# widget behavior
archive_type: 'monthly'
show_count: false

# Miscellaneous
google_analytics:
favicon: /favicon.png
twitter:
google_plus:
fb_admins:
fb_app_id:

Plugin version(npm ls --depth 0)

hexo-vanilla-site@0.0.0 ~/hexo-vanilla-site
├── hexo@3.1.1
├── hexo-generator-archive@0.1.4
├── hexo-generator-category@0.1.3
├── hexo-generator-index@0.2.0
├── hexo-generator-tag@0.2.0
├── hexo-renderer-ejs@0.1.1
├── hexo-renderer-marked@0.2.10
├── hexo-renderer-stylus@0.3.1
└── hexo-server@0.1.3

For BUG

I am getting the this error almost every time I run "hexo serve". I am on WIndows 10 if that makes a difference.

fs.js:584
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open 'C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.Module._extensions..js (module.js:421:20)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at readdir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\readdirp.js:55:25)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:82:15)

log with hexo --debug
This is what happens when I do "hexo serve --debug"

λ hexo serve --debug
17:26:02.458 DEBUG Hexo version: 3.2.2
17:26:02.464 DEBUG Working directory: ~\Documents\GitHub\trio-hexo\
17:26:02.587 DEBUG Config loaded: ~\Documents\GitHub\trio-hexo\_config.yml
17:26:02.605 DEBUG Plugin loaded: hexo-generator-category
17:26:02.609 DEBUG Plugin loaded: hexo-generator-index
17:26:02.614 DEBUG Plugin loaded: hexo-generator-archive
17:26:02.620 DEBUG Plugin loaded: hexo-renderer-ejs
17:26:02.625 DEBUG Plugin loaded: hexo-renderer-stylus
17:26:02.627 DEBUG Plugin loaded: hexo-generator-tag
17:26:02.689 DEBUG Plugin loaded: hexo-server
17:26:02.696 DEBUG Plugin loaded: hexo-renderer-marked
17:26:02.710 DEBUG Loading database.
17:26:02.937 INFO  Start processing
17:26:03.721 DEBUG Processed: Design/index.md
17:26:03.722 DEBUG Processed: Execution/index.md
17:26:03.722 DEBUG Processed: Realization/index.md
17:26:03.723 DEBUG Processed: _posts/hello-world.md
17:26:03.723 DEBUG Processed: _posts/second-post.md
17:26:03.724 DEBUG Processed: vision/index.md
fs.js:584
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open 'C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.Module._extensions..js (module.js:421:20)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at readdir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\readdirp\readdirp.js:55:25)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\Users\joshn\Documents\GitHub\trio-hexo\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:82:15)

The text was updated successfully, but these errors were encountered:

yangg · 2016-09-06T02:26:20Z

Same error, anyone can fix it?

8ctopotamus · 2016-09-07T16:35:59Z

Hey @yangg, I just keep running hexo s over and over and eventually it starts up again.

Obviously this is not ideal as the issue still persists. Would be nice if someone could suggest a solution!

yangg · 2016-09-08T01:03:57Z

@8ctopotamus I'm running hexo on Windows 7 (this PC has a bad performance)
hexo g has the same problem.
As the theme use hexo-renderer-scss instead of gulp recently, I just removed the dir theme/even/node_modules, and it works now.

jcimoch · 2016-09-26T16:31:53Z

I have same problem under windows bash which I described here microsoft/WSL#1126 Is there any workaround or fix?

felixbillon · 2016-10-16T15:56:18Z

Same problem on Windows 10 :(

8ctopotamus · 2016-10-17T16:06:48Z

Any update on this? I still have to run hexo s repeatedly until it finally starts up...

pimskie · 2016-11-16T19:34:48Z

Same here. Tried ulimit -u 3200 without result. Server won't start until the error goes away.

I fixed it by removing my own node_modules folder from my themes folder (hexo/themes/[my theme name]/node_modules).
#2179

lucax88x · 2016-12-21T19:31:39Z

updates? this is really annoying..

ghost · 2016-12-21T21:25:35Z

This happens because hexo is watching ALL files in sub folders. If you're using grunt, gulp in your theme there are thousands of files and hexo doesn't ignore then.

The only workaround is currently to keep your node_modules outside the hexo root and create a gulp task which copy the files to theme folder.

lucax88x · 2016-12-22T09:41:49Z

Agree with you. This was the first thing I did when I saw this error.

Anyway. I couldn't make my theme work without node_modules because it required some files.

Probably there's an issue with the theme itself (tranquilpeak) but shouldn't hexo avoid to check folders like node_modules, bower_components, etc? maybe making this configurable.

seaoak · 2016-12-23T08:58:47Z

I seem to find out how to exclude some directories from the targets of watch().
But I have no Windows environment.
(In my Linux (Ubuntu 16.04.1) environment, no error occurs)

Could you please try following patch?

If you use Hexo "v3.1.1", modify node_modules/hexo/lib/box/index.js:24

$ diff index.js.org index.js
24c24
<     ignored: /[\/\\]\./
---
>     ignored: /[\/\\](\.[^\/\\]+|node_modules|bower_components)([\/\\]|[\/\\]?$)/
$

If you use Hexo "v3.2.2", insert a line into node_modules/hexo/lib/box/index.js:24

$ diff -c index.js.org index.js
*** index.js.org        2016-12-23 17:02:13.315065623 +0900
--- index.js    2016-12-23 17:07:11.864219252 +0900
***************
*** 21,26 ****
--- 21,27 ----
    EventEmitter.call(this);

    this.options = _.assign({
+     ignored: [base + 'node_modules', base + 'bower_components', base + '.git'],
      persistent: true
    }, options);

$

Please check node_modules/hexo/package.json to know the version.

Of course, I will make targets configurable when I make a pull request.

Thank you in advance for your cooperation.

lucax88x · 2016-12-27T17:07:18Z

Sure!

I have hexo 3.2.0

I applied the .2 patch, but unfortunately, it still not works.

Error: EMFILE: too many open files, open 'C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\readdirp\stream-api.js'
    at Error (native)
    at Object.fs.openSync (fs.js:640:18)
    at Object.fs.readFileSync (fs.js:508:33)
    at Object.Module._extensions..js (module.js:549:20)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)
    at Function.Module._load (module.js:409:3)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at readdir (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\readdirp\readdirp.js:56:25)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:355:5)
    at FSWatcher.NodeFsHandler._handleDir (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:406:18)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:455:19)
    at FSWatcher.<anonymous> (C:\git\lucax88x.github.ioold\lucax88x.github.io\node_modules\chokidar\lib\nodefs-handler.js:460:16)
    at FSReqWrap.oncomplete (fs.js:123:15)

seaoak · 2016-12-28T07:19:23Z

Could you taste my new patch?

The patch which I wrote in above comment is not enough.
It makes Hexo turning-off only watch() for node_modules.
Hexo continuously calls readdir() for all subdirectories in node_modules.

I make new patch to turn-off readdir()/watch() for specified directories.
Under the patch, Hexo will ignore node_modules directory and .git directory and all their subdirectories.
Hexo will NEVER call either readdir() nor watch() for these directories.

And also you can configure which directores are ignored.
You can write ignored_path: setting in _config.yml (such as after skip_render: setting) as follows:

ignored_path:
 - node_modules
 - .git
 - bower_components
 - /[\/\\](\.git[^\/\\]*)[\/\\]?$/i

At this time, you can not use glob expressions,
but you can use regular expressions instead.
I'm just getting ahead in myself.
I will reverse them (suit to skip_render: setting).

My repository is:
https://github.com/seaoak/hexo.git

The branch is:
feature/allow-turning-off-reading-and-watching-for-specified-directories

Please checkout the HEAD of above branch.

p.s. When you execute hexo, you will see many debugging messages.
You can filter them by grep -v 'seaoak '.
I'm sorry.

seaoak · 2016-12-28T13:12:35Z

In addition, if you are not familiar with git command,
please follow the procedure below:

Download index.js as node_modules/hexo/lib/box/index.js
Download default_config.js as node_modules/hexo/lib/hexo/default_config.js

These links point my commit 743ced890fe29d5bb6be91ba6efa7add1b289ea0
seaoak@743ced8

For example,

$ cd hexo-blog
$ hexo clean
$ (cd node_modules/hexo/lib/box && cp -p index.js index.js.org)
$ curl https://raw.githubusercontent.com/seaoak/hexo/743ced890fe29d5bb6be91ba6efa7add1b289ea0/lib/box/index.js > node_modules/hexo/lib/box/index.js
$ (cd node_modules/hexo/lib/hexo && cp -p default_config.js default_config.js.org)
$ curl https://raw.githubusercontent.com/seaoak/hexo/743ced890fe29d5bb6be91ba6efa7add1b289ea0/lib/hexo/default_config.js > node_modules/hexo/lib/hexo/default_config.js
$ hexo server --debug

And also you can recover these files:

$ hexo clean
$ (cd node_modules/hexo/lib/box && mv index.js.org index.js)
$ (cd node_modules/hexo/lib/hexo && mv default_config.js.org default_config.js)
$ hexo server --debug

lucax88x · 2016-12-28T13:39:16Z

I'm ok with git ;)

Anyway, with the 2 files patched, seems to be working, and hexo serves also much much faster this way.

ghost · 2017-04-22T09:27:51Z

@seaoak I would love to see this solved in official hexo, can you try to make a pull request?

seaoak · 2017-04-27T01:20:31Z

Thanks for your request!

Before I make a pull request, I should make test cases for the patch.
But I found that existing test script for watch() function was not enough.
So I'm trying to improve it.
https://github.com/hexojs/hexo/blob/master/test/scripts/hexo/hexo.js

I'm sorry for the delay, but I'll make a pull request. 😉

NoahDragon · 2017-04-27T14:36:21Z

@seaoak thanks for your active contribution. I have invited you to join Hexo as a collaborator. 😃

NoahDragon · 2017-09-27T16:49:24Z

@seaoak Sorry for my bad memory, did this issue fix?

erikman · 2019-04-10T21:09:12Z

Probably solved by #1316, documentation issue still open #2855

stevenjoezhang · 2020-03-10T13:57:11Z

Closing as resolved: #3797

8ctopotamus changed the title ~~Error: EMFILE: too many open files (Windows 10)~~ Error: EMFILE: too many open files (ulimit does not work on Windows 10) Aug 16, 2016

NoahDragon mentioned this issue Dec 23, 2016

Hexo for multiple sites #2320

Closed

stale bot added the wontfix This will not be worked on label Sep 27, 2017

NoahDragon removed the wontfix This will not be worked on label Sep 27, 2017

hexojs deleted a comment from stale bot Sep 27, 2017

NoahDragon added the bug Something isn't working label Sep 27, 2017

SukkaW mentioned this issue Sep 21, 2019

Changing 302 redirects to 301 redirects hexojs/hexo-server#99

Closed

seaoak mentioned this issue Oct 24, 2019

Disable fs.watch() for dirs/files specified by "ignore" option #3797

Merged

2 tasks

stevenjoezhang closed this as completed Mar 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: EMFILE: too many open files (ulimit does not work on Windows 10) #2090

Error: EMFILE: too many open files (ulimit does not work on Windows 10) #2090

8ctopotamus commented Aug 11, 2016 •

edited

Loading

yangg commented Sep 6, 2016

8ctopotamus commented Sep 7, 2016

yangg commented Sep 8, 2016 •

edited

Loading

jcimoch commented Sep 26, 2016

felixbillon commented Oct 16, 2016

8ctopotamus commented Oct 17, 2016

pimskie commented Nov 16, 2016 •

edited

Loading

lucax88x commented Dec 21, 2016

ghost commented Dec 21, 2016

lucax88x commented Dec 22, 2016

seaoak commented Dec 23, 2016

lucax88x commented Dec 27, 2016

seaoak commented Dec 28, 2016 •

edited

Loading

seaoak commented Dec 28, 2016

lucax88x commented Dec 28, 2016

ghost commented Apr 22, 2017

seaoak commented Apr 27, 2017

NoahDragon commented Apr 27, 2017

NoahDragon commented Sep 27, 2017 •

edited

Loading

erikman commented Apr 10, 2019

stevenjoezhang commented Mar 10, 2020

Error: EMFILE: too many open files (ulimit does not work on Windows 10) #2090

Error: EMFILE: too many open files (ulimit does not work on Windows 10) #2090

Comments

8ctopotamus commented Aug 11, 2016 • edited Loading

Environment Info

For BUG

yangg commented Sep 6, 2016

8ctopotamus commented Sep 7, 2016

yangg commented Sep 8, 2016 • edited Loading

jcimoch commented Sep 26, 2016

felixbillon commented Oct 16, 2016

8ctopotamus commented Oct 17, 2016

pimskie commented Nov 16, 2016 • edited Loading

lucax88x commented Dec 21, 2016

ghost commented Dec 21, 2016

lucax88x commented Dec 22, 2016

seaoak commented Dec 23, 2016

lucax88x commented Dec 27, 2016

seaoak commented Dec 28, 2016 • edited Loading

seaoak commented Dec 28, 2016

lucax88x commented Dec 28, 2016

ghost commented Apr 22, 2017

seaoak commented Apr 27, 2017

NoahDragon commented Apr 27, 2017

NoahDragon commented Sep 27, 2017 • edited Loading

erikman commented Apr 10, 2019

stevenjoezhang commented Mar 10, 2020

8ctopotamus commented Aug 11, 2016 •

edited

Loading

yangg commented Sep 8, 2016 •

edited

Loading

pimskie commented Nov 16, 2016 •

edited

Loading

seaoak commented Dec 28, 2016 •

edited

Loading

NoahDragon commented Sep 27, 2017 •

edited

Loading