New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't raise "open files" ulimit #1688

Closed
aseering opened this Issue Feb 10, 2017 · 10 comments

Comments

Projects
None yet
9 participants
@aseering
Contributor

aseering commented Feb 10, 2017

Please use the following bug reporting template to help produce actionable and reproducible issues. Please try to ensure that the reproduction is minimal so that the team can go through more bugs!

  • A brief description

The "open files" ulimit can't currently be increased.

I know of two use cases for setting this ulimit:

  • NodeJS (and therefore lots of common NodeJS tools -- npm, grunt, etc) has a common design idiom of "process all of these files", whose default implementation is to open all of the files asynchronously and process them as the OS gets around to reading them. "All of the files" can be quite a lot :-) So NodeJS users often have to either bump this limit, or modify their own tools to use graceful-js. Modifying existing tools can be hard, so simply bumping the limit is sometimes the preferred workaround. (I think this is the real issue behind #1679 )

  • Some databases (I personally know that Vertica does this; I think I recall hearing that MySQL can do the same?) can, in certain situations, open very large numbers of file descriptors. The objective is typically to have some clever effect on query performance. Decreasing the number of available file descriptors sacrifices query performance and/or causes errors.

  • Expected results

$ ulimit -n 65536
$

And the actual limit is updated

  • Actual results (with terminal output if applicable)
$ ulimit -n 65536
bash: ulimit: open files: cannot modify limit: Operation not permitted
  • Your Windows build number

15031

  • Steps / All commands required to reproduce the error from a brand new installation

See above

  • Strace of the failing command

If you really want an strace of bash -c 'ulimit -n 65536', I can post one. It's going to be long, though, and I bet y'all already know what the important part is :-)

  • Required packages and commands to install

None -- bash builtin.

See our contributing instructions for assistance.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Feb 10, 2017

Member

Addling links to related work items: #1679 #1682 #1686

Member

benhillis commented Feb 10, 2017

Addling links to related work items: #1679 #1682 #1686

@stehufntdev

This comment has been minimized.

Show comment
Hide comment
@stehufntdev

stehufntdev Feb 16, 2017

Collaborator

There was an accidental regression where the RLIMIT_NOFILE checks were too strict, and that was fixed in early February. The fix should be out for insider builds soon though the rlimits are still not actually applied.

Collaborator

stehufntdev commented Feb 16, 2017

There was an accidental regression where the RLIMIT_NOFILE checks were too strict, and that was fixed in early February. The fix should be out for insider builds soon though the rlimits are still not actually applied.

@sklivvz

This comment has been minimized.

Show comment
Hide comment
@sklivvz

sklivvz May 12, 2017

Bumping this, it makes Ubuntu on windows unusable for professional developer use with node and as a consequence with pretty much anything that uses node to comple large numbers of assets.

Can we get a little attention, or at least have this scheduled?

Thanks.

sklivvz commented May 12, 2017

Bumping this, it makes Ubuntu on windows unusable for professional developer use with node and as a consequence with pretty much anything that uses node to comple large numbers of assets.

Can we get a little attention, or at least have this scheduled?

Thanks.

@atomicthumbs

This comment has been minimized.

Show comment
Hide comment
@atomicthumbs

atomicthumbs May 29, 2017

Any chance this'll be fixed soon?

atomicthumbs commented May 29, 2017

Any chance this'll be fixed soon?

@jackchammons jackchammons added the bug label May 31, 2017

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Jul 11, 2017

Member

@atomicthumbs - taking a look at this now.

Member

benhillis commented Jul 11, 2017

@atomicthumbs - taking a look at this now.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Aug 3, 2017

Member

Support was added in 16257.

Member

benhillis commented Aug 3, 2017

Support was added in 16257.

@benhillis benhillis removed their assignment Aug 3, 2017

@leixuezhang

This comment has been minimized.

Show comment
Hide comment
@leixuezhang

leixuezhang Aug 8, 2017

@benhillis I have updated to the build of 16257, the "ulimit -n" can be changed. However, as an atmospheric scientist, i want to know that if the "ulimit -s" to be changed to unlimited, as we want to use the wsl on the workstation. Now the stack size is static 8192K, too limited for modeling.

leixuezhang commented Aug 8, 2017

@benhillis I have updated to the build of 16257, the "ulimit -n" can be changed. However, as an atmospheric scientist, i want to know that if the "ulimit -s" to be changed to unlimited, as we want to use the wsl on the workstation. Now the stack size is static 8192K, too limited for modeling.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Aug 8, 2017

Member

@leixuezhang - Currently changing the stack size is not supported, this is also requested in #633, and on User Voice.

Member

benhillis commented Aug 8, 2017

@leixuezhang - Currently changing the stack size is not supported, this is also requested in #633, and on User Voice.

@michaelklishin

This comment has been minimized.

Show comment
Hide comment
@michaelklishin

michaelklishin Oct 9, 2017

Ubuntu (well, Linux kernel) defaults were fantastic for those running Gnome on their desktop in early 2000s. Many developers, regardless of the tools they use, have to bump the limit significantly (say, at least 20K) at some point. The number of threads on the Web about ulimit, PAM limit management and such is enormous and that's easy to verify.

Please consider increasing the default to 20K: the downside is a tiny increase in the amount of RAM used, absolutely negligible in the year of 2017.

michaelklishin commented Oct 9, 2017

Ubuntu (well, Linux kernel) defaults were fantastic for those running Gnome on their desktop in early 2000s. Many developers, regardless of the tools they use, have to bump the limit significantly (say, at least 20K) at some point. The number of threads on the Web about ulimit, PAM limit management and such is enormous and that's easy to verify.

Please consider increasing the default to 20K: the downside is a tiny increase in the amount of RAM used, absolutely negligible in the year of 2017.

@therealkenc

This comment has been minimized.

Show comment
Hide comment
@therealkenc

therealkenc Mar 26, 2018

Collaborator

Support was added in 16257.

#633 for more.

Collaborator

therealkenc commented Mar 26, 2018

Support was added in 16257.

#633 for more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment