Can't raise "open files" ulimit

[aseering]

Please use the following bug reporting template to help produce actionable and reproducible issues. Please try to ensure that the reproduction is minimal so that the team can go through more bugs!

• A brief description

The "open files" ulimit can't currently be increased.

I know of two use cases for setting this ulimit:

• NodeJS (and therefore lots of common NodeJS tools -- npm, grunt, etc) has a common design idiom of "process all of these files", whose default implementation is to open all of the files asynchronously and process them as the OS gets around to reading them. "All of the files" can be quite a lot :-) So NodeJS users often have to either bump this limit, or modify their own tools to use graceful-js. Modifying existing tools can be hard, so simply bumping the limit is sometimes the preferred workaround. (I think this is the real issue behind Defult ulimit way too low #1679 )

• Some databases (I personally know that Vertica does this; I think I recall hearing that MySQL can do the same?) can, in certain situations, open very large numbers of file descriptors. The objective is typically to have some clever effect on query performance. Decreasing the number of available file descriptors sacrifices query performance and/or causes errors.

• Expected results

$ ulimit -n 65536
$

And the actual limit is updated

• Actual results (with terminal output if applicable)

$ ulimit -n 65536
bash: ulimit: open files: cannot modify limit: Operation not permitted

• Your Windows build number

15031

• Steps / All commands required to reproduce the error from a brand new installation

See above

• Strace of the failing command

If you really want an strace of bash -c 'ulimit -n 65536', I can post one. It's going to be long, though, and I bet y'all already know what the important part is :-)

• Required packages and commands to install

None -- bash builtin.

See our contributing instructions for assistance.

[benhillis]

Addling links to related work items: #1679 #1682 #1686

[stehufntdev]

There was an accidental regression where the RLIMIT_NOFILE checks were too strict, and that was fixed in early February. The fix should be out for insider builds soon though the rlimits are still not actually applied.

[sklivvz]

Bumping this, it makes Ubuntu on windows unusable for professional developer use with node and as a consequence with pretty much anything that uses node to comple large numbers of assets.

Can we get a little attention, or at least have this scheduled?

Thanks.

[atomicthumbs]

Any chance this'll be fixed soon?

[benhillis]

@atomicthumbs - taking a look at this now.

[benhillis]

Support was added in 16257.

[leixuezhang]

@benhillis I have updated to the build of 16257, the "ulimit -n" can be changed. However, as an atmospheric scientist, i want to know that if the "ulimit -s" to be changed to unlimited, as we want to use the wsl on the workstation. Now the stack size is static 8192K, too limited for modeling.

[benhillis]

@leixuezhang - Currently changing the stack size is not supported, this is also requested in #633, and on User Voice.

[michaelklishin]

Ubuntu (well, Linux kernel) defaults were fantastic for those running Gnome on their desktop in early 2000s. Many developers, regardless of the tools they use, have to bump the limit significantly (say, at least 20K) at some point. The number of threads on the Web about ulimit, PAM limit management and such is enormous and that's easy to verify.

Please consider increasing the default to 20K: the downside is a tiny increase in the amount of RAM used, absolutely negligible in the year of 2017.

[therealkenc]

Support was added in 16257.

#633 for more.

[sunilvirus]

I get only one chance to increase the limit ?
When I try to increase limit first time, it allows any value (upto 65536).
But if I decrease the limit to a smaller value, it cannot increase it to anything above it.
Is this expected behaviour ? I am using build 17134.

$ ulimit -n
1024
$ ulimit -n 65536
$ ulimit -n 65535
$ ulimit -n 65536
bash: ulimit: open files: cannot modify limit: Invalid argument
$ ulimit -n 10000
$ ulimit -n 10001
bash: ulimit: open files: cannot modify limit: Invalid argument
$ ulimit -n 7000
$ ulimit -n 7001
bash: ulimit: open files: cannot modify limit: Invalid argument

[vingrad]

This issue is stil exists even with WSL 2

[therealkenc]

This issue is stil exists even with WSL 2

This issue does not (ulimit works fine). But you are probably hitting variation this. We got no PAM.

[vingrad]

I tried of course "session required pam_limits.so", but it did not help. This issue is still exists.

[BerserkerGaruk]

I was running into this error with WSL2 also. After changing everything outlined through this issue the limits for ulimit wouldn't be raised and wouldn't let me modify the values past the limits, but when checking the rights for the user I was running my command as: sudo su username --shell /bin/bash --command "ulimit -n" The ulimit would return the raised value. I then logged into myself from myself su username --shell /bin/bash and I was able to raise the ulimit values as expected, but only for that session. This makes me think that the initial WSL shell is restricted in some way.

[erichwalls]

I too am having this issue with ulimit in WSL 2... and vm.max_map_count isn't available in WSL 1, so it makes running Elasticsearch pretty much impossible in ANY WSL distro :(
Wish someone from MS would address this... maybe give us an answer here.

[19317362]

Today, with the lates WSL2 - ubuntu, I can not change ulimit -n for common user.
I can change the value for root.
How to deal with this?

[martincox]

Same issue here in WSL2. Trying to change nofile limit in /etc/security/limits.conf, but it does not take effect.

[kf6nux]

su username

I edited /etc/security/limits.conf and then used the above. Great workaround. Thanks @BerserkerGaruk !

[robinvanemden]

su username

I edited /etc/security/limits.conf and then used the above. Great workaround. Thanks @BerserkerGaruk !

@BerserkerGaruk's workaround does indeed resolve this!

[cfjedimaster]

I'm running into this now ad @BerserkerGaruk workaround doesn't work for me. Any suggestions?

[jacquinot-eric]

you can add at the end of your .profile:

exec sudo su user --shell /bin/bash (replace user with your own user account)

don't forget to add user to sudo with no password using visudo by adding at the end:

user ALL=(ALL) NOPASSWD:ALL

you will also need to edit /etc/security/limits/conf and add:

user soft nproc 16384
user hard nproc 16384
user soft nofile 16384
user hard nofile 16384

then after you start wsl you should get:

user@windows10-g56jr:~$ ulimit -n
16384

[JohnSully]

It's crazy that this is somehow marked closed and fixed. The sudo su workaround works for me, but that really shouldn't be necessary. Needing to disable sudo passwords is a security risk.

[pecigonzalo]

It actually does not even work, as doing sudo su - username removes all the WSL_ vars.

[rbabayoff]

Workaround not working for me on wsl 2. @benhillis can you please reopen this? Makes using wsl as an alternative to osx impossible for node developers. Downgrading to wsl 1 in the meantime.

[FFdhorkin]

The tips in this issue didn't work for me with WSL2, but I found a potential workaround here: #1576 (comment)

Basically:

sudo prlimit -p "$$" --nofile=10000:10000
exec zsh
ulimit -n 10000

More explanation on the link

[avoidik]

another option is sudo sh -c "ulimit -n 65535 && exec su $LOGNAME"

[pecigonzalo]

@benhillis @therealkenc could you re-open this?

[drogus]

I'm running into the same issue on WSL2 with Ubuntu. The workaround from @FFdhorkin worked, but it's still annoying, because after using exec zsh some commands no longer work (like for example VS Code integration, I can't run code . then)

[rbabayoff]

I ended up making root the default user for wsl and everything works.

[andrii0lomakin]

This issue is still present on WSL 2, could you reopen it?

[therealkenc]

#4575. Discussion on ulimit usage can continue over on discussions.