MacOS: Too many open files

[bryphe]

esy version: 0.6.2
Operating System: MacOS

Issue:

Intermittent error:
error: Too many open files

when running esy install.

Package.json
https://github.com/onivim/oni2/blob/master/package.json

I see this error intermittently when using esy install

Workaround:

As a workaround, I run ulimit -Sn 4096, which increases the number of file descriptors available for the shell session

Related:
onivim/oni2#1276

[anmonteiro]

you just need to set the number of max open file descriptors to a reasonable value. macOS's default is nonsensical. e.g. ulimit -n 4096

I literally only read the first part of the issue...

[jicksta]

I get this when simply cloning the revery repo and doing esy install on macOS Catalina 10.15.1.

❯ esy install
info install 0.6.4 (using package.json)
info fetching: done
error: Too many open files
  unpacking esy-libjpeg-turbo@github:revery-ui/libjpeg-turbo#dbb3dd5
  installing esy-libjpeg-turbo@github:revery-ui/libjpeg-turbo#dbb3dd5
esy: exiting due to errors above

If esy can't fix it the issue directly, it would be nice if esy could at least print a useful suggestion to change the open file descriptor limit with ulimit to get around this instead of folks Googling their way here to this thread.

[bryphe]

Would be worth seeing if the getrlimit or setrlimit APIs could be used programmatically: http://man7.org/linux/man-pages/man2/setrlimit.2.html (to either check the limit, or even set it so the user doesn't have to worry about this, on OSX)

[jfrolich]

Same issue here

[EduardoRFS]

As it is supported, perhaps esy could detect that and call setrlimit? It seems to be full supported on macOS

https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/setrlimit.2.html

[jicksta]

What's the reason that esy needs so many files open?

[EduardoRFS]

@jicksta most of time it's because there is a lot of things happening in parallel, esy could easily run it in serial, but 256 it will not only fails with esy, it's a better UX if we just increase it even if only for esy itself