Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: close image files after use during palette check #68

Open
wants to merge 1 commit into
base: master
from

Conversation

@andythenorth
Copy link
Contributor

andythenorth commented Dec 1, 2019

With pypy, nmlc trivially trips file handle limits on macOS during palette checks, resulting in: "[Errno 24] Too many open files".

This is a common enough problem in python, which can be resolved by either

  • end user setting ulimit -n 4096 on a per shell basis
  • closing PIL Image files with Image.close(), where Image.open() has been used

This PR closes images opened for palette checks, which is a trivial case. Errno 24 no longer triggers for me. Tests pass.

Errno 24 does not trigger for me with any cPython 3.x, but pypy cuts run times by up to 50% in my testing compared to cPython 3.8, so we do want to be compatible with pypy.

There are a couple of other uses of Image.open() but they looked less trivial, and they're not tripping Errno 24 for me.

For the record, Image.load() self-closes so wouldn't have the same issue, but Image.open() is often the wanted command, especially for reading metadata.

…ring file handle limits
@nielsmh

This comment has been minimized.

Copy link
Contributor

nielsmh commented Dec 1, 2019

I'm wondering if using a with block (context manager) isn't the more correct thing here? Assuming the Image class supports it, but I'd expect it to.

@andythenorth

This comment has been minimized.

@planetmaker

This comment has been minimized.

Copy link
Contributor

planetmaker commented Dec 16, 2019

I agree with fsj: the closing should be done by means of using the with contextmanager
In good script kiddie manner, here's some examples:
https://stackoverflow.com/questions/713794/catching-an-exception-while-using-a-python-with-statement

Copy link
Member

LordAro left a comment

To reflect above comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.