Fix: bring the image size limit of PIL in line #373
Conversation
ahyangyi
force-pushed
the
fix-png-upload-size-limit
branch
from
fa3cfdd
to
dbe7ef7
Compare
TrueBrain
force-pushed
the
fix-png-upload-size-limit
branch
from
dbe7ef7
to
c539d57
Compare
|
Turns out, this need a bit more thinking. That is to say, this PR is better than the code was, but we hit another problem: a 16kx16k map consumes so much memory, that the backend is killed for overusing its memory allocation. Of course we can increase that allocation, but it requires 1+GB of RAM to process a single map. That is ... not within the realm of possibilities :D An additional issue is that processing even a 8kx8k map takes over 30 seconds .. So I need to give this a better look; mostly, I am wondering if we should actually use PIL, or if we can somehow stream the image, and not have everything in memory :) I will give this some more thought in the next few weeks .. basically: it is not your PR, but it does make it more visible :) |
| @@ -47,14 +51,11 @@ def read(self, fp): | |||
|
|
|||
| try: | |||
| im = Image.open(fp) | |||
| except Image.DecompressionBombError: | |||
| raise ValidationException("Image is too large.") | |||
There was a problem hiding this comment.
If the number of pixels is greater than twice [MAX_IMAGE_PIXELS](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.MAX_IMAGE_PIXELS), then a DecompressionBombError will be raised instead.
I was wondering why my 20kx20k map wasn't triggering this, but I have to go slightly bigger before it does so :) Seems we need both statements or something to make this work properly :)
|
After a lot of fiddling, decided to go the evil way, and came up with #391. Basically, this drops Pillow, and resolves all other issues we have with heightmaps (slow, memory hungry, etc). Nevertheless, tnx for this work, and I hope you feel up for picking up a few more BaNaNaS tickets :) This only was one .. that had more issues than was visible on the surface :) Nothing I even knew when we were talking about this. |
The intention of the current code is to limit heightmap to at most 16384 * 16384. However, the actual limit is smaller due to PIL having a similar protection mechanism.
This patch sets the PIL limit to match our intention.
Locally tested.