"TIFFAppendToStrip" when saving to BytesIO() stream

[jbarlow83]

What did you do?

Transcode a 1-bit PNG image to a CCITT Group 4 TIFF in memory.

The file is:

What did you expect to happen?

Expected writing to a BytesIO stream to work correctly, or for Pillow to reject saving to a BytesIO stream if a file descriptor is required.

What actually happened?

See error output below. It seems the problem is related to there not being a fileno associated with BytesIO.

What versions of Pillow and Python are you using?

macOS 10.11.6
Python 3.5.1 (homebrew)
Pillow==3.4.2
libtiff: stable 4.0.6 (bottled)

Code

from PIL import Image
from io import BytesIO
from tempfile import TemporaryFile

bio = BytesIO()
im = Image.open('_1.png')
im.save(bio, format='TIFF', compression='group4')  # produces error

tempio = TemporaryFile()
im.save(tempio, format='TIFF', compression='group4')  # works fine

Output

TIFFAppendToStrip: Write error at scanline 1700.
TIFFAppendToStrip: Write error at scanline 1700.
.../venv-3.5/lib/python3.5/site-packages/PIL/Image.py in save(self, fp, format, **params)
   1696 
   1697         try:
-> 1698             save_handler(self, fp, filename)
   1699         finally:
   1700             # do what we can to clean up

.../venv-3.5/lib/python3.5/site-packages/PIL/TiffImagePlugin.py in _save(im, fp, filename)
   1482                 break
   1483         if s < 0:
-> 1484             raise IOError("encoder error %d when writing image file" % s)
   1485 
   1486     else:

OSError: encoder error -9 when writing image file

[radarhere]

I'll just note that the problem occurs when using any compression argument that triggers libtiff. Omitting compression means that the image saves to BytesIO without an issue.

[wiredfool]

We've had something similar to this before: pr #1011, which fixed BytesIO saving of an image from libtiff. The tests for that image did test writing to a BytesIO, but not for group4. (https://github.com/python-pillow/Pillow/blob/master/Tests/test_file_libtiff.py#L457) I've added g4 compression to my local test, but did not trigger the issue with the existing test image. So it's not an fd issue. Saving to a BytesIO does work in at least one case.

That being said, I can replicate your issue with your image.

[jbarlow83]

I had also observed that not all input images trigger this behavior. I apologize; that point should have been in the original description. Perhaps file size or size after compression are factors. In any case it does come down to some difference between TemporaryFile and BytesIO.

[wiredfool]

Under the hood, there's a big difference. Anything with a real file pointer gets it passed to libtiff, anything without is written to memory, and then moved into the BytesIO.

It looks like what's happening here is that the overflow check on realloc of the buffer is failing due to an invalid cast, or it was just an invalid check in the first place.

If you can, try this branch: https://github.com/wiredfool/Pillow/tree/tiff_bytesio

[jbarlow83]

Tested (./setup.py install, installed in py3 venv) and it works. Thank you.