Tiff to JPEG with libvips 8.8.0 for some images

[rps01]

With libvips 8.8.0 , some of tiff assets aren't successfully converting to jpeg

command: vipsthumbnail Asset.tif -s 1280x1280 -o new.jpeg

Logs:

vipsthumbnail: unable to thumbnail Asset.tif
TIFFSetField: Asset.tif: Unknown pseudo-tag 65538
TIFFSetField: Asset.tif: Unknown pseudo-tag 65538
VipsJpeg: Bogus marker length

This issue wasn't there with previous version 8.7.4.

Related older issue: #311
Using below it does works:
command: vipsthumbnail Asset.tif -s 1280x1280 -o new.jpeg[strip]

[jcupitt]

Hello @rps01,

That's interesting. Could you send a sample file that fails? You can mail me privately if that's easier.

[rps01]

@jcupitt Won't be possible to share copyright asset. Tiff assets for which there is issue, they have metadata of about 67kb/70kb.

We plan to use [strip] option for now. Do you see any issue if we use [strip] with assets which doesn't even have XMPmetadata.

[strip] option is something i found from the past issue , is there any documentation where such options are mentioned. Thanks.

[jcupitt]

I think XMP over 64kb is difficult for libjpeg, so this is not supported.

The jpeg save docs are here:

https://jcupitt.github.io/libvips/API/current/VipsForeignSave.html#vips-jpegsave

strip is common to all savers and simply removes all metadata.

It will remove ICC profiles as well, so you should convert to sRGB if these are for display. This page has some notes:

https://libvips.github.io/libvips/API/current/Using-vipsthumbnail.md.html

[chregu]

Have the same issue, also when converting to png. It's somehow random and hardly reproducible but have it in my logs from time to time (from the PHP extension).
What definitely does trigger it, is to call vipsthumbnail with a 2nd not-existing file.

vipsthumbnail pseudoerror.tiff notexisting.jpg

And here's the triggering image:
http://files.chregu.tv/pseudoerror.tiff

[jcupitt]

Thanks for the example @chregu.

I think it is working, eg.:

$ vipsthumbnail pseudoerror.tiff banana.tif

(vipsthumbnail:16029): VIPS-WARNING **: 10:45:23.509: Incompatible type for "RichTIFFIPTC"; tag ignored

[trim]

vipsthumbnail: unable to thumbnail banana.tif
TIFFSetField: pseudoerror.tiff: Unknown pseudo-tag 65538
TIFFSetField: pseudoerror.tiff: Unknown pseudo-tag 65538
TIFFSetField: pseudoerror.tiff: Unknown pseudo-tag 65538
TIFFSetField: pseudoerror.tiff: Unknown pseudo-tag 65538
VipsForeignLoad: file "banana.tif" not found

You see the extra warnings from TIFFSetField because of the error on banana.tif, but the thumbnail of pseudoerror.tif did work (there's a thumbnail).

I'll see if I can see what's causing the TIFFSetField() warnings.

[chregu]

Ah yes, it works for vipsthumbnail, but in the php extension, does TIFFSetField warnings do seem to throw an exception, which is kind of strange, since it's just a warning.

[chregu]

Still hard to reproduce, it happens only every few requests. But figured, that the error may come from somewhere else, from an "out of order read" error. The whole message actually in the exception actually is.

TIFFSetField: memory input: Unknown pseudo-tag 65538
tiff2vips: out of order read at line 1503

[jcupitt]

Huh interesting. It's a plane-separated image (R, G and B are stored in separate subimages), perhaps that's a clue?

The Unknown pseudo-tag 65538 messages are because libvips is requesting expansion of jpg output to RGB from YCbCr when the jpg compressor is not being used.

I thought this was harmless, but it seems it makes libtiff fret. I'll add a test.

[jcupitt]

I fixed the spurious Unknown tag warning, thanks!

Otherwise, I can't get it to fail here :( I've set this going:

john@yingna ~/pics $ export VIPS_WARNING=1
john@yingna ~/pics $ for i in {1..10000}; do echo $i; vipsthumbnail pseudoerror.tiff || break; done
1
2
3
4
...

I'll leave it running while I work today and see if it falls over.

[chregu]

Thanks. for that. I also can reproduce with a php-fpm server and hitting reload lots of time. But not with a simple bash script :(

[jcupitt]

Yes, mine hit 10000 iterations and had no problems :(

[jcupitt]

Someone found a sample file that triggers the "bogus marker length" error:

#1372

It may be fixed in 8.8 HEAD and the fix should be in 8.8.2. Would you be able to test this?

[jcupitt]

I think this may be fixed in 8.8.2 and later.

Please open a new issue if this comes up again.