vipsthumbnail generates corrupt images #36
Comments
|
Oh dear, that's bad. What version are you using? I tried in 7.30.4 and git master and it worked OK there. |
|
try vipsthumbnail --vips-versionto see the version of the libvips library you are using. |
|
I tried on a win7 machine using the current version of libvips on Windows (7.30.3) and it worked OK for me. http://www.vips.ecs.soton.ac.uk/supported/7.30/win32/vips-dev-7.30.3.zip Are you running an old version? |
|
I'm running libvips 7.30.3-Mon Oct 1 14:38:45 BST 2012 Its not a specific image that has the problem and its not consistent. Here are 3 consecutive runs of the program and it worked the 2nd time. I wonder if its significant the number of "read warnings" are different? C:>vipsthumbnail.exe --size=2404 --verbose -n 42-35276890-MEDHI_IPTC.jpg --output=tn_%s.jpg:70 C:>vipsthumbnail.exe" --size=2404 --verbose -n 42-35276890-MEDHI_IPTC.jpg --output=tn_%s.jpg:70 C:>vipsthumbnail.exe" --size=2404 --verbose -n 42-35276890-MEDHI_IPTC.jpg --output=tn_%s.jpg:70 |
|
Hi again, I've been able to test on a win machine and I see this problem too. I don't get it on linux, which is strange. I'll investigate. It looks like a race condition caused by differences in the threading systems between the two operating systems. Thanks for the report ferryfax. |
|
Hi again, I think I've found the problem, it was on linux as well, just not quite as often. libvips caches a few scanlines behind the read point when loading jpeg images for sequential processing. The number of lines cached was set to exactly what should be needed and had no room for "slop". When running with many threads some can get ahead or behind by small amounts. The cache needs to have a bit of spare room to allow for this. I've raised the linecache size by 50% and this seems to have stopped the problem. With this change: I was able to run your vipsthumbnail command 100 times with no problems. I'm making a new windows binary now, I'll test there too. |
|
That's great! Thanks very much for the quick response. Now if I could only get it to maintain the APP0 and Exif resolution settings from the original that would be awesome. |
|
Here's a test windows binary with this fix: http://www.vips.ecs.soton.ac.uk/development/vips-dev-7.30.4.zip Would you be able to test it? I should be able to test as well tomorrow. |
|
Oh, it's supposed to keep the exif and app0 resolution settings, are they getting lost? Here's what I see on linux: $ header -a img_0075.jpg | grep -i res
memory: high-water mark 0 bytes
xres: 7.086614
yres: 7.086614
exif-Compression: 6 (JPEG compression, Short, 1 components, 2 bytes)
exif-X-Resolution: 180 (180, Rational, 1 components, 8 bytes)
exif-Y-Resolution: 180 (180, Rational, 1 components, 8 bytes)
exif-Resolution Unit: 2 (Inch, Short, 1 components, 2 bytes)
exif-Compressed Bits per Pixel: 2 ( 2, Rational, 1 components, 8 bytes)
exif-Focal Plane X-Resolution: 7766.9902912621355 (7766.990, Rational, 1 components, 8 bytes)
exif-Focal Plane Y-Resolution: 7741.9354838709678 (7741.935, Rational, 1 components, 8 bytes)
exif-Focal Plane Resolution Unit: 2 (Inch, Short, 1 components, 2 bytes)
resolution-unit: in
$ vipsthumbnail img_0075.jpg
$ header -a tn_img_0075.jpg | grep -i res
memory: high-water mark 0 bytes
xres: 7.086614
yres: 7.086614
exif-Compression: 6 (JPEG compression, Short, 1 components, 2 bytes)
exif-X-Resolution: 180 (180.00000000, Rational, 1 components, 8 bytes)
exif-Y-Resolution: 180 (180.00000000, Rational, 1 components, 8 bytes)
exif-Resolution Unit: 2 (Inch, Short, 1 components, 2 bytes)
exif-Compressed Bits per Pixel: 2 (2.0000000000, Rational, 1 components, 8 bytes)
exif-Focal Plane X-Resolution: 7766.9902907178612 (7766.990291, Rational, 1 components, 8 bytes)
exif-Focal Plane Y-Resolution: 7741.9354827080242 (7741.935483, Rational, 1 components, 8 bytes)
exif-Focal Plane Resolution Unit: 2 (Inch, Short, 1 components, 2 bytes)
resolution-unit: inexif overrides app0, I think. |
|
That version works great thanks. As for the resolution here is what JPEGsnoop shows for the original 5500x5524 and for a thumbnail created at 2404x2414. As you can see it changes the APP0 JFIF resolution to be an aspect ration instead of dpi and it messes with the Exif. It even changes the embbeded exif IFD1 thumbnail resolution for good measure. *** Marker: APP0 (xFFE0) *** *** Marker: APP1 (xFFE1) *** EXIF IFD0 @ Absolute 0x00000332 EXIF IFD1 @ Absolute 0x0000152A EXIF SubIFD @ Absolute 0x00000C6A After vipsthumbnail:- *** Marker: APP0 (xFFE0) *** *** Marker: APP1 (xFFE1) *** EXIF IFD0 @ Absolute 0x00000026 EXIF IFD1 @ Absolute 0x00001236 EXIF SubIFD @ Absolute 0x000009A6 |
|
Great, glad it's working, I've made it an official 7.30.4. I'll close this issue and put your jpeg problem here: https://github.com/jcupitt/libvips/issues/37 It'll make it easier for people to find if we split the two problems up. |
|
Funny, I just ran into a simmiliar/identical issue on linux. Generated corrupted thumbnails even from fairly small images. Got the latest libvips from github, recompiled and works like a charm, thanks for fixing it. Just in case this was a different issue here is the source image:And here is the generated thumbnail:Used command and version below: |
|
I'm setting the same problem as well with vips-7.30.6. It seems to happen when I try to create a really small thumbnail. The error message is I only get this message in a modified version of vipsthumbnails.c to generate multiple thumbnail sizes from an image. |
|
Hi, I've just found a nasty bug in the processing pipeline used by vipsthumbnail (among others). On heavily-loaded systems there was a deadlock between two caches, one of the locks would then time out and you'd get a corrupted thumbnail. This might be the problem you're seeing. I think this is fixed now, would you be able to test again? http://www.vips.ecs.soton.ac.uk/supported/7.30/vips-7.30.7.tar.gz |
|
Seems to be better now but if I use with -c 1, it will give some warnings: |
|
Looks like -c is actually turning on --vips-cache-trace, a thing to show the actions of the vips8 operation cache. Try --vips-concurrency 1 and see if you get any warnings. I'll look into why the cache tracer is turning on, it shouldn't, I think. |
|
Here's a patch to remove a lot of the internal flag abbrevisions: it'll be in the next version. Thanks for reporting that. I'll close this issue -- reopen if you're still seeing corruption. |
When shrinking a 5500 x 5524 image to 2951 x 2964 vipsthumbnail on Windows gives a warning and the resulting thumbnail is corrupt.
thumbnailing 42-35276890-MEDHI_IPTC.jpg
detected format as jpeg
using fast jpeg shrink, factor 1
thumbnailing 42-35276890-MEDHI_IPTC.jpg as tn_42-35276890-MEDHI_IPTC.jpg:70
integer shrink by 2
residual scale by 0.968139
bilinear interpolation
vips warning: VipsJpeg: read gave 40 warnings
vips warning: VipsJpeg: Application transferred too many scanlines
The text was updated successfully, but these errors were encountered: