IM GIF decoder detects new frame where there is none

[cyburgee]

Using e9d84ca, IM detects an invalid frame where there doesn't seem to be any, according to my inspection of the file. I'm running OS X 10.12.6.

Here's an example file:
https://www.dropbox.com/s/mj46nggav4jeujm/example.gif?dl=0

$ identify --version
Version: ImageMagick 7.0.7-7 Q8 x86_64 2017-10-05 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC HDRI 
Delegates (built-in): bzlib fftw fontconfig freetype jng jpeg lcms lzma openexr pangocairo png tiff webp x xml zlib
$ identify example.gif
example.gif[0] GIF 200x11 200x11+0+0 8-bit sRGB 256c 12902B 0.010u 0:00.009
example.gif[1] GIF 198x11 200x11+1+0 8-bit sRGB 128c 12902B 0.010u 0:00.009
example.gif[2] GIF 200x11 200x11+0+0 8-bit sRGB 256c 12902B 0.010u 0:00.009
example.gif[3] GIF 198x11 200x11+2+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[4] GIF 197x11 200x11+1+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[5] GIF 199x11 200x11+1+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[6] GIF 195x11 200x11+2+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[7] GIF 197x11 200x11+1+0 8-bit sRGB 128c 12902B 0.010u 0:00.009
example.gif[8] GIF 197x11 200x11+2+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[9] GIF 1x1 200x11+38+5 8-bit sRGB 256c 12902B 0.010u 0:00.009
example.gif[10] GIF 198x11 200x11+0+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[11] GIF 198x11 200x11+2+0 8-bit sRGB 64c 12902B 0.010u 0:00.009
example.gif[12] GIF 168x9 200x11+30+2 8-bit sRGB 32c 12902B 0.010u 0:00.009
example.gif[13] GIF 182x11 200x11+16+0 8-bit sRGB 32c 12902B 0.010u 0:00.009
example.gif[14] GIF 193x11 200x11+3+0 8-bit sRGB 32c 12902B 0.010u 0:00.009
example.gif[15] GIF 194x9 200x11+2+1 8-bit sRGB 16c 12902B 0.010u 0:00.009
example.gif[16] GIF 195x9 200x11+2+2 8-bit sRGB 32c 12902B 0.010u 0:00.009
example.gif[17] GIF 32x7 200x11+10+1 8-bit sRGB 3c 12902B 0.010u 0:00.009
example.gif[18] GIF 161x6 200x11+37+2 8-bit sRGB 32c 12902B 0.000u 0:00.009
example.gif[19] GIF 190x10 200x11+8+1 8-bit sRGB 32c 12902B 0.000u 0:00.009
example.gif[20] GIF 193x11 200x11+6+0 8-bit sRGB 16c 12902B 0.000u 0:00.009
example.gif[21] GIF 161x7 200x11+37+3 8-bit sRGB 32c 12902B 0.000u 0:00.009
example.gif[22] GIF 196x9 200x11+3+2 8-bit sRGB 32c 12902B 0.000u 0:00.009
example.gif[23] GIF 190x6 200x11+6+3 8-bit sRGB 16c 12902B 0.000u 0:00.009
example.gif[24] GIF 192x10 200x11+6+1 8-bit sRGB 32c 12902B 0.000u 0:00.009
example.gif[25] GIF 163x9 200x11+37+1 8-bit sRGB 9c 12902B 0.000u 0:00.009
example.gif[26] GIF 8159x64597 200x11+52602+47982 8-bit sRGB 256c 12902B 0.000u 0:00.000
example.gif[27] GIF 154x5 200x11+39+4 8-bit sRGB 8c 12902B 0.000u 0:00.000
example.gif[28] GIF 194x11 200x11+3+0 8-bit sRGB 32c 12902B 0.000u 0:00.000
example.gif[29] GIF 198x11 200x11+2+0 8-bit sRGB 64c 12902B 0.000u 0:00.000
example.gif[30] GIF 193x11 200x11+5+0 8-bit sRGB 32c 12902B 0.000u 0:00.000
example.gif[31] GIF 197x9 200x11+1+2 8-bit sRGB 32c 12902B 0.000u 0:00.000
example.gif[32] GIF 142x5 200x11+37+3 8-bit sRGB 4c 12902B 0.000u 0:00.000
example.gif[33] GIF 156x5 200x11+38+3 8-bit sRGB 256c 12902B 0.000u 0:00.000
example.gif[34] GIF 102x5 200x11+96+6 8-bit sRGB 16c 12902B 0.000u 0:00.000
identify: corrupt image `example.gif' @ error/gif.c/PingGIFImage/929.

The GIF decoding in IM seems to think that there's a very large frame after no. 25, which simply isn't there.

[mikayla-grace]

For security reasons, ImageMagick throws an exception if it finds any metadata in an image that does not conform to the image specification. ImageMagick is finding an improper data size @ frame 18 and concludes the image is corrupt. Some other applications may ignore these kinds of problems and attempt to recover. Note, older versions of ImageMagick would make an attempt to ignore image corruption.

Try these commands

identify 'example.gif[0-17]'
identify 'example.gif[0-18]'

The first works and returns without complaint. The second returns a corrupt image exception.

We used an online GIF repair website, the repaired image file could be read by ImageMagick without complaint.

[cyburgee]

By my inspection of the hex below, there doesn't seem to be an improper data size.
Frame No. 17

21 F9 04 05 07 00 02 00 2C 0A 00 01 00 20 00 07 00 80 40 40 42 54 53 56 02 10 8C 8E A9 CB ED 0F A3 9C B4 5A 65 AE DE 09 88 02 00 

Frame No. 18

21 F9 04 05 0E 00 16 00 2C 25 00 02 00 A1 00 06 00 84 37 C6 27 3A 77 33 4A 4A 4C 4F 4E 51 53 53 55 56 56 59 59 56 5B 6F 6E 71 80 7F 83 83 83 84 86 86 88 8B 8B 8D 8E 8E 91 90 8F 92 99 99 9A A3 A2 A4 A6 A6 A8 A7 A8 A8 B3 B3 B4 BA B9 BB BD BC BE C6 C6 C8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05 4C A0 25 8E E4 28 94 68 AA AE 6C EB BE 70 2C CF 64 F1 9C D6 42 EF 7C EF FF C0 55 02 62 89 04 8F C8 A4 72 74 18 4D 66 84 96 64 49 AD 2A 9F BE 48 22 82 B3 26 03 00 AF D8 C2 68 31 2A 24 04 C5 F0 18 58 12 BA B1 7C 1E 6C 28 5C 8E 51 08 00

Nothing seemed amiss in my manual inspection but I certainly could have made a mistake. Do you have any insight into what data size is improper?

I was able to reproduce the result from your commands above, but oddly, if I try to extract no. 17 I get an invalid gif result.

$ convert example.gif[0-16] example_16.gif
$ convert example.gif[0-17] example_17.gif
convert: corrupt image `example.gif' @ error/gif.c/DecodeImage/419.
convert: corrupt image `example.gif' @ error/gif.c/ReadGIFImage/1384.
convert: no images defined `example_17.gif' @ error/convert.c/ConvertImageCommand/3275.

Is that to be expected?

[urban-warrior]

An improper data size suggests that there is extra bytes or missing bytes somewhere in the stream or perhaps an improperly compressed pixel stream. The question of course is the stream improper or is it a bug in ImageMagick? We will need proof its a bug before we can investigate further.

[cyburgee]

So, it seems that IM thinks that the color table size of frame 17 is 9 bytes when it is in fact 6. There seems to be some confusion in the decoder between the color table size and image->colors. If transparency is present, the number of colors in the image could be the table size + 1. I have fix for the improper read size issue but I'm still seeing some issues when it comes to encoding the transparency.

[mikayla-grace]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in GIT master branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ https://www.imagemagick.org/download/beta/ by sometime tomorrow.