memory exhaustion in ReadVIFFImage

[whiteHat001]

root@ubuntu:/home/hjy/Desktop# magick convert oom-ReadVIFFImage1 /dev/null==26800==ERROR: AddressSanitizer failed to allocate 0x84004000 (-2080358400) bytes of LargeMmapAllocator: 12
==26800==Process memory map follows:
0x08048000-0x094d3000 /usr/local/bin/magick
0x094d3000-0x094d4000 /usr/local/bin/magick
0x094d4000-0x09555000 /usr/local/bin/magick
0x09555000-0x09558000
0x1ffff000-0x24000000
0x24000000-0x28000000
0x28000000-0x40000000
0xb2d00000-0xb2e00000
0xb2f00000-0xb3000000
0xb3100000-0xb3200000
0xb3300000-0xb3400000
0xb3500000-0xb3600000
0xb3700000-0xb3800000
0xb3900000-0xb3a00000
0xb3b00000-0xb3c00000
0xb3d00000-0xb3e00000
0xb3f00000-0xb4100000 /usr/lib/locale/locale-archive
0xb4100000-0xb4200000
0xb4300000-0xb4400000
0xb4500000-0xb4600000
0xb4700000-0xb4800000
0xb4900000-0xb4a00000
0xb4b00000-0xb4c00000
0xb4d00000-0xb4e00000
0xb4f00000-0xb5000000
0xb5100000-0xb5200000
0xb5300000-0xb5400000
0xb5500000-0xb5600000
0xb5700000-0xb5800000
0xb5900000-0xb5a00000
0xb5ac0000-0xb5c00000
0xb5c01000-0xb5c46000
0xb5c46000-0xb5c47000 /usr/lib/locale/locale-archive
0xb5c47000-0xb6dfa000
0xb6dfa000-0xb6e1e000 /lib/i386-linux-gnu/liblzma.so.5.0.0
0xb6e1e000-0xb6e1f000 /lib/i386-linux-gnu/liblzma.so.5.0.0
0xb6e1f000-0xb6e20000 /lib/i386-linux-gnu/liblzma.so.5.0.0
0xb6e20000-0xb6e23000 /lib/i386-linux-gnu/libdl-2.19.so
0xb6e23000-0xb6e24000 /lib/i386-linux-gnu/libdl-2.19.so
0xb6e24000-0xb6e25000 /lib/i386-linux-gnu/libdl-2.19.so
0xb6e25000-0xb6fce000 /lib/i386-linux-gnu/libc-2.19.so
0xb6fce000-0xb6fd0000 /lib/i386-linux-gnu/libc-2.19.so
0xb6fd0000-0xb6fd1000 /lib/i386-linux-gnu/libc-2.19.so
0xb6fd1000-0xb6fd4000
0xb6fd4000-0xb6fec000 /lib/i386-linux-gnu/libpthread-2.19.so
0xb6fec000-0xb6fed000 /lib/i386-linux-gnu/libpthread-2.19.so
0xb6fed000-0xb6fee000 /lib/i386-linux-gnu/libpthread-2.19.so
0xb6fee000-0xb6ff0000
0xb6ff0000-0xb700b000 /lib/i386-linux-gnu/libgcc_s.so.1
0xb700b000-0xb700c000 /lib/i386-linux-gnu/libgcc_s.so.1
0xb700c000-0xb700d000 /lib/i386-linux-gnu/libgcc_s.so.1
0xb700d000-0xb700e000
0xb700e000-0xb7052000 /lib/i386-linux-gnu/libm-2.19.so
0xb7052000-0xb7053000 /lib/i386-linux-gnu/libm-2.19.so
0xb7053000-0xb7054000 /lib/i386-linux-gnu/libm-2.19.so
0xb7054000-0xb7082000 /usr/lib/i386-linux-gnu/libgomp.so.1.0.0
0xb7082000-0xb7083000 /usr/lib/i386-linux-gnu/libgomp.so.1.0.0
0xb7083000-0xb7084000 /usr/lib/i386-linux-gnu/libgomp.so.1.0.0
0xb7084000-0xb709c000 /lib/i386-linux-gnu/libz.so.1.2.8
0xb709c000-0xb709d000 /lib/i386-linux-gnu/libz.so.1.2.8
0xb709d000-0xb709e000 /lib/i386-linux-gnu/libz.so.1.2.8
0xb709e000-0xb71f3000 /usr/lib/i386-linux-gnu/libxml2.so.2.9.1
0xb71f3000-0xb71f4000 /usr/lib/i386-linux-gnu/libxml2.so.2.9.1
0xb71f4000-0xb71f8000 /usr/lib/i386-linux-gnu/libxml2.so.2.9.1
0xb71f8000-0xb71f9000 /usr/lib/i386-linux-gnu/libxml2.so.2.9.1
0xb71f9000-0xb71fa000
0xb71fa000-0xb7220000 /lib/i386-linux-gnu/libpng12.so.0.50.0
0xb7220000-0xb7221000 /lib/i386-linux-gnu/libpng12.so.0.50.0
0xb7221000-0xb7222000 /lib/i386-linux-gnu/libpng12.so.0.50.0
0xb7222000-0xb7223000
0xb7223000-0xb726c000 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
0xb726c000-0xb726d000 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
0xb726d000-0xb726e000 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
0xb726e000-0xb727e000
0xb727e000-0xb731d000 /usr/lib/i386-linux-gnu/libasan.so.1.0.0
0xb731d000-0xb731f000 /usr/lib/i386-linux-gnu/libasan.so.1.0.0
0xb731f000-0xb7320000 /usr/lib/i386-linux-gnu/libasan.so.1.0.0
0xb7320000-0xb7779000
0xb7779000-0xb778f000
0xb778f000-0xb7790000 [vdso]
0xb7790000-0xb77b0000 /lib/i386-linux-gnu/ld-2.19.so
0xb77b0000-0xb77b1000 /lib/i386-linux-gnu/ld-2.19.so
0xb77b1000-0xb77b2000 /lib/i386-linux-gnu/ld-2.19.so
0xbf874000-0xbf895000 [stack]
==26800==End of process memory map.
==26800==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cc:66 "(("unable to mmap" && 0)) != (0)" (0x0, 0x0)
#0 0xb72d24c1 (/usr/lib/i386-linux-gnu/libasan.so.1+0x544c1)
#1 0xb72d66a9 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/i386-linux-gnu/libasan.so.1+0x586a9)
#2 0xb72dbe22 (/usr/lib/i386-linux-gnu/libasan.so.1+0x5de22)
#3 0xb729699b (/usr/lib/i386-linux-gnu/libasan.so.1+0x1899b)
#4 0xb7297488 (/usr/lib/i386-linux-gnu/libasan.so.1+0x19488)
#5 0xb72cc84a in __interceptor_malloc (/usr/lib/i386-linux-gnu/libasan.so.1+0x4e84a)
#6 0x862b4eb in ReadVIFFImage coders/viff.c:514
#7 0x88980c8 in ReadImage MagickCore/constitute.c:497
#8 0x889bb49 in ReadImages MagickCore/constitute.c:866
#9 0x8ea0ba0 in ConvertImageCommand MagickWand/convert.c:641
#10 0x8fa97d1 in MagickCommandGenesis MagickWand/mogrify.c:183
#11 0x8074e7a in MagickMain utilities/magick.c:149
#12 0x805572a in main utilities/magick.c:180
#13 0xb6e3ea82 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x19a82)
#14 0x80744ba (/usr/local/bin/magick+0x80744ba)

POC https://github.com/whiteHat001/FUZZ_POC/blob/master/oom-ReadVIFFImage1

[mikayla-grace]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in GIT master branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ http://www.imagemagick.org/download/beta/ by sometime tomorrow.

[fgeek]

Please use CVE-2017-12692 for this issue.