CPU exhaustion in ReadPSDChannelZip

[henices]

Hello all.
We found a denial of service (DoS) issue in ImageMagick 7.0.7-12 Q16 x86_64 , which can cause huge CPU consumption. (cpu 100%)

The policy.xml is as following

<policymap>
  <policy domain="resource" name="temporary-path" value="/tmp"/>
  <policy domain="resource" name="memory" value="256MiB"/>
  <policy domain="resource" name="map" value="512MiB"/>
  <policy domain="resource" name="width" value="8KP"/>
  <policy domain="resource" name="height" value="8KP"/>
  <policy domain="resource" name="area" value="16KP"/>
  <policy domain="resource" name="disk" value="1GiB"/>
  <policy domain="resource" name="file" value="768"/>
  <policy domain="resource" name="thread" value="2"/>
  <policy domain="resource" name="throttle" value="0"/>
  <policy domain="resource" name="time" value="120"/>
  <policy domain="system" name="precision" value="6"/>
  <policy domain="coder" rights="none" pattern="MVG" />
  <policy domain="filter" rights="none" pattern="*" />
  <policy domain="delegate" rights="none" pattern="HTTPS" />  
  <policy domain="path" rights="none" pattern="@*"/>  
</policymap>

convert ReadPSDChannelZip2-cpu-exhaustion /dev/null

gdb backtrace

#0  inflate (strm=0x7fffffff2be0, flush=2) at inflate.c:1241
#1  0x00007ffff47595b8 in ReadPSDChannelZip (image=0x607c0000e100, channels=1, type=0, compression=ZipWithoutPrediction, compact_size=91, exception=0x600c0000b0c0) at coders/psd.c:1264
#2  0x00007ffff475a109 in ReadPSDChannel (image=0x607c0000e100, image_info=0x607a0000d100, psd_info=0x7fffffff4020, layer_info=0x60480001f280, channel=0, compression=ZipWithoutPrediction,
    exception=0x600c0000b0c0) at coders/psd.c:1385                                             
#3  0x00007ffff475ac3a in ReadPSDLayer (image=0x607c00011900, image_info=0x607a0000d100, psd_info=0x7fffffff4020, layer_info=0x60480001f280, exception=0x600c0000b0c0) at coders/psd.c:1468
#4  0x00007ffff475dcba in ReadPSDLayersInternal (image=0x607c00011900, image_info=0x607a0000d100, psd_info=0x7fffffff4020, skip_layers=MagickFalse, exception=0x600c0000b0c0)
    at coders/psd.c:1832                       
#5  0x00007ffff476036f in ReadPSDImage (image_info=0x607a0000d100, exception=0x600c0000b0c0) at coders/psd.c:2180
#6  0x00007ffff41c56d6 in ReadImage (image_info=0x607a00010500, exception=0x600c0000b0c0) at MagickCore/constitute.c:497
#7  0x00007ffff41c7d3c in ReadImages (image_info=0x607a00013900, filename=0x60040000c710 "/tmp/cpu3.poc", exception=0x600c0000b0c0) at MagickCore/constitute.c:866
#8  0x00007ffff39e179c in ConvertImageCommand (image_info=0x607a00013900, argc=3, argv=0x60060000ed10, metadata=0x7fffffffc080, exception=0x600c0000b0c0) at MagickWand/convert.c:641
#9  0x00007ffff3b84a11 in MagickCommandGenesis (image_info=0x607a00016d00, command=0x4010d0 <ConvertImageCommand@plt>, argc=3, argv=0x7fffffffe4c8, metadata=0x0, exception=0x600c0000b0c0)
    at MagickWand/mogrify.c:183                
#10 0x000000000040164d in MagickMain (argc=3, argv=0x7fffffffe4c8) at utilities/magick.c:149   
#11 0x00000000004017e2 in main (argc=3, argv=0x7fffffffe4c8) at utilities/magick.c:180

when debug we found a infinite loop in the following code (coders/psd.c). stream.avail_out is always 352

1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) 
1264            ret=inflate(&stream,Z_SYNC_FLUSH);
(gdb) 
1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) 
1264            ret=inflate(&stream,Z_SYNC_FLUSH);
(gdb) 
1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) 
1264            ret=inflate(&stream,Z_SYNC_FLUSH);
(gdb) 
1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) 
1264            ret=inflate(&stream,Z_SYNC_FLUSH);
(gdb) 
1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) 
1264            ret=inflate(&stream,Z_SYNC_FLUSH);
(gdb) 
1265            if ((ret != Z_OK) && (ret != Z_STREAM_END))
(gdb) 
1262          while (stream.avail_out > 0)
(gdb) p stream.avail_out
$1 = 352

testcase:
https://github.com/henices/pocs/raw/master/ReadPSDChannelZip2-cpu-exhaustion

Credit: NSFocus Security Team <security (at) nsfocus (dot) com>

[urban-warrior]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in GIT master branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ http://www.imagemagick.org/download/beta/ by sometime tomorrow.

[nohmask]

This was assigned CVE-2017-17681.