CVE-2017-14174: denial of service (DoS) issue in ReadPSDLayersInternal():1707 in coders/psd.c

[shqking]

Hello all.
We found a denial of service (DoS) issue in Imagemagick-7.0.7-0 Q16 x86_64, which can cause huge CPU consumption.
Note that this issue is quite similar to issue #712 we have reported.

The vulnerable code is shown as below.

1707                 for (j=0; j < (ssize_t) length; j+=8)
1708                 {
1709                   size_t blend_source=ReadBlobLong(image);
1710                   size_t blend_dest=ReadBlobLong(image);
1711                   if (image->debug != MagickFalse)
1712                     (void) LogMagickEvent(CoderEvent,GetMagickModule(),
1713                       "        source(%x), dest(%x)",(unsigned int)
1714                       blend_source,(unsigned int) blend_dest);
1715                 }

A crafted PSD image file, which claims large length but does not contain sufficient backing data, would cause a large loop at line 1707 since there is no EOF check inside.
PoC: https://github.com/shqking/imagemagick-poc/blob/master/x_psd_poc.psd
The command we was using is convert x_psd_poc.psd test.jpg
In our tests we used a machine with Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 4 CPU cores and 16GB RAM.
This issue caused 100% CPU for more than 3 and a half minutes.

Note that this issue was found by Xiaohei and Wangchu from Alibaba Security Team.
Thanks.

[urban-warrior]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in GIT master branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ https://www.imagemagick.org/download/beta/ by sometime tomorrow.

[shqking]

This is CVE-2017-14174.