memory exhaustion in ReadXWDImage

[jgj212]

ImageMagick 7.0.5-6

$magick identify $FILE

When identify XWD file, imagemagick will allocate memory to store data in function ReadXWDImage in coders\xwd.c, line 325
//////////////////////////////////
colors=(XColor *) AcquireQuantumMemory(length,sizeof(*colors)); // can be controlled
\\\\\\\\\\\\\\\\\\

length can be controlled, as it is assigned as follow(line 324):
//////////////////////////////////
length=(size_t) header.ncolors;
\\\\\\\\\\\\\\\\\\

header.ncolors are diretly from XWD file without checking( line 224)
//////////////////////////////////
count=ReadBlob(image,sz_XWDheader,(unsigned char *) &header); // can be controlled by modify XWD file
\\\\\\\\\\\\\\\\\\

header is a instance of struct _xwd_file_header as follow:
//////////////////////////////////
typedef struct _xwd_file_header {
/* header_size = SIZEOF(XWDheader) + length of null-terminated
* window name. */
CARD32 header_size B32;

CARD32 file_version B32;	/* = XWD_FILE_VERSION above */
CARD32 pixmap_format B32;	/* ZPixmap or XYPixmap */
CARD32 pixmap_depth B32;	/* Pixmap depth */
CARD32 pixmap_width B32;	/* Pixmap width */
CARD32 pixmap_height B32;	/* Pixmap height */
CARD32 xoffset B32;		/* Bitmap x offset, normally 0 */
CARD32 byte_order B32;		/* of image data: MSBFirst, LSBFirst */

/* bitmap_unit applies to bitmaps (depth 1 format XY) only.
 * It is the number of bits that each scanline is padded to. */
CARD32 bitmap_unit B32;		

CARD32 bitmap_bit_order B32;	/* bitmaps only: MSBFirst, LSBFirst */

/* bitmap_pad applies to pixmaps (non-bitmaps) only.
 * It is the number of bits that each scanline is padded to. */
CARD32 bitmap_pad B32;		

CARD32 bits_per_pixel B32;	/* Bits per pixel */

/* bytes_per_line is pixmap_width padded to bitmap_unit (bitmaps)
 * or bitmap_pad (pixmaps).  It is the delta (in bytes) to get
 * to the same x position on an adjacent row. */
CARD32 bytes_per_line B32;
CARD32 visual_class B32;	/* Class of colormap */
CARD32 red_mask B32;		/* Z red mask */
CARD32 green_mask B32;		/* Z green mask */
CARD32 blue_mask B32;		/* Z blue mask */
CARD32 bits_per_rgb B32;	/* Log2 of distinct color values */
CARD32 colormap_entries B32;	/* Number of entries in colormap; not used? */
CARD32 ncolors B32;		/* Number of XWDColor structures */
CARD32 window_width B32;	/* Window width */
CARD32 window_height B32;	/* Window height */
CARD32 window_x B32;		/* Window upper left X coordinate */
CARD32 window_y B32;		/* Window upper left Y coordinate */
CARD32 window_bdrwidth B32;	/* Window border width */

} XWDFileHeader;
\\\\\\\\\\\\\\\\\\

So, modifying the ncolors can cause ImageMagick to allocate a anysize amount of memory, this may cause a memory exhaustion

Reproducer: https://github.com/jgj212/poc/blob/master/ImageMagick-7.0.5-6-memory-exhaustion.XWD
Credit: ADLab of Venustech

[dlemstra]

Thanks for reporting this. Can you help us figure out what the limit for ncolors should be? I found the following documentation: http://www.fileformat.info/format/xwd/egff.htm

EntryNumber is the number of the color map entry. This value starts at 00h. Color maps typically do not exceed 256 entries in size.

But it does not provide a hard limit for the number of colors.

[jgj212]

@dlemstra from the document, EntryNumber can not bigger than filesize/sizeof(X11COLORMAP)

[bastien-roucaries]

This is CVE-2017-11166

[bastien-roucaries]

It does not seems to be corrected for V6

[dlemstra]

This is the IM6 commit: 5964475