Compiling with SSE2 on Linux 32-bit causes crashes in OpenJPEG

[smuehlst]

This was tested with the current OpenJPEG HEAD revision bbef2a9.

I configured a Linux 32-bit build with SSE2 support like this:

CC=gcc cmake ../openjpeg -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS="-mfpmath=sse -msse2"

The resulting opj_decompress executable crashes for several files of the OpenJPEG testsuite, for with openjpeg-data/input/conformance/g4_colr.j2c:

(gdb) r -i ../openjpeg-data/input/conformance/g4_colr.j2c -o g4_colr.j2c.raw

 Starting program: /home/pdflib/src/openjpeg_debug/bin/opj_decompress -i ../openjpeg-data/input/conformance/g4_colr.j2c -o g4_colr.j2c.raw

[INFO] Start to read j2k main header (0).
[INFO] Main header has been correctly decoded.
[INFO] No decoded area parameters, set the decoded area to the whole image
[INFO] Header of tile 1 / 2 has been read.

Program received signal SIGSEGV, Segmentation fault.
0xb7fb7a46 in _mm_load_si128 (__P=0x81e0d78)
    at /usr/lib/gcc/i686-linux-gnu/4.8/include/emmintrin.h:678
678  return *__P;
(gdb) x/5i $pc
=> 0xb7fb7a46 <opj_mct_decode+147>: movdqa (%eax),%xmm0
   0xb7fb7a4a <opj_mct_decode+151>: movdqa %xmm0,-0x108(%ebp)
   0xb7fb7a52 <opj_mct_decode+159>: movdqa -0x128(%ebp),%xmm0
   0xb7fb7a5a <opj_mct_decode+167>: movdqa %xmm0,-0xf8(%ebp)
   0xb7fb7a62 <opj_mct_decode+175>: movdqa -0x118(%ebp),%xmm0
(gdb) p $eax
$8 = 136187256
(gdb) p $eax % 16
$9 = 8

If I understand the description of the movdqa instruction correctly, the memory operand must be 16-byte aligned, which is not the case here.

[malaterre]

See #625 progress