[CVE-2016-10506] division-by-zero in function opj_pi_next_rpcl of pi.c (line 366)

[trylab]

Title

division-by-zero in function opj_pi_next_rpcl of pi.c (line 366)

Testing Environment

Ubuntu + OpenJPEG (GitHub master, 2016/05/06)

Exception Information

Program received signal SIGFPE, Arithmetic exception.
0xb69892b1 in opj_pi_next_rpcl (pi=0xb3a03ec0) at 
    /home/trylab/Desktop/repo/openjpeg/src/lib/openjp2/pi.c:366
366 if (!((pi->x % (OPJ_INT32)(comp->dx << rpx) == 0) || 
        ((pi->x == pi->tx0) && ((trx0 << levelno) % (1 << rpx))))){

PoC

https://raw.githubusercontent.com/trylab/PoCs/master/openjpeg/SIGFPE_opj_pi_next_rpcl@366/poc.j2k

Credit

Ke Liu of Tencent's Xuanwu LAB

[szukw000]

bin/opj_decompress -i /tmp/ISSUE-780/issue780-poc.j2k -o issue780-poc.j2k.png

[INFO] Start to read j2k main header (0).
[INFO] Main header has been correctly decoded.
[INFO] No decoded area parameters, set the decoded area to the whole image
[INFO] Stream reached its end !
[INFO] Header of tile 6 / 64 has been read.
Floating point exception

NAME(/tmp/ISSUE-780/issue780-poc.j2k)
LENG(568)

ENTER read_jp2c
[0]marker(0xff4f)
soc len(0)
[2]marker(0xff51)
siz len(47)
capabilities(0)[extended: 0]
x(0 : 64) y(0 : 64)
xt(0 : 64) yt(0 : 1)
IMAGE w(64) h(64) TILE w(64) h(1)

----------------------

Tested with openjpeg-master-2016-05-09.

winfried

[rouault]

Fixed per d27ccf0