Malformed jpeg causes segfault in lt-cjpeg at get_text_rgb_row (rdppm.c:171)

[geeknik]

While fuzzing jpegtran with American Fuzzy Lop, I was able to cause a SEGFAULT in lt-cjpeg with a malformed jpeg.

~/jpegtran/.libs/lt-cjpeg -dct fast -nosmooth -arithmetic -outfile /dev/null -rgb test11.jpg

==19677== Invalid read of size 1
==19677==    at 0x407E71: get_text_rgb_row (rdppm.c:171)
==19677==    by 0x401ACD: main (cjpeg.c:642)
==19677==  Address 0x1fca8f64 is not stack'd, malloc'd or (recently) free'd
==19677== 
==19677== 
==19677== Process terminating with default action of signal 11 (SIGSEGV)
==19677==  Access not within mapped region at address 0x1FCA8F64
==19677==    at 0x407E71: get_text_rgb_row (rdppm.c:171)
==19677==    by 0x401ACD: main (cjpeg.c:642)
==19677==  If you believe this happened as a result of a stack
==19677==  overflow in your program's main thread (unlikely but
==19677==  possible), you can try to increase the size of the
==19677==  main thread stack using the --main-stacksize= flag.
==19677==  The main thread stack size used in this run was 8388608.
Segmentation fault

Program received signal SIGSEGV, Segmentation fault.
0x0000000000407e71 in get_text_rgb_row ()
(gdb) bt
#0  0x0000000000407e71 in get_text_rgb_row ()
#1  0x0000000000401ace in main () at cjpeg.c:642
(gdb) i r
rax            0x1a7daf1c   444444444
rbx            0x1a7daf1c   444444444
rcx            0x0  0
rdx            0x61abc0 6400960
rsi            0x98 152
rdi            0x7fffffffdfc0   140737488347072
rbp            0x7fffffffdfc0   0x7fffffffdfc0
rsp            0x7fffffffdeb0   0x7fffffffdeb0
r8             0x7ffff7fde700   140737354000128
r9             0x1aa8   6824
r10            0x1  1
r11            0x0  0
r12            0x61aae0 6400736
r13            0x61ee71 6418033
r14            0x61ee68 6418024
r15            0x7ffff7f41028   140737353355304
rip            0x407e71 0x407e71 <get_text_rgb_row+209>
eflags         0x10206  [ PF IF RF ]
cs             0x33 51
ss             0x2b 43
ds             0x0  0
es             0x0  0
fs             0x0  0
gs             0x0  0

Minimizing the test case changes the valgrind output a bit:

~/jpegtran/.libs/lt-cjpeg -dct fast -nosmooth -arithmetic -outfile /dev/null -rgb test11-min.jpg

==59264== Invalid read of size 1
==59264==    at 0x407E71: get_text_rgb_row (rdppm.c:171)
==59264==    by 0x401ACD: main (cjpeg.c:642)
==59264==  Address 0x552bb80 is 381,680 bytes inside an unallocated block of size 4,171,600 in arena "client"
==59264== 
Premature end of input file

Program received signal SIGSEGV, Segmentation fault.
0x0000000000407e71 in get_text_rgb_row ()
(gdb) bt
#0  0x0000000000407e71 in get_text_rgb_row ()
#1  0x0000000000401ace in main () at cjpeg.c:642
(gdb) i r
rax            0x61a80  400000
rbx            0x61a80  400000
rcx            0x0  0
rdx            0x61abc0 6400960
rsi            0xffffffff   4294967295
rdi            0x7fffffffdfc0   140737488347072
rbp            0x7fffffffdfc0   0x7fffffffdfc0
rsp            0x7fffffffdeb0   0x7fffffffdeb0
r8             0x7ffff7fde700   140737354000128
r9             0x1ab0   6832
r10            0x1  1
r11            0x246    582
r12            0x61aae0 6400736
r13            0x61ee71 6418033
r14            0x61ee68 6418024
r15            0x61afd0 6402000
rip            0x407e71 0x407e71 <get_text_rgb_row+209>
eflags         0x10206  [ PF IF RF ]
cs             0x33 51
ss             0x2b 43
ds             0x0  0
es             0x0  0
fs             0x0  0
gs             0x0  0

[kornelski]

I think this might be the fix: mozilla/mozjpeg@5ba6c7e (mozilla/mozjpeg#141)

[vkrasnov]

@geeknik @pornel thank you very much for your interest. We are however not using the cjpeg utility, and only use jpegtran.
Once we merge our speed improvements to the other forks, we will stop maintaining this fork.
I think those issues should be raised with other, maintained forks.