Global-buffer-overflow in cu_get_quant_params() --> davs2/source/common/quant.h:115

[arayzw]

Describe the bug

=============================================================================================

static ALWAYS_INLINE
void cu_get_quant_params(davs2_t * h, int qp, int bit_size,
int *shift, int *scale)
{
*shift = IQ_SHIFT[qp] + (h->sample_bit_depth + 1) + bit_size - LIMIT_BIT; <------ out of bounds here
*scale = IQ_TAB[qp];
}

=============================================================================================

To Reproduce

cd /path/to/davs2/build/linux/
./configure --enable-pic
vim config.mak (add -fsanitize=address to CFLAGS, and -fsanitize=address -lasan to LDFLAGS)
make
./davs2 -i /path/to/poc4 -o test.yuv

ASAN Crash log

=================================================================
==107413==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55ec49b52e7c at pc 0x55ec49a5bb2d bp 0x7fabbb269cb0 sp 0x7fabbb269ca0
READ of size 2 at 0x55ec49b52e7c thread T1
#0 0x55ec49a5bb2c in cu_get_quant_params /home/arayz/arayz/work/davs2/source/common/quant.h:115
#1 0x55ec49a5bb2c in cu_read_all_coeffs /home/arayz/arayz/work/davs2/source/common/cu.cc:662
#2 0x55ec49a5bb2c in cu_read_info /home/arayz/arayz/work/davs2/source/common/cu.cc:872
#3 0x55ec49a5bb2c in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1327
#4 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#5 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#6 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#7 0x55ec499abb6b in decode_one_lcu_row /home/arayz/arayz/work/davs2/source/common/decoder.cc:438
#8 0x55ec499b12ee in davs2_decoder_decode_picture_data /home/arayz/arayz/work/davs2/source/common/decoder.cc:1168
#9 0x55ec499d8b6a in davs2_threadpool_thread /home/arayz/arayz/work/davs2/source/common/threadpool.cc:216
#10 0x7fabce4aa608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
#11 0x7fabce083132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

0x55ec49b52e7c is located 4 bytes to the left of global variable 'dmh_pos' defined in '/home/arayz/arayz/work/davs2/source/common/cu.cc:77:21' (0x55ec49b52e80) of size 36
0x55ec49b52e7c is located 28 bytes to the right of global variable 'davs2_IQ_SHIFT' defined in '/home/arayz/arayz/work/davs2/source/common/cu.cc:91:15' (0x55ec49b52dc0) of size 160
SUMMARY: AddressSanitizer: global-buffer-overflow /home/arayz/arayz/work/davs2/source/common/quant.h:115 in cu_get_quant_params
Shadow bytes around the buggy address:
0x0abe09362570: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
0x0abe09362580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9
0x0abe09362590: f9 f9 f9 f9 00 00 00 00 00 00 f9 f9 f9 f9 f9 f9
0x0abe093625a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0abe093625b0: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00
=>0x0abe093625c0: 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9[f9]
0x0abe093625d0: 00 00 00 00 04 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
0x0abe093625e0: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00
0x0abe093625f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0abe09362600: 00 00 00 00 00 00 00 00 f9 f9 f9 f9 00 00 00 00
0x0abe09362610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
Thread T1 created by T0 here:
#0 0x7fabce4ff815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
#1 0x55ec499d8f36 in davs2_threadpool_init /home/arayz/arayz/work/davs2/source/common/threadpool.cc:271

==107413==ABORTING

Additional context

• OS: Ubuntu 20.04 (Desktop)
• Compiler: gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

PoC:
poc4.zip