Skip to content

Commit

Permalink
opj_t1_encode_cblks: fix UBSAN signed integer overflow
Browse files Browse the repository at this point in the history
Fixes uclouvain#1053 / CVE-2018-5727

Note: I don't consider this issue to be a security vulnerability, in
practice.
At least with gcc or clang compilers on x86_64 which generate the same
assembly code with or without that fix.
  • Loading branch information
rouault committed Mar 29, 2019
1 parent 25b815d commit a1d32a5
Showing 1 changed file with 10 additions and 1 deletion.
11 changes: 10 additions & 1 deletion src/lib/openjp2/t1.c
Expand Up @@ -2168,9 +2168,18 @@ OPJ_BOOL opj_t1_encode_cblks(opj_t1_t *t1,
t1->data = tiledp;
t1->data_stride = tile_w;
if (tccp->qmfbid == 1) {
/* Do multiplication on unsigned type, even if the
* underlying type is signed, to avoid potential
* int overflow on large value (the output will be
* incorrect in such situation, but whatever...)
* This assumes complement-to-2 signed integer
* representation
* Fixes https://github.com/uclouvain/openjpeg/issues/1053
*/
OPJ_UINT32* OPJ_RESTRICT tiledp_u = (OPJ_UINT32*) tiledp;
for (j = 0; j < cblk_h; ++j) {
for (i = 0; i < cblk_w; ++i) {
tiledp[tileIndex] *= (1 << T1_NMSEDEC_FRACBITS);
tiledp_u[tileIndex] <<= T1_NMSEDEC_FRACBITS;
tileIndex++;
}
tileIndex += tileLineAdvance;
Expand Down

0 comments on commit a1d32a5

Please sign in to comment.