The check_overflow() function can work incorrectly in some cases

[pkopylov]

The check_overflow() function has been implemented to check an overflow of the product of three numbers.
Here is the declaration:
void check_overflow(unsigned int val1, unsigned int val2, unsigned int val3)

Unfortunately, the implementation you can see here isn't tolerant against an arithmetic overflow at least on 64bit systems. Please, pay attention that sizeof(unsigned long long) == 8, so it can hold values no more than 2^64-1, but each of val1, val2, and val3 can hold values up to 2^32-1, so the product of (2^32-1) * (2^32-1) * (2^32-1) overflows the 64-bit variable.

However, not every overflow leads to a failing check, most cases still detect an overflow even the product has been truncated, but there are combinations of arguments when things go wrong, for example:

• check_overflow(UINT_MAX, UINT_MAX, 2147483648),
• check_overflow(UINT_MAX-100, UINT_MAX-5, 2849931574)
  and so on.

It seems the CVE-2022-39377 is still exploitable in some conditions.

[sysstat]

Thanks for this alert.
Fixed by commit 954ff2e.

[wenpei-z]

Hello, I think that in normal use, the product of these three values ​​is much smaller than the maximum value of int, under what circumstances will it overflow?

[pkopylov]

Hello, I think that in normal use, the product of these three values ​​is much smaller than the maximum value of int, under what circumstances will it overflow?

For instance the following combination of parameters leads to overflow:
check_overflow(UINT_MAX-100, UINT_MAX-5, 2849931574)

In general, the product of three 4 bytes values may overflow 8 bytes accumulator. Please, see my comment above.

[sysstat]

Hello, I think that in normal use, the product of these three values ​​is much smaller than the maximum value of int, under what circumstances will it overflow?

It may happen, e.g. if you try to read a forged saDD datafile.