Undefined behavior

[Code7R]

Should be fixed by somebody who is sober enough and has time to deal with math...

Repro steps: build with -fsanitize=integer , run, move windows around, let the tool tips appear on the edges of the screen, open menus on the edges.

Bonus points: I think I have seen one of them with icewmbg too.

src/ypopup.cc:156:27: runtime error: unsigned integer overflow: 333 + 4294967295 cannot be represented in type 'unsigned int'
src/ypopup.cc:156:15: runtime error: unsigned integer overflow: 242 - 332 cannot be represented in type 'unsigned int'
src/ypopup.cc:168:22: runtime error: unsigned integer overflow: 4294967206 + 333 cannot be represented in type 'unsigned int'
src/ypopup.cc:183:16: runtime error: unsigned integer overflow: 4294967206 + 333 cannot be represented in type 'unsigned int'
src/ypopup.cc:147:26: runtime error: unsigned integer overflow: 183 + 4294967295 cannot be represented in type 'unsigned int'
src/ypopup.cc:147:15: runtime error: unsigned integer overflow: 408 - 411 cannot be represented in type 'unsigned int'
src/ypopup.cc:161:22: runtime error: unsigned integer overflow: 4294967293 + 233 cannot be represented in type 'unsigned int'
src/ypopup.cc:164:15: runtime error: unsigned integer overflow: 4294967293 + 411 cannot be represented in type 'unsigned int'

[Code7R]

More, open window list:

src/wmwinlist.cc:467:20: runtime error: unsigned integer overflow: 102 - 165 cannot be represented in type 'unsigned int'
src/wmwinlist.cc:468:20: runtime error: unsigned integer overflow: 51 - 133 cannot be represented in type 'unsigned int'
src/wmwinlist.cc:469:20: runtime error: unsigned integer overflow: 4294967233 + 330 cannot be represented in type 'unsigned int'
src/wmwinlist.cc:471:20: runtime error: unsigned integer overflow: 4294967214 + 266 cannot be represented in type 'unsigned int'

[gijsbers]

The messages point out that it is not about math but about types.
Just stop using unsigned for width and height if you frequently need to combine
them with signed integers in the same calculation. Simple.

[Code7R]

Dealing with signedness is not a problem. The problem is that some of those variables are unsigned because they used directly or indirectly in X11 API calls. And there it's hard to tell whether there are reserved ranges or special values, unless you know the APIs by heart.

In some cases it might be possible to go with int64_t but it's IMHO ultima ratio and not applicable everywhere and still needs to be considered properly.

[bbidulock]

I changed a lot of heights and widths to unsigned to expose this very issue. Many calculation seem to be ignorant of the fact that their results cannot be less than or equal to zero.

[gijsbers]

It caused a number of bugs, some of which are still not solved...
There was enough work to be done already...