Properly set size of shadow atlas quadrant when subdivision is 8 or higher.

[clayjohn]

Fixes: #89312

I still can't figure out why this seemingly regressed between dev1 and dev2. Both of these problems have been present for a long time.

Problem 1: Shadow atlas quadrant allocation

Each atlas stores shadows for up to subdivision * subdivision lights (omnilights count for 2). When we set the quadrant subdivisions we reserve enough space in the shadows array for that many lights. We use the length of this shadows array to validate how many shadows can go in each quadrant.

When we set the size of the atlas, we have to clear the quadrants and then resize them. But there we were reserving enough space for 1 << subdivision lights. This is not always the same as subdivision * subdivision! For the last quadrant, this led the system to think that it could store 256 shadows, when it really only had space for 64. So the 65th shadow would try to render outside the framebuffer and cause the error.

Oddly enough I caught this bug in the OpenGL renderer when implementing shadows. I remember at the time noticing it, but not doing anything in the RD renderers because it wasn't causing any issues and I didn't have time to investigate why.

Problem 2: No validation for render pass rect size

draw_list_begin is supposed to validate the size of the render pass rect. But it was missing a pair of parentheses, so instead of checking if the rect was fully inside the framebuffer. It checked if the rect began outside the framebuffer on the X-axis, but otherwise fit within the framebuffer. This bug has been present for a long time.

Not every device crashed/froze despite us allowing values that should not have been allowed as different drivers handle breaking spec differently.

[akien-mga]

If those two issues pre-exist the apparent regression, should we consider cherry-picking this to 4.2?

[akien-mga]

Thanks!

[clayjohn]

If those two issues pre-exist the apparent regression, should we consider cherry-picking this to 4.2?

I would need to make a separate PR as this one won't cherrypick cleanly. However, the crash/freeze doesn't happen in versions before 4.3 dev2. In earlier versions these issues are present, but the symptom is not