PixelShaderGen: fix OOB indirect texcoord indices #8296
Conversation
Previously we set the indirect texture coordinate to zero, now we set the indirect texture coordinate *index* to zero. This fixes the ripple effect of the Mario painting in Luigi's Mansion.
Tilka
changed the title
| } | ||
| // TODO: are out-of-range values set to zero or clamped? | ||
| // TODO: same thing probably with direct texcoord indices? | ||
| if (texcoord >= uid_data->genMode_numtexgens) |
There was a problem hiding this comment.
What shall happen if uid_data->genMode_numtexgens == 0 ?
|
Is there a hero willing to write hardware tests so Mario can comfortably sit in his painting with the privacy blur on? Or are you really gonna let Luigi stare at him like this. |
|
@dolphin-emu-bot rebuild |
|
JMC is right! We need a hero that can test this. Luigi has been staring at Mario for way too long! |
|
Hardware tested (permalink to this version of the test). This fix is semi-correct, but I got some additional glitchy behavior on console (one of two triangles seems to use the previous scale and had garbage data that changed on each frame when there was no previous texture). That extra glitchy behavior doesn't seem to apply to Luigi's Mansion. Here is a comparison of what I got on hardware and with my reimplementation of this change (which I did in the software renderer). Note that the painting likely does this by accident; I analyzed it a bit and determined that they have a 64 by 64 noise texture as texture 1 that they meant to use, but they set the number of tex gens to 1 so they use tex coords for the 128 by 128 mario texture. If tex gens is set to 2, a more coarse distortion happens (even without the fix here), which is what Nintendo probably intended to do. When I tested 0 tex gens, I got a completely black image on console, as the direct texture didn't render. Dolphin renders a solid color (that wasn't black and changed for different textures) in that case. That case probably should just get an assertion, as I don't think any game will intentionally render a texture without texture coordinates. I still need to hardware test whether it clamps or sets it to 0, and what happens with direct coordinates. |
|
Superseeded by #9651. |
Previously we set the indirect texture coordinate to zero, now we set the indirect texture coordinate index to zero. This fixes the ripple effect of the Mario painting in Luigi's Mansion (issue 11462).
Needs some hardware tests.