Skip indirect operation for out of bounds indirect stages #9757
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.


















This fixes rendering issues in Viewtiful Joe (bug 12525). The issue was introduced in #9651, specifically by the "always apply fb_addprev and tex coord wrapping" aspect; with that, I also always applied the indirect operation. Doing that is is not hardware accurate, but skipping it (which is what this PR does) is not either; this test case (specifically the files ending in 0Y) trigger this undefined behavior on console. However, skipping gives a result that's close enough to avoid visual issues in Viewtiful Joe.
Based on the fifolog there, Viewtiful Joe tends to set the indirect configuration at the start of most objects, by clearing it for TEV stages 0 and 1 (with BP commands 10 and 11). On Object 78, it does that, and then later updates the configuration for TEV stage 0 to enable indirect functionality (at offset 242a2). It also updates the BP genmode to have 1 indirect stage enabled (at 2431c). The next object triggers an EFB copy, and then draws the copied EFB to screen a few times. However, it does not clear the indirect configuration, but does set the number of indirect stages to 0 (at 2451f); this means that TEV stage 0's previous indirect configuration applies, but now the indirect stage is not enabled. This triggers undefined behavior. I'm guessing it wasn't particularly evident on console due to the copies already being drawn multiple times, and a small matrix being used. Things went wrong in Dolphin because the indirect stage had bias enabled, and was using ITF_8 (0) as its format; this resulted in the computed coordinate being offset by 128 units, which is much more obvious.