VideoCommon: Fix color channel logic when per-pixel lighting is in use #10164
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Fixes bug 12698.
This was broken in #10012 (specifically by 06579e4 and c3dec34) when I attempted to simplify the logic incorrectly.
For color channel 0, this is approximately the code that is generated prior to that PR:
Here is a table of results:
vertex_color_0=>rawcolor0float4(0, 0, 0, 0)float4(1, 1, 1, 1)-vertex_color_0=>missing_color_valuewould also workfloat4(0, 0, 0, 0)vertex_color_0=>rawcolor0LightingShaderCode(vertex_color_0)=>LightingShaderCode(rawcolor0)LightingShaderCode(vertex_color_0)=>LightingShaderCode(missing_color_value)LightingShaderCode(vertex_color_0)=>LightingShaderCode(missing_color_value)For color channel 0, this is approximately the code that is generated after that PR:
Here is a table of results:
vertex_color_0=>rawcolor0rawcolor0float4(0, 0, 0, 0)float4(0, 0, 0, 0)LightingShaderCode(vertex_color_0)=>LightingShaderCode(rawcolor0)LightingShaderCode(vertex_color_0)=>LightingShaderCode(rawcolor0)LightingShaderCode(vertex_color_0)=>LightingShaderCode(missing_color_value)LightingShaderCode(vertex_color_0)=>LightingShaderCode(missing_color_value)As you can see, the results disagree in several places.
For color channel 0, this is approximately the code that is generated with this PR:
Here is a table of results:
vertex_color_0=>rawcolor0float4(0, 0, 0, 0)vertex_color_0=>missing_color_valuefloat4(0, 0, 0, 0)vertex_color_0=>rawcolor0LightingShaderCode(vertex_color_0)=>LightingShaderCode(rawcolor0)vertex_color_0=>missing_color_valueLightingShaderCode(vertex_color_0)=>LightingShaderCode(missing_color_value)These also don't match the old behavior, but now the pixel shader is always getting the raw color, never a pre-lit value.
In addition to fixing the darkness in Xenoblade Chronicles, this also makes the debug cubes in Super Mario Sunshine have the same brightness with per-pixel lighting enabled as with it disabled. (This only shows up with the fifoplayer;
missing_color_valueis set to 0 by gameini when launching the game normally.) Here are images from both games, along with the CRC32 hashes of those images to more easily identify distinct results:2B87C475
2B87C475
2B87C475
733708B3
6DF359AE
49E8A160
378454C1
378454C1
378454C1
6FC10472
64A4CC02
6FC10472
Both specialized shaders and ubershaders give exactly identical results for all of these.