Implement Zero-Configuration Resolution Scaling

[riperiperi]

Click for a surprise.

A note to the users

If you're using a 1080p screen, I recommend 2x scaling (don't go above, you will lose super sampling). For 4K, I recommend 2x or greater - though you should only go higher when your GPU can definitely handle it. Remember, 4x (8K) is 16x the switch's pixel count!

All of this is very much a work in progress and will improve over time. In cases where the scaling fails, your game will likely just run at 1x. If your game has some weird issue when scaled up (like LM3's black screen) (fixed!), then try booting with native res and switching when ingame - it may work. Make sure to point out edge cases so that they can be fixed in future!

If your target game has a form of anti aliasing, you probably want to make a mod to disable it so you can get the raw upscaled resolution.

Zero-Configuration Resolution Scaling

This PR introduces a feature and configuration option that lets you scale compatible render targets in a game using a single multiplier value, which you can set via a combo list from 1x to 4x... or if you're feeling brave, any floating point number.

The goal is to scale shadows and screen-bound render targets using the scaling factor.

https://gyazo.com/4a3cac36fadba7d4943ec79d1e9bb55d

You can change this value while emulation is running to get a feel for how it affects game graphics and performance. The changes are applied instantly, so have fun with it.

Method

Block chain AI machine learning on the cloud. Just kidding, it's really simple.

Any Texture2D or Texture2DArray format texture is immediately eligible for scaling. When the texture is taken as a render target and it is still eligible (or its first request is as a render target), its host texture is copied to a new one with the configured scale.

Textures have a variety of ways they can become blacklisted from scaling. Any texture that is not of the two formats specified above is immediately blacklisted, and any texture that is a view of a blacklisted texture will also become blacklisted itself.

Since textures can become views of each other after they are created, this means that a texture could be scaled before it becomes a view. Textures that are modified on the CPU memory after their creation or flushed back to it also become blacklisted before promising the change. When a texture is blacklisted after it is scaled, it is copied back to a host texture with 1x scale. If this happens to a view, its parent will be scaled to 1x and all views will be recreated and blacklisted.

When rendering to a texture, we need to make sure that the scales of all bound targets are equal. To facilitate this, all eligible targets are scaled, we ensure that all scaled targets match the scale in the configuration (that can be changed at runtime, mind), and if one target is blacklisted, then all of them are. This scale is used for a few things:

• Scaling all Viewports
• Scaling all Scissor Rectangles
• Sent to the Fragment Shader (fp_renderScale[0])

When using a scaled texture in a shader, thankfully there are only a few cases in which the scale of the texture can negatively affect anything:

• gl_FragCoord.xy: This is in screen space. As in, it is measured in pixels. If we render to a larger target, the values returned by this will be multiplied by the scaling factor, whereas the shaders that the game uses will be expecting the original, unmodified scale. To solve this problem, simply divide by the scaling factor (fp_renderScale[0]). Problem solved.
• texelFetch: This method of sampling a texture samples using an integer vector. The unit is pixels. Repeat what I said above, but this time we need scaling factors for each texture passed to the shader. The texture descriptor index is used to index the renderScale array, which contains scales for bound textures starting at index 1.
  • There are cases where we don't currently allow scaling for shaders that use this feature. See below for details.

Related Gpu Changes

A few changes were put into place to facilitate this change, and a few others that I'm planning or could be useful in the future. There were also a few texture related bugs that I ran into when ensuring runtime switching worked as intended.

• Fixed a bug where scaled copy's source/destination region would be affected by targets previously bound to the copy framebuffers (eg. an old depth stencil was active during a future copy of a colour texture of different size). Now, they unset their texture after the copy.
• Fixed a bug where transforming a texture to a view after its creation would not set its _firstLayer, _firstLevel information.
• Added FeatureFlags to the IR stage of translation, for consumption in the CodeGenerator. These will typically be used to change the declarations used in the shader on the backend. Here, we use it to add the renderScale uniform and the TexelFetchScale helper when needed.
• Added TextureUsageFlags to texture descriptors, set based on how they are used in the texture, and intended for use when the texture is bound. This PR sets and uses a flag to blacklist textures that use texelFetch in unsupported situations.
• Width and Height for the host texture now reflects its scaled width and height.

Compatibility notes

• Games that use spatially aware effects aggressively (such as Xenoblade's AO) will likely only work correctly with power of two scales (1, 2, 4) - with others they will run into unusual issues. Games that use texelFetch will run into issues in general with non integral scales. With the games I have played, these issues are few and far between.
• Games that use a form of Anti Aliasing will still think they are running at 1x, and will likely blur the image. For these games (such as xenoblade... again) you should use a LayeredFS mod to remove the AA. For completeness, I have attached a mod for XC:DE to this PR.
• Scales of less than 1 can reduce texture quality on rendered textures as well. You have been warned!

Drawbacks

While every game is technically supported, there are cases where the game will fall back to 1x on targets you might not want to, and the game might have some graphical issues in the worst case. Here's some specific information:

• Only non-indexed textures on the fragment+compute shaders are fully supported for texelFetch scaling. If a texture outside of this category uses texelFetch, the texture will be blacklisted for scaling, as described above.
• Bindless textures are not fully supported, and do not have a blacklist fallback. If they are used with texelFetch, the scale on the texture will be incorrect.
• If a game uses a render target to store data (such as the output of a fragment shader DXT compression algorithm), scaling will not work correctly on it. You can only hope that it will blacklist and render the target again at a later point. This is a theory as I haven't tested any games that do this.

Screenshots

Here are some pictures at 4K. A lot of these are taken at 4x (8K) downsampled to 4K, which results in a supersampling effect on NVIDIA. Note that you should not run at more than double your output resolution if you want a supersampling effect, so use 2x for 1080p.

Astral Chain Comparison


(LM3 must start in 1x scaling mode, then you can change the scale from settings ingame. This is being investigated :) )

Finally, here's the "remove aa" mod I made for Xenoblade, so you can replicate these results (need to merge mageven's PR)
XenobladeDE remove-aa.zip

[Thog]

lgtm apart from what I listed. Also could you edit the Config.json for debug builds to reflect the change of version number?

[riperiperi]

lgtm apart from what I listed. Also could you edit the Config.json for debug builds to reflect the change of version number?

Ah, I forgot there was also a version number in there.

[riperiperi]

Addressed. I also included a link to a comparison tool for one of the screenshots.

[jduncanator]

Nice work, as usual! Even I’m surprised how seamlessly this works in the games I’ve tested! 👍🏼

[Thog]

lgtm

[gdkchan]

lgtm, thanks, great work! I tested around 10 games with it (before the latest changes) and found no problems with it, except for one game where it didn't have any effect, and a blue line appeared on the top after enabling it. The game was Kirby Star Allies.

[jduncanator]

See our blog post here for more information and some mouth watering screenshots 🔥

[TibiaZ]

This is so good! Great job man! 👏🏻👏🏻