rsx: Implement shader interpreters

[kd-11]

Overview

This PR adds shader interpreters to rpcs3 to take over for the normal rendering pipeline when no shader is available and the compiler has yet to finish generating one. In this case, the interpreter is invoked to render the missing graphics while the compilation is ongoing in the background.

How to test

A new config option is added called "Shader Interpreter Mode". It takes 3 values, "Enabled", "Disabled" and "Forced". For this initial testing phase, it will default to "Disabled" for reasons I will discuss briefly which means everything should work the same as master. "Enabled" enables hybrid rendering and "Forced" is for developers only and redirects all rendering to the interpreters. Note that "Forced" mode will likely pin your GPU usage at 100% at very poor performance.

Known issues

Currently there are two major problems keeping things from moving forward:

• Due to how vulkan works, we have to either seed all the possible pipeline combinations or maintain a second shader cache for the interpreters. This is unreasonable; it makes more sense to restructure the one compiler system we already have instead of maintaining a second async compiler system. This is a major blocker to having this feature enabled as it can cause a long freeze the first time interpreter is invoked. Note however that this is not persistent stutter, it will typically be one or two very long freezes the first time the interpreter system is invoked and the problem goes away on subsequent playthroughs.
• Performance is not the best, although recent optimizations have made the interpreters viable to begin with. I am working on this, but it requires some re-structuring of the rpcs3 code before this code can be added in.

Notes

Accuracy of the interpreters is intentionally kept lower than the properly recompiled shaders. This is to keep performance from taking too much of a hit.

[xddxd]

Works like a charm on GT5 BCES00569 with Shader Interpreter set to Enabled. When the shaders are being interpreted performance is low as expected, but the graphics are all there. Once the shaders have been compiled (or some specific heavy shader?) performance goes back to normal. RX 580 on Windows 10.

[JohnHolmesII]

Well I fired up Demon's Souls on my laptop just to see. You weren't kidding about those long pauses!
But yeah, everything Just Werked:tm:. Very impressive stuff. On RADV + iGPU.

[toccata10]

Archlinux, r9 fury, mesa.
Teken Tag Tournament 2 [BLES01702]
With Ogl: works rather well: slows down at the beginning of a stage but everything is displayed (otherwise, characters start naked/have black textures on them). Nice improvement.
With Vulkan (radv), after character selection, the music continues but the screen is black.
Here's the log:
RPCS3-TekenTag2.zip

[ghost]

GOW OF WAR 3 all Games Crash and boot compiler Cache New

RPCS3.log

[kd-11]

@toccata10 Are you using ACO? ACO is known to have problems with complex shaders.

[toccata10]

Yes, I was using ACO. With the traditionnal llvm radv, it works fine on teken tag tournament 2 (and a few other games I tested). Some stalls of course, but looks like a very nice improvement.

[toccata10]

On SoulCalibur IV [BLES00296], with Ogl, with hybrid: the screen displays either garbage or is dark.
I have to log on another screen. The dmesg contains things like:
amdgpu 0000:01:00.0: GPU fault detected: 147 0x0ac90802 for process rpcs3 pid 1827 thread rpcs3:cs0 pid 1852
I'm always a bit surprised that a problem in a game crashes the entire graphical sytem (I then have to reboot), but it probably means that amdgpu driver has still some room for improvement...
Anyway:

• ogl async multithreaded is still ok

• vulkan with hybrid is ok
  Here comes my dmesg and the rpcs3 logs:
  gpu3.zip
  RPCS3-sc4.zip