rsx: Improve shader decompiler output

[kd-11]

• Rewrites the vertex decoder to emit much simpler and smaller code which in turn drastically speeds up link time. On GCN GPUs this can result in almost 2x speedup of shader link time which means less time wasted waiting for shaders to compile. Resulting bytecode is almost 30% smaller as well, which may help performance on low end GPUs and/or IGPs.
• Reimplements fp32->fp16 casting to use GPU native instructions. This seems to work on my AMD card correctly and still preserves INF/NaN special encoding after the cast. 10-15% instruction count savings.
• Use intrinsics wherever possible to take advantage of native hardware instructions. Replaces many shift-and operations with a single bfe and shift-or with bfi. Generates slightly smaller code but more importantly - it is much cleaner and easier to read.

To Testers: Just check if games are working the same as before. While we already found some bugs and fixed them with internal testing, bugs may still lurk.

[kd-11]

Relegated to WIP, looks like I broke something when doing squash/rebase.

[kd-11]

Regressions fixed.

[AniLeo]

i7-6700HQ, HD 530

Render: Vulkan
Driver: Mesa 20.1.7 (anv)
SPU Block Size: Mega
Shader ASynchronous Mode

Test performed with clean emulator shaders_cache and clean .cache/mesa_shaders_cache, SPU code cached beforehand. Time from first shader to last shader compialtion without moving the character.

Persona 5	Before	After
School Corridor (190 shaders)	62s	51s
Backstreets (164 shaders)	62s	47s

[Kravickas]

NHL Legacy

Black screen in cutscenes with UI rendering.
Old cache, new cache with this PR tested
Master is ok

RPCS3.log.gz
RSX Capture https://drive.google.com/file/d/13kuKqLya-NNh6fgxYQHMS29kYb9qeEFj/view?usp=sharing (in open rsx capture it doesnt show black screen with ui, it shows like first frame ingame with correct captured ui)

[kd-11]

@Kravickas Capture opengl renderdoc with shader compiler set to legacy (single thteaded). Also remember to set 'use opengl legacy buffers' debug option.

[Kravickas]

Weeell OGL works :/

[kd-11]

You can capture vulkan if you have amd polaris or nvidia pascal, but if opengl works then it is likely not a problem with the shaders as code generator is shared.

[Kravickas]

I dont have either, so if no one find any bug, dont hold the merge because of that.