D3D: Generate HLSL from SPIRV*

[iwubcode]

This generates HLSL from SPIRV.*

There are a couple reasons this is worthwhile:

1. Reduces code requirement for Dolphin developers adding new code. They only have to maintain one shader language (GLSL).
2. Users that create shaders don't have to worry about writing HLSL (ex: post processing shaders)

The second advantage was the main driver behind implementing this feature (to prepare for the pp rewrite).

Here's the task list:

• Specialized Shaders - both D3D11 and D3D12 working
• Hybrid Shaders - both D3D11 and D3D12 working
• Stereoscopic 3D - both D3D11 and D3D12 working
• GPU Texture Decoding (Compute Shader) - both D3D11 and D3D12 working.
• Logic Op Games - both D3D and D3D12 working.
• Refactor Vulkan Implementation - done

* Unfortunately SPIRV-Cross only supports Vertex, Pixel, and Compute shaders. That means Geometry shaders still use HLSL. Luckily, there isn't much code there to support that path and there are no user written geometry shaders. But it does make things a little klunky in Dolphin code as one of the helper functions is used there and in the vertex/pixel shader so we have to handle it in a special manner.

[pizuz]

Please mind that macOS builds already include SPIRV-Cross as part of the MoltenVK dylib.

[iwubcode]

Upstream review was merged, I opened a new PR for the external separately. This change is now ready for review and testing!

[Miksel12]

Have you tested how this affects shader generation time? Best way I know to test this is by measuring frametimes while generating shaders but that could be a bit noisy.

[iwubcode]

@Miksel12 - no I have not. I did eyeball the generation during my tests and didn't notice anything excessive. I am open to someone doing some more formal tests in this area.

I wouldn't expect this to improve shader compilation times at all. DirectX still needs the shaders in its proprietary bytecode format, so converting GLSL -> SPIRV -> HLSL actually introduces a bit more overhead. The only advantage to be gained would be optimizing the SPIRV. That could come in the future but not sure if that would help or hurt compile times.

@phire has mentioned that eventually we might be able to mitigate some of this overhead by doing this at build time but it's not clear to me how that would work.

If the performance hit is too high, we can always scale back and just do this for our user-generated shaders but I was hopeful to reduce the maintenance overhead for Dolphin devs as well.

[Miksel12]

Are you familiar with naga(https://github.com/gfx-rs/naga)? They tout very fast translation: https://gfx-rs.github.io/2021/05/09/dota2-msl-compilation.html and could even replace glslang. I have no idea how difficult it is to incorporate a Rust dependency or how production ready it is so maybe this is not feasible but it sounds promising.

[iwubcode]

@Miksel12 - I hadn't heard of that but I don't think that would be a good fit. At least it doesn't look like they have any C bindings and I get the impression it's not as developed as SPIRV-Cross. If it was usable, I'd be willing to try it but not really sure I want to jump through a lot of hoops, SPIRV-Cross on the other hand is a C++ library.

---

I talked with @phire and I guess I'm going to look at adding a shader compilation time statistic as a separate PR. I will then have a directly comparable metric with master to compare this with.

[Zopolis4]

I might be missing something here but doesn't this PR still contain the changes in #10698?

[iwubcode]

@Zopolis4 - yes, at least other PRs I've seen in the past will split the external dependency out into a separate PR so that can be merged first, thus making the rest of the code easier to merge?

[Zopolis4]

No but like doesn't splitting out the external dependency mean you dont have it in the main PR? Or are you keeping it so everything builds correctly, and then will remove it once 10698 is merged?

[iwubcode]

Yeah, if the other one gets merged first, I will update this. I'm keeping the external in this code base so people are able to test, or if this one gets merged first I can just close out the other.

[iwubcode]

After using #10706 on master and on this branch, the compilation impact is in.

Compiling from GLSL -> DX bytecode (using SPIR-V) is roughly 30% slower than HLSL -> DX bytecode. This means that any shader stutter that exists would last 30% longer. I tested both hybrid and specialized shaders.

The advantage as stated earlier is that all custom HLSL in Vertex, Pixel, and Compute shaders can be dropped.

In the future, we could possibly look at writing raw SPIRV or @phire has floated the idea of generating these shaders at build time somehow. It seems most of our time spent is in the GLSL -> SPIRV stage ( > 75% of the time ). Given that we now have two backends using SPIRV (D3D, Vulkan) and one that could potentially use it (OGL 4.6), it may be beneficial to look into how to create that without needing GLSL. That is a much larger task however.

I'd be curious to hear opinions from the rest of the dev team on this.

[Anuskuss]

This means that any shader stutter that exists would last 30% longer.

I did some tests years ago and I've been using DX11 exclusively since then because it's the only backend that is stutter-free when using Hybrid (even though Vulkan gives me more FPS). So from a user's perspective this change hurts. Maybe you can regain some of the lost performance by generating the SPIR-V directly but otherwise this doesn't seem worth it (where VK = raw performance and DX11 = fastest shaders).

[JMC47]

This won't actually affect stuttering in Hybrid Ubershaders. It'll just mean that it uses the Ubershaders 30% longer in that case. If you didn't see a stutter before, it means that it was handing things off correctly already.

[iwubcode]

Nothing new really. Just pushed up some cmake changes thanks to @TellowKrinkle !

[shuffle2]

I really haven't looked at this PR at all, but is there a chance it will allow removing glslang? glslang is the slowest code to compile in the entire codebase... (not that it's super important, but it seems like 1/2 the PRs these days are just adding new Externals)

[iwubcode]

@shuffle2 - sorry to say I'll double disappoint you.

First, this PR uses glslang, now not just for Vulkan but also for DirectX to generate the SPIRV from GLSL. And then for my second disappoint, it also uses Spirv-Cross (a new external) to generate the HLSL from the SPIRV. The metal renderer that is being worked on would also do this same approach.

I thought I've heard of other emulators generating SPIRV directly but honestly don't know if that'd be possible. As I mentioned before, @phire expressed interest in trying to generate our shaders during build time which would be another option..

[Anuskuss]

I thought I've heard of other emulators generating SPIRV directly but honestly don't know if that'd be possible.

OT but that's currently on the roadmap for Cemu.

[iwubcode]

@Anuskuss - yeah. Thinking about that angle some more, I'll retract my statement. For more modern emulators it makes sense, since they are already dealing with shader instructions. For Dolphin, I'm not sure if it's reasonable. SPIRV is a low level representation of a shader and wouldn't really be nice to write or read the way Dolphin handles shaders.

[TellowKrinkle]

BTW if you don't want to have everything named TEXCOORD##, spirv-cross has a remapping system through add_vertex_attribute_remap

And side note, I played around with generating MSL directly and it looked something like this, so that's the alternative to adding spirv-cross if we want to have a Metal renderer. It requires wrapping every constant buffer access in a macro, since MSL doesn't throw all constant buffer contents into the global namespace.

[iwubcode]

BTW if you don't want to have everything named TEXCOORD##, spirv-cross has a remapping system through add_vertex_attribute_remap

Yup, thought of that but was thinking it'd be simpler as is. But maybe not. If this isn't merged soon, I will look into it, otherwise I'll do a separate PR.

And side note, I played around with generating MSL directly and it looked something like

Neat that you got that working! However, that macro heavy approach was actually what phire and I were hoping to avoid. We currently do that sort of thing for post-processing (and I had to do that even more in my rewrite) and it just makes for ugly code to read/write.

But more importantly, if a user wants to write a custom shader, it means we have to expose all those details to them. I'd prefer not to do that.

[OatmealDome]

Yeah, considering the complexity it adds, I think continuing to unify around GLSL and using spirv-cross is the best approach.

[JMC47]

Considering we have some changes on the docket that will reduce stuttering by as much (or more?) than this costs, I think we can justify merging this. It also brings us closer to a metal backend, which will help our macOS users a lot.

[Anuskuss]

Considering we have some changes on the docket that will reduce stuttering

@JMC47 Sorry for necroing (I just read it) but are we talking "ideas" here or some actual PRs I could try out? Ever since I've switched to Linux I've been stuck with OpenGL for the stutter-free experience.

For comparison, on my old setup with a much weaker CPU and Intel GPU, it looked like this (I haven't actually checked but I made some notes years ago):

Windows 10	Hybrid (FPS)	Exclusive (FPS)
OpenGL	👎 (220)	👍 (140)
Direct3D 11	👍 (230)	👍 (90)
Direct3D 12	👎 (280)	💩 (80)
Vulkan	👎 (290)	👍 (100)

But now, better CPU and Nvidia GPU:

Linux	Hybrid (FPS)	Exclusive (FPS)
OpenGL	👍 (340)	👍 (340)
Vulkan	👎 (440)	👎 (440)

I really think that the only way is to generate SPIR-V directly. Another emulator I can list which does that is Yuzu, which generates SPIR-V with Vulkan and lets you decide between SPIR-V and GLSL with OpenGL.

Regarding the data, I know I'm talking about several hundreds of FPS here. But that's only the way I test. Playing at 3x and when there's a lot of movement going on, the difference between Vulkan and OpenGL can really be playable or not.