Shader source strategy for multiple renderers

[colincornaby]

When I wrote the Metal renderer - I directly translated the shaders from SM1.1 assembly to MSL and created duplicate files in the repo.

Going forward we should figure out:

• Do we want each renderer to have it's own set of shaders?
• Or do we want to adopt a shader language that supports transpilation into our other backends?

There are a small amount of shaders and they are fairly static - so each backend maintaining it's own shaders is not out of the question. But shaders like the fixed function emulation ubershader do get a fair bit of changes.

We should affirmatively make a choice as to how we want to handle this issue. I think each backend maintaining it's own shaders is a decent outcome. But we should be deliberate about that.

Possible Alternatives

Since I first wrote the Metal pipeline, there have been some new alternatives we could evaluate.

Slang

https://shader-slang.org

Slang is fairly popular. But Metal support is still listed as experimental. While they do support GLSL for OpenGL - they don't list it as a focus. They also don't support D3D9.

HLSL/Apple Game Porting toolkit

https://developer.apple.com/games/game-porting-toolkit/

Apple now offers an HLSL compiler. We could standardize on HLSL for our other renderers. But HLSL support for OpenGL requires SPIR-V, which requires a modern version of OpenGL. Apple's HLSL compiler also requires modern HLSL.

Metal concerns

I think each rendering backend would have it's own issues that would need to be worked out. But for Metal specifically:

• Build time output should be an AIR archive so the runtime performance impact of compilation is minimal.
• The output and shader language should support function constants so Metal can quickly specialize the shader at runtime. (These are called specialization constants in Vulkan.)
• We should continue being able to optimize for Apple Silicon - specifically for concepts like tile memory
• The generated Metal code should be human readable
• Shader code should be easy to debug within the Metal debugger

[dpogue]

On the GL side, one consideration is that shaders can't really be "compiled" for versions without SPIR-V support, so the shader essentially has to be text. I had been considering some sort of CMake build step that would take shader source files and gzip them into binary blobs (similar to how the DirectX compiled assembly is handled).

The DirectX shaders are only used in special cases, and that's handled through the plShader system that lives in PubUtilLib/plSurface. Right now, those plShader classes are very limited to a set of "named" shaders and their implementations. Each named shader only supports a single implementation.

Currently the Metal shaders are handled as their own object libraries underneath FeatureLib/pfMetalPipeline. The Metal pipeline looks at what name plShader would be used and picks from its own set of shaders instead.

I'd like to find a way to try to make the plShader system more useful across all these pipelines.

[Hoikas]

Minor nitpick: the engine doesn't have any HLSL shaders. It does have shaders writen in shader model 1, 1.1, and 2 assembly, though.

[colincornaby]

Minor nitpick: the engine doesn't have any HLSL shaders. It does have shaders writen in shader model 1, 1.1, and 2 assembly, though.

Thanks, I'll correct. I'm guessing the other discussion should be corrected too.