Is it really feasible to covert between WGSL and SPIR-V seemlessly?

[krogovin]

One goal I noticed was that WGSL was to be easy to covert to and from SPIR-V. However, SPIR-V is for local variable access is actually SSA:

SSA. All results of intermediate operations are strictly SSA. However, declared variables reside in memory and use load/store for access, and such variables can be stored to multiple times.

The upshot is that something like

[code]
int i;

i = 0;
float accum = 0.0;

loop()
{
if (i >= N) break;

accum += do_something(i);
++i;
}
[/code]

when converted to SSA will become a semi-unreadabe bloc of SSA with phi's. I can't see how the conversion from SPIR-V back to WGSL is feasible.

As a side note, even just the conversion of WGSL to SPIR-V is non-trivial since it will change the human written/readable form to SSA with phi's and block structure. I don't think the conversion from any programing language to SPIR-V is going to be invertible and that conversion is also not simple and will require quite a bit of work on the implementors side as well.

[dj2]

The SPIRV-Cross project has already shown it's possible to go from SPIR-V to GLSL, HLSL and MSL. So, we know the conversion from SPIR-V back to WGSL is feasible.

[pjoe]

For reference: https://github.com/KhronosGroup/SPIRV-Cross

[kvark]

@krogovin in the WGSL code you posted, both i and accum would be considered "variables". So all the reads of them are transformed into SPIR-V OpLoad and all writes - into OpStore (as a simple possible implementation).

SPIR-V's control flow and OpPhi is a non-trivial topic. SPIR-V represents the control flow as a graph, however Shader capability (which we only care about here) requires the control flow to still be structured. That means that semantically SPIR-V would represent all the same if/loop/continue unambiguously, and it's technically possible to transform it back into WGSL statements. The practical aspect of that is sorta demonstrated by SPIRV-Cross.

Of course, in the end, only well-working implementations are going to prove that point. But so far, the evidence is strong to suggest that going to and from SPIR-V is feasible.

[krogovin]

That will do one of two things I imagine:

1. a driver will then need to internally change the OpLoad's and OpStores to SSA style assignment for it to realize the variables as registers on the register file. Plenty of trickiness needed to handle the pointer fiasco too.

OR

2. a driver shader compiler will not do that making all variable reads and writes to actual memory, to something like a per-thread scratch space.

Option 1 will require a heroic shader compiler and whereas option 2 will mean the shader performs horribly.

SPIR-V Cross only claims to go from SPIR-V to human readable high level languages (but do not expect it to figure out modifying a variable, expect every SSA assignment to be a new variable).

As I think about this more, making a high level language convert to SPIR-V seems to imply, to me, that either the converter to SPIR-V needs to be quite sophisticated to do CFG and SSA magic or drop then SSA and reduce to OpLoad and OpStore and hope a driver shader compiler will be heroic... which seems chancy to me because the expectation from drivers is that the SPIR-V delivered to them will be compiled to SSA with CFG blocks and so on.

I guess to convert from WGSL to SPIR-V, an implementation could rely on LLVM to do all the heavy lifting and to write a somewhat easy backend from LLVM IR to SPIR-V; but that road makes the conversion quite one-way ish.

[dj2]

In Tint we'll run SPIRV-Opt afterwards which does the mem2reg conversion.

[kvark]

I guess to convert from WGSL to SPIR-V, an implementation could rely on LLVM to do all the heavy lifting and to write a somewhat easy backend from LLVM IR to SPIR-V; but that road makes the conversion quite one-way ish.

If we want to involve LLVM here, we failed. The conversion is supposed to be simple and fast.

For the variables, I raised the concern earlier on the Matrix chat. Now filed #622

[krogovin]

In Tint we'll run SPIRV-Opt afterwards which does the mem2reg conversion.

The upshot is that then Tint is actually doing optimizations that a compiler does, that is no longer just covert but really compiling. Looking at the source code for spirv-opt at https://github.com/KhronosGroup/SPIRV-Tools/tree/master/source/opt , I see the belly of a compiler.

I think that the goal "All features in WGSL are directly translatable to SPIR-V. (No polymorphism, no general pointers, no overloads, etc)" is not at all true in spirit at this point.

I suggest a slightly different goal which will is closer to what is really going to happen:

All features in WGSL are translatable to HLSL, MetalSL and SPIR-V; feature wise, the lowest common denominator of these.

The element "Each item in this spec must provide the mapping to SPIR-V for the construct" I suggest to drop as well since the natural mapping of variable stuff to SPIR-V is the worst possible and requires essentially a compiler technology to get a shader that would perform well.

What is really going to happen: here is a human writable/readable shading language that a browser will convert (or for the SPIR-V case compile) to a native jazz. From there do not be specific about the conversion/translatable, just the feature set. Also need to be careful on discard/branching bits between the different API's.

Alternatively, just use an existing high level shading language but add a feature or two (so if GLSL was chosen just add GL_KHR_vulkan_glsl).

I do not want to close with "use GLSL" as that is not my intent of this. The intent is to get the goal of to match with what is actually going to happen.

[dneto0]

Hey @krogovin : the prototyping work that led to the initial draft of WGSL showed that simple translation to and from SPIR-V is very much feasible. It does not require a lot of compiler smarts.

I grant you that the algorithm to go from SPIR-V to (the current draft of) WGSL is not obvious.
I'm currently in the process of landing this algorithm in production form, at https://dawn.googlesource.com/tint/+/refs/heads/master/src/reader/spirv/
There isn't much there yet. The serious fun happens when control flow is reconstructed.

[litherum]

I'm not sure what's remaining to do in this issue. I propose we close it as "no change."

[Kangz]

The question has been answered in details, closing.

[nevakrien]

Hey @krogovin : the prototyping work that led to the initial draft of WGSL showed that simple translation to and from SPIR-V is very much feasible. It does not require a lot of compiler smarts.

I grant you that the algorithm to go from SPIR-V to (the current draft of) WGSL is not obvious. I'm currently in the process of landing this algorithm in production form, at https://dawn.googlesource.com/tint/+/refs/heads/master/src/reader/spirv/ There isn't much there yet. The serious fun happens when control flow is reconstructed.

I got this version of tint seems honestly kinda strange like I am not finding any docs