-
-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gpu: experiment: pure-Zig WebGPU implementation #133
Comments
Very interesting! I think chances to add switch/PS5 support is a big win for zig-based webgpu implementation. |
i don't know how big dawn codebase is, but we may can start work separately on dawn fork and ziggify it |
Dawn makes heavy use of C++ features and patterns. If we have to write a webgpu impl, then so it be from beginning. That would be more practical. Converting such a large project may generate more problems than solutions. @alichraghi |
The thing that attracted me to dawn/tint in the first place was tint. All of my engines are hampered by the shader translation and compilation pipeline, and by the heaviness and fragility of the LunarVG/Khronos toolchain. Dawn per se was less interesting to me, because I've got backends to target native or webgpu through an abstraction anyway. I wanted a relatively lightweight and easy to integrate library for online shader compilation, much as we enjoyed for decades at this point with GLSL and HLSL. Tint can definitely do that, especially if you nop out the SPIRV optionals, and just have Tint emit SPIRV without optimization or whatever. I went down the road of spinning up dawn in my engine in order to have a completely testable and verified system from tint -> gpu in engine, before shunting tint over to my own abstraction, with dawn being relegated to one of the backends. Want to make sure things work in principle, before changing over systems in a major one-way go-no-go transition. I therefore register interest in the idea of spinning the tint build as its own zig-build independent of dawn, just like what was already done for mach's wrapping of glfw, which would also support the development you're proposing here of a wgpu-zig effort. I'd further propose that tint the lib and tint the CLI are independent targets in that build. |
Here's the new package structure I am thinking of:
The pure Zig experimental implementation (not sure what to call it yet, suggestions?) could then just implement the same interface and mostly stay out of the way. |
@meshula Interesting thoughts. One challenge with splitting out Tint from Dawn is they need to be kept pretty closely in sync, and also share a number of dependencies with Dawn. It's not impossible, but I do think it'd add a fair amount of overhead to keep the two in sync, ensure we aren't duplicating building of the shared dependencies, etc. Given that, I'm curious: how strong is your use case for using Tint separately from Dawn? How important is having Tint separate from Dawn as a dependency (sounds like your project may have a Dawn backend, too?) |
See #133 (comment) Helps #109 Helps #133 Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
See #133 (comment) Helps #109 Helps #133 Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
The very top of my pipeline starts with a parser that compiles a program for my render VM. The parser emits, by talking to the backends, state objects, that the VM procedurally combines at runtime to render frames. Currently I use "magic" (a set of tools and operations that I really don't like) to turn the input file into GLSL, MSL, or SPIRV, as well as the program for the VM. I want to eliminate the magic shader translation part in favor of an online conversion; having my pipeline programs express their shaders in WGSL instead of GLSL, and output GLSL, MSL, HLSL, or SPIRV as appropriate would make all that go away. I'm working on integrating Dawn as a backend, but irrespective of how Dawn uses Tint, my plan is to use Tint at the front of the pipeline, independently of Dawn. That's work in progress, and currently built via CMake. you can see what a pipeline program looks like here, this example codes the shaders in GLSL: https://gist.github.com/meshula/335b39960e9a0af9f4eab0c2638e84ec and you can probably guess the structure of the renderVM from that. It's a custom VM, implemented via a vector of C++ closures, rather than as a byte code interpreter. |
I can definitely see the dependencies concern. It wouldn't be so important to split out a separate tint project ~ just as you suggested options on the dawn/tint build, options for no-dawn, no-spirvtools, would be pretty close to what I'd like to use, and what I'm currently trying to craft in my cmake files. I'm very convinced about zig-build as a path forward, away from cmake to a large degree for my work, which is why I'm here in your issues bugging you so much ;) |
That makes sense, and should be pretty easy to achieve! |
I've done some local work on a pure-zig WebGPU implementation based on Vulkan. Vulkan is supported on Linux and Windows natively, and on macOS through MoltenVK, which makes it a nice target for this. It's also relatively straightforward to wrap, since most of the WebGPU APIs map cleanly onto the equivalent Vulkan APIs. It's incomplete, but I'm happy to publish the code if you think it might be helpful as a starting point. |
@silversquirl I'd absolutely love to see your code there if possible! Sounds like you've gotten a bit further than I have honestly |
I've pushed the code here. (Ignore the readme; the zig implementation evolved out of a wgpu wrapper) It's old, so won't build on latest Zig. I also have some local work that I think might've actually gotten some very basic examples building, but I'll need to fix it up to get it running again. Will do that later. |
This looks quite good honestly! I just cloned it. The repo is missing some files needed for compilation. Its not just a matter of old code. |
See hexops/mach#133 (comment) Helps #109 Helps #133 Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
See hexops/mach#133 (comment) Helps #109 Helps #133 Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
This issue is for tracking an experiment for a pure-Zig WebGPU implementation.
Tradeoffs of using Google's Dawn
Today Mach relies on Dawn (Google Chrome's WebGPU implementation) as it's graphics abstraction layer, specifically we maintain a fork of Dawn and we do a lot of heavy lifting in
mach/gpu
to translate all of Dawn's build configuration files to Zig's build system, maintain native system SDKs, etc.This has several benefits:
zig
compiler, we already have cross-compilation working for macOS and Linux, Windows is not far off.There are drawbacks:
Overall, Dawn is a battle-tested production-worthy WebGPU implementation. There are good and bad aspects to that.
Tradeoffs of using gfx-rs/wgpu-native?
My assessment is that gfx-rs is a quite strong WebGPU implementation, likely to be on par with Dawn in the future, but overall compile times are still slow, cross compilation would be harder, and I do not want a hard dependency on a Rust toolchain.
The case for a pure-Zig WebGPU implementation
Of course, it cannot be understated that this is still a massive undertaking. And so:
Lastly, we will still have Dawn as an option - potentially even with binary builds to work around the compilation speed issue - so that one can just flip a build switch and go between the pure-Zig or Dawn implementation.
How this will work
mach/gpu
expose a Zig WebGPU interface (similar to thestd.mem.Allocator
interface) which can plug various implementations:webgpu.h
-backed implementation.Such an interface is useful for many reasons: one could implement a WebGPU interface that wraps another and provides API tracing/perf measurements, record/replay, serializing over a network, etc.
I've began toying with a Metal implementation (not very far at all, just far enough to realize how large an undertaking this is) and will work on completing #1 so we have "something" in place.
Outcomes
It's very possible we learn this is too much work and/or not worth it. In such a case, most of it would be scrapped!
The text was updated successfully, but these errors were encountered: