Reconsidering the binding strategy

[almarklein]

I think it'd be good to discuss and think about how we want this project to evolve further.

I've been away for quite a while (for personal reasons; prioritizing my family's well-being after moving to a new city). The fresh look on the project, plus @Korijn's proposed cffi refactor, gave rise to some questions ... luring me into a bit of a rabit hole, exploring several different options. Let me share my thoughts so far.

Primary goals

I we are going to change something fundamental about wgpu-py (or move our efforts to a new lib with similar goals), we'd want to have:

• A public API that resembles WebGPU
• A backend that is stable
• Performance
• Maintainability
• Stable ABI (when using compiled code)
• Support for Pyodide
• Support for shared memory (share buffers and fences with Torch/Vulkan/Metal)

Some more thoughts and questions

I've been pretty strict on following WebGPU's spec as closely as possible, for two reasons: 1) it makes it easy for ppl to understand our API if they already know WebGPU, plus ample documentation available. 2) it should make it easier to implement the Pyodide/JS backend.

Question for @Vipitis: what is your current thought on this? I currently think that using a backend that compile to Pyodide would be preferable, because it safes us a lot of work and maintenance.

If we don't need our own JS backend, it gives us some breathing room to deviate some more from the WebGPU spec, where it makes sense. Or grow towards it more organically instead of enforcing it with codegen.

If we can support pyodide through the backend, I'd argue for just leaning into that backend and flatten the repo a bit.

Options worth exploring

---

Based on wgpu

Compared to Dawn, wgpu is certainly more of a community project. This feels nicer, but yeah ... it also feels that way because it lags behind and because wgpu-native is not getting the attention that it needs.

The Rust tooling is defintely friendlier than C++.

I think wgpu has more native-specific features.

Does anyone have any more arguments for either Dawn or wgpu?

wgpu-native wrapped with cffi (ABI mode)

Our current implementation.

• Not the best performance.
• Quite easy to maintain, since contributors don't need a compiler to dev-install wgpu-py!
• Stable ABI because there is no compilation.
• Support for pyodide needs to happen through a separate backend that calls into the JS WebGPU API (pure python js_webgpu backend using pyodide #753)
• Support for shared memory will take a while, but we may be able to get limited support via wgpu-native.

wgpu-native wrapped with cffi (out-of-line)

@Korijn's work discussed in #827. Similar to our current implementation, but a dynamic lib is compiled as a build step.

• Good runtime performance
• A bit harder to maintain, but IIUC users can still download precompiled wgpu-native so they only need a C compiler, not Rust.
• Stable ABI is possible.
• Support for pyodide via wgpu: I don't know
• Support for shared memory: still depend on wgpu-native

wgpu wrapped with pyo3 and maturin

I took a small stab at this. It bypasses wgpu-native - and it's lack of development.
Using the Rust compiler feels nice.

• Excellent runtime performance
• Likely harder to maintain since now we have Rust code in our repo. But some other parts are easier, like basic type conversion just works.
• Stable ABI is possible.
• Support for pyodide via wgpu: apparently Pyodide uses the wasm32-unknown-emscripten target, but wgpu only supports wasm32-unknown-unknown, emscripten seems not (yet) supported, also see wasm32-unknown-emscripten: unresolved __wbindgen_placeholder__ import prevents WebAssembly.instantiate() gfx-rs/wgpu#10274 (which is the exact error I got when I tried).
• Support for shared memory: much easier because we have access to HAL now.

If the Pyodide issue would be resolved, this might be my favorite option.

wgpu wrapped with nanobind

Nanobind is the successor to PyBind. It's for C++ but IIUC can also do C. I have not yet explored this option, but the result should be similar to cffi out-of-core. Except now the generated bindings are C++ instead of Python.

wgpu-native wrapped with mirror-bridge

https://chico.dev/Mirror-Bridge

Have not looked very closely yet. But wrapping C with it is probably not going to bring much benefit, and according to Claude, Dawn's C++ is not a good fit for mirror-bridge.

---

Based on Dawn

Dawn is ahead of wgpu, and battle-tested since it ships with all Chromium browsers. It has builtin support for sharing memory.

Dawn comes (from what I understand) with a performant CPU-based Vulkan backend. I see this as a big plus for running our image tests against, since the chance of it producing the exact same result is higher (maybe not when running in Docker with emulated aarch64). Plus it's a fallback that's actually feature complete!

Dawn wrapped with cffi ABI

Using the same approach as we currently do, but switching to Dawn. Not the best performance, but we keep maintainability and get a more mature WebGPU implementation, Pyodide support (probably), shared memory, and more.

Need to verify if we actually get all that. If this works, cffi out-of-core might also be feasible?

Dawn's C-API wrapped with cffi out-of-core

We might be able to change Korijn's work to go this route?

Big question: can we compile wasm wheels that work with pyodide?

Dawn wrapped with nanobind

Nanobind is the successor to PyBind. It's for C++ but IIUC can also do C.

I had Claude make a POC, but still need to take a look.

Big question: can we compile wasm wheels that work with pyodide?

Dawn wrapped with Cython

Quite similar in effect as the above, with friendlier code but more complex tooling, I assume.

---

Happy to hear more thoughts and experiences!

[Vipitis]

Hey, it's good to have you back. I am also in the process of moving, which should give me more free time for contributing and personal projects (some make us of wgpu) going forward. As I further try to incorporate wgpu and derived projects into my job and coursework.

---

Whatever decision we end up making, we should put effort into not breaking existing projects. There is more and more use of wgpu-py that I see on github, including some really large projects: here and here. So perhaps we can declare our interface in _classes.py stable?

I am very partial towards wgpu, because that is what I have learned to work with over the past years. Plus I have also been listening into the maintainers meeting(always wednesday at ~this time), where I will raise wgpu-native tomorrow. There is two open PRs that add a lot of the native only functionality (not from_hal) and also provide a yaml/json specification next to the wgpu.h header. These should open up some new opportunities, also making codegen easier.

Regarding all the different binding options, I can't comment too much as I am not familiar with rust. But I much prefer getting more into rust than C++ honestly.

Not just for pyodide, but we should improve the division for spec and native only features. Currently some native only features will just work if you include additional keys (think wireframe), are merged like limits, features and textures or need to be specifically imported from wgpu.backends.wgpu-native.extras. (I meant to write an issue to track all of the native only features we current already and not yet expose). Pyodide support via a compiled lib instead of using the ffi will need this too. It's not even clear if a compiled lib with pyodide is more performant over the js ffi - but hard to tell without having both at hand.
There is other ideas like a pyjs transpiler based on types and has a demo for webgpu even (not confused with another project called pyjs - these should also work with pyodide but I haven't tried that.
My PR branch has quite a few issues around stack switching, but I saw upcoming fixes in the pyodide main branch (3.15 should release soon) regarding stack switching specifically. I got the free weekends I need to get some of it working again and hopefully improve the codegen approach there.
If you don't want to have it in tree, I can look into breaking my approach out as a standalone repository (copying the _classes.py spec or depending on wgpu-py) to publish a pyodide only package. Which in turn could be conditionally imported by wgpu-py on emscripten. So we at least have something available until the compiled backend works?

Generally, how much future maintainance work will there be? A couple of newer webgpu features are still outdated in wgpu (like texture tiers for example). But the webgpu.h headers are meant to be stable by now. So will mostly see additions as native only features or new WebGPU extensions (which often exist as native only features before). I am sure there is plenty of performance wins to make with the current approach too. I think 3.15 includes several profiling improvements that should enable profiling into native bindings which is on my to-do list to try.

I am excited about interoperability with pytorch, so finding a webgpu solution would be ideal (although after a few hours of looking around and working towards it, I am not so sure anymore), however a wgpu solution will also be fine. If we can extend supported platforms in the future (I am myself often with windows intel-xpu locally, but deploy training runs on different linux cuda clusters). Maybe we end up being the common interface for it.

[almarklein]

we should put effort into not breaking existing projects

Yeah, good point. We could also roll a new library, but then we'd have to convince our users to move to it, or maintain two libraries instead of 1.

Not just for pyodide, but we should improve the division for spec and native only features.

I fully agree. And this is harder for features which are exposed through extra argument, or even additional values that can be used for existing arguments. Maybe we should prefix such arguments / enum-values with "native_". Then we can also move the extras into the common namespace but with that same prefix. An issue that lists all cases would be great!

Regarding the JS backend, let's just add it in this repo when it works, which should make it much easier to keep it aligned. But keep in mind that maybe we don't need it in the end. On the other hand, the fact that it is close to done, means we have more options going forward.

Related to this, I found out after my initial post that an extension module with wgpu/wgpu-native for Pyodide will not work (see gfx-rs/wgpu#10274 for details). Having the pure JS backend would allow circumventing that limitation, even though it's more code to maintain.

I'm going to explore some of the options somewhat deeper, and will report my findings here. I'm also curious to the results of the wgpu meeting!

[hmaarrfk]

Seems like if we shown dawn + piodide you would be satires satisfied right?

edit: satires -> satisfied

[almarklein]

I've tried creating some POCs with the help of bots with mixed success. But there are several decisions to be made (e.g. nanobind vs cffi vs ..) and these may have a big effect on maintenance, pyodide support etc.

[hmaarrfk]

I think you did an excellent job writing out your concerns at the top level comment. That is the kind of high quality context that one needs to get the kind of results I shared the PRs for. (I think they are good results, and could use a bit of cleanup and might be mergable).

I did tell it that I was particularly focused on performance, which is why it likely biased it toward running those benchmark experiments.

I sent Opus 5.5 on it first building my Dawn package for conda-forge, then working through integrating it within the context of wgpu-py

Claude usage

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Settings  Status   Config   Usage   Stats

Session

Total cost:            $134.08
Total duration (API):  3h 21m 36s
Total duration (wall): 13h 16m 31s
Total code changes:    4671 lines added, 111 lines removed
Usage by model:
    claude-haiku-4-5:  1.0k input, 12 output, 0 cache read, 0 cache write ($0.0011)
     claude-opus-5-5:  401.6k input, 1.2m output, 458.6m cache read, 3.1m cache write ($134.08)
Prompt cache (main):   411 requests · 100% of input tokens from cache · no misses · cold — idle 5h 55m 46s, next turn re-caches ~601.5k tokens

[hmaarrfk]

Maintenance of projects has really turned into "periodically relaunching" claude.

I guess its the fact that the tools for both coding AND maintenance have evolved.

[almarklein]

That's not how I want to maintain my software TBH

[hmaarrfk]

Ultimately I think it will be your balance to find. I don't want to push too hard on this font.

But as a "consumer" of pygfx I remember when pygfx/pygfx#1002 was merged, we were very excited, however the "maintenance" of migrating our codebase to adopt the new best practices was a rough process.

Today, I don't doubt that Opus would be able to execute the transition for us. Perhaps with similar or better code quality than what we would have done (it took us a while to understand the different queues and I still think we make mistakes with it).

What this has meant for me as a programmer, is that today, I'm much more open minded about large architectural changes in ways that I would not have been before due to the "worry of breakage".

If #839 can be useful to you as a "proof of feasibility" that is great.

[claydugo]

wgpu wrapped with pyo3 and maturin

My 2 cents is that this feels like the strongest proposal. In my limited amount of playing around with pyo3, I have found it really nice to work with.

Likely harder to maintain since now we have Rust code in our repo.

This is not nothing but conda-forge maintains feedstocks for rust, maturin ect and if you lean into pixi you can make doing things like run ALL the tests (rust+python) something like pixi run test with a task.

Heres a mockup environment

[workspace]
name = "wgpu-pyo3"
requires-pixi = ">=0.80"
channels = ["conda-forge"]
platforms = ["linux-64", "osx-arm64", "win-64"]

[feature.build.dependencies]
rust = ">=1.97.1"
rust-src = ">=1.97.1"
maturin = ">=1.15,<2"

[tasks]
test = { cmd = "cargo test --locked && python -m pytest"}

[almarklein]

True, and once you have Rust installed, things work really smooth.

This is my current favorite too. I'm wrapping some things up, and will then post an update.

[almarklein]

After considering many options and investigating consequences I propose we go with pyo3, wrapping wgpu directly.

Arguments

This approach would be very performant. Since we'd bypass wgpu-native, we can benefit from its updates much faster, and can also use the internals, e.g. for shared memory. Also, we can turn on/off wgpu's compile-time features, since we build wgpu from source. Much more control!

In terms of maintenance, this would turn this project in mostly Rust project, which definitely makes contributing for users harder. Some us us will have to be more accustomed to Rust. On the other hand, the Rust compiler will do a lot of validation, so I expect the codegen to be much simpler (only the IDL for the public API, really). And much of the code that we need is (in a slightly different form) already in wgpu-native.

pyo3 supports compiling for the stable Python ABI, so we can publish wheels on Pypi that also work for future versions.

With Emscripten 6.0.10, wgpu can also be compiled for Pyodide, which means we'd eventually only need a single backend. We'd have to wait for Pyodide to update to that version of Emscripten before we can actually make use of it though, see github.com/pyodide/pyodide/issues/6476

Some thoughts on Dawn

Using Dawn has some temping features, like ready to use support for Emscripten, even though Mark's PR shows it requires some juggling to provide a Pyodide wheel for that.

I also found the idea that Dawn can be with compiled to include Swiftshader quite appealing; it'd come with the same software backend on each platform. But we should be able to compile SwiftShader seperately and use it via wgpu also.

Dawn being more ahead is of less concern as time progresses; wgpu is quite stable also (WebGPU is currently enabled by default in FF on Win/MacOS). On the other hand, wgpu has much more native features, which may be interesting for our partly scientific user base.

A significant argument against using Dawn is that we'd introduce a split (we'd need a new name, since wgpu is obviously tight to wgpu), which would surely haunt us and cost valuable time.

And finally, Rust, although not an easy language to learn, has a more modern compiler ecosystem than C++.

wgpu vs wgpu-native

Binding wgpu-native gave us a clear C-API, which is a kind of binding we're very used to. But it is also another indirection, and we did end up spending quite some time contributing to wgpu-native. With pyo3 we'd have a richer binding with a strict compiler and more control.

Going forward

The safest route is likely to add it as a new backend, and then deprecate and remove the wgpu-native backend. We can also push the raw JS backend further, since it seems close to finished. As @Vipitis said, we don't know yet wheter our raw JS backend is faster or slower than going through wgpu.

I have some other priorities in the next couple of days, but plan to start on a more complete POC next week.

[Vipitis]

awesome, I am looking forward to this. I think it's a great choice as we aren't losing anything either. We can provide some of the mapping tricks from wgpu-native ourselves (where the webgpu.h is in sync with webgpu.idl spec but wgpu is still on native only features like subgroups and texture tiers).

I basically got the v30 branch ready as soon as wgpu-native merges and has a release I will open the PR. Maybe one tweak around limits still needed. (It's mostly new native only features, pretty stable by now).

We will need a better design for the spec/native divide so I will get the issue going.

I recall once linking this in an earlier discussion #487 (comment) and there are probably more project that already bind some parts of wgpu to python. So those will be good reference to have. There is a lot of rust ideas I will get to learn now.

For pyodide I will continue on the current approach because we need to understand the async details either way. Perhaps we will touch on it during the meeting!

[Korijn]

we'd introduce a split

Perhaps:

• wgpu-py as a pyo3 wrapper of wgpu
• dawn-py as a pyo3 wrapper of dawn
• webgpu-py as a library on top of the two backends

?

[almarklein]

What would be the benefit? If wgpu works, there is no point, I think.

If we get two implementations in the ecosystem, I fear we are going to have to deal even more with interoperability between libraries, etc. And we'd have two wrappers to maintain, that likely behave very slightly differently in subtle ways.

I'd rather focus on one wrapper, and make it solid so it becomes the thing that ppl use do do gpu in Python.