Pyodide support

[Korijn]

I've reviewed the requirements for making pygfx compatible with pyodide. It seems like all dependencies are already either built-in to pyodide (e.g. Jinja2 and cffi), or they are pure python. wgpu-py and pylinalg are already compatible I think. The only challenge lies in pygfx in (you guessed it) freetype-py and uharfbuzz which are most likely not available in emscripten format (I think).

[almarklein]

I expect we need some work in wgpu-py too: pygfx/wgpu-py#407

Although ... you mentioning that cffi is supported ... would wgpu-native be compilable with emscriptem and would it use webGPU then automatically? Eeeh, probably not, but worth checking.

[Korijn]

I don't think so. We'd have to use the JS API to talk to the browser's web GPU implementation.

[legut2]

Okay. I was experimenting around with getting pygfx running in a browser this past weekend. I thought this would be a simple thing to do but it quickly got complicated. The first thing I tried was pyodide. I hit a wall when I tried to render.

I got fairly far and it was a messy process and you can see the code over here:
pyodide repo

You can inspect it through your browser at this
static site hosted on github pages. It computes, but doesn't render. I wanted to find a hacky way to render, but couldn't.

This involved tearing out anything text and freetype related in pygfx and building a custom wheel of pygfx. You can probably see all the code I had to comment out by opening the wheel. I think I even custom built uharfbuzz. I lost track of the many things I had to do just to get it to work as much as it does now.

With pyodide I'd usually hit this kind of error whenever initiating a renderer of pretty much any kind:

Uncaught (in promise) PythonError: Traceback (most recent call last):
  File "/lib/python312.zip/_pyodide/_base.py", line 596, in eval_code_async
    await CodeRunner(
  File "/lib/python312.zip/_pyodide/_base.py", line 410, in run_async
    coroutine = eval(self.code, globals, locals)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<exec>", line 12, in <module>
  File "/lib/python3.12/site-packages/pygfx/renderers/wgpu/engine/renderer.py", line 142, in __init__
    self._shared = get_shared()
                   ^^^^^^^^^^^^
  File "/lib/python3.12/site-packages/pygfx/renderers/wgpu/engine/shared.py", line 260, in get_shared
    Shared()
  File "/lib/python3.12/site-packages/pygfx/renderers/wgpu/engine/shared.py", line 67, in __init__
    self._adapter = wgpu.gpu.request_adapter(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.12/site-packages/wgpu/_classes.py", line 100, in request_adapter
    return gpu.request_adapter(
           ^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.12/site-packages/wgpu/backends/js_webgpu/__init__.py", line 16, in request_adapter
    raise NotImplementedError("Cannot use sync API functions in JS.")
NotImplementedError: Cannot use sync API functions in JS.

Then I tried using voila with a notebook hosted on a huggingface space to get pygfx published and running in a browser:
https://huggingface.co/spaces/devmandan/voila-dashboard-test

and I'd hit a wall with this kind of error:

Traceback (most recent call last):
  File "/home/user/env/lib/python3.10/site-packages/voila/notebook_renderer.py", line 261, in _jinja_cell_generator
    output_cell = await self.executor.execute_cell(
  File "/home/user/env/lib/python3.10/site-packages/voila/execute.py", line 68, in execute_cell
    result = await self.async_execute_cell(cell, cell_index, store_history)
  File "/home/user/env/lib/python3.10/site-packages/nbclient/client.py", line 1058, in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
  File "/home/user/env/lib/python3.10/site-packages/nbclient/client.py", line 914, in _check_raise_for_error
    raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------

import pygfx as gfx
import pylinalg as la

cube = gfx.Mesh(
    gfx.box_geometry(200, 200, 200),
    gfx.MeshPhongMaterial(color="#336699"),
)


def animate():
    rot = la.quat_from_euler((0.005, 0.01), order="XY")
    cube.local.rotation = la.quat_mul(rot, cube.local.rotation)


disp = gfx.Display()
disp.before_render = animate
disp.stats = True
disp.show(cube)

------------------


---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[1], line 18
     16 disp.before_render = animate
     17 disp.stats = True
---> 18 disp.show(cube)

File ~/env/lib/python3.10/site-packages/pygfx/utils/show.py:168, in Display.show(self, object, up)
    165     from wgpu.gui.auto import WgpuCanvas
    167     self.canvas = WgpuCanvas()
--> 168     self.renderer = WgpuRenderer(self.canvas)
    169 elif self.renderer is not None:
    170     self.canvas = self.renderer.target

File ~/env/lib/python3.10/site-packages/pygfx/renderers/wgpu/engine/renderer.py:142, in WgpuRenderer.__init__(self, target, pixel_ratio, pixel_filter, show_fps, blend_mode, sort_objects, enable_events, gamma_correction, *args, **kwargs)
    139 self.pixel_filter = pixel_filter
    141 # Make sure we have a shared object (the first renderer creates the instance)
--> 142 self._shared = get_shared()
    143 self._device = self._shared.device
    145 # Init counter to auto-clear

File ~/env/lib/python3.10/site-packages/pygfx/renderers/wgpu/engine/shared.py:260, in get_shared()
    253 """Get the globally shared instance.
    254 
    255 Creates it if it does not yet exist. This should not be called at the import
    256 time of any module. Use this to get the global device:
    257 ``get_shared().device``.
    258 """
    259 if Shared._instance is None:
--> 260     Shared()
    262 return Shared._instance

File ~/env/lib/python3.10/site-packages/pygfx/renderers/wgpu/engine/shared.py:67, in Shared.__init__(self, canvas)
     65     self._adapter = Shared._selected_adapter
     66 else:
---> 67     self._adapter = wgpu.gpu.request_adapter(
     68         power_preference=Shared._power_preference or "high-performance"
     69     )
     71 # Create logical device from adapter. There should be just one per
     72 # process. Having a global device provides the benefit that we can draw
     73 # any object anywhere. Supporting different devices per renderer/canvas
     74 # is technically possible, but would require an extra layer of
     75 # indirection in the pipeline objects (device -> environment -> passes).
     76 # So out of scope for the time being.
     77 self._device = self.adapter.request_device(
     78     required_features=list(Shared._features), required_limits={}
     79 )

File ~/env/lib/python3.10/site-packages/wgpu/_classes.py:100, in GPU.request_adapter(self, power_preference, force_fallback_adapter, canvas)
     97 # If this method gets called, no backend has been loaded yet, let's do that now!
     98 from .backends.auto import gpu  # noqa
--> 100 return gpu.request_adapter(
    101     power_preference=power_preference,
    102     force_fallback_adapter=force_fallback_adapter,
    103     canvas=canvas,
    104 )

File ~/env/lib/python3.10/site-packages/wgpu/backends/wgpu_native/_api.py:253, in GPU.request_adapter(self, power_preference, force_fallback_adapter, canvas)
    251 if adapter_id is None:  # pragma: no cover
    252     error_msg = error_msg or "Could not obtain new adapter id."
--> 253     raise RuntimeError(error_msg)
    255 return self._create_adapter(adapter_id)

RuntimeError: Request adapter failed (1): Validation Error

Caused by:
    No suitable adapter found

The code for that was a slightly modified version of this repo.

All in all, I think there is something so wrong about pygfx not being easily publishable to the web. I feel it violates the spirit of the underlying technologies (webgpu and webassembly). It's in the name of those technologies to have something like pygfx run within a web browser easily. I hope what I shared makes it easier to making it happen.

[Korijn]

It's on our roadmap for later this year. It's fairly complex because the JS API is async in various places. We have to start on wgpu-py level first and then work on pygfx. But we do have a plan and it is feasible. See here: pygfx/wgpu-py#391 and here: #615

Thank you for sharing your notes and feedback. I'll close this ticket doesn't create more confusion.

[legut2]

Thank you for pointing that out. Maybe over time I can start to contribute, but I don't think this problem is one that is good to start off with. I have a computer vision focus and pygfx scratches that visual itch I have. I'll try playing around more with wgpu-py when I get a chance to learn more about it.