integrate with ipython event loop

[schlegelp]

Hi! First off: thanks for your work on this project! I'm a long-time user (and fan of) vispy which I'm using as the backend for an interactive 3d viewer in navis - the interactivity is super handy for quickly inspecting data. I've been itching to take pygfx for a spin since you started working on it and with things starting to stabilize I ran a few recently experiments. Probably preaching to the choir by saying this but the performance is out of this world! The geometries, scene setup and the example gallery in particular are very intuitive.

The one thing I've been bashing my head against is how to integrate pygfx with the ipython event loop:

What I want to be able to do is to spawn the window from an iPython terminal without blocking the REPL. For vispy it was sufficient to just run e.g. %gui qt6 to make the canvas non-blocking. For pygfx the same trick doesn't work. I tried various things based on iPython's docs on event loops but to no avail. By now I think I'm on the entirely wrong track - I'm not even sure whether the issue is with pygfx or more like on the side of PyQt6. FWIW, I also tried glfw as window manager but that doesn't seem to be playing well with ipython since it is hogging the main event loop.

Here's a minimal working example to illustrate:

# Integrate QT6 in the ipython event loop
%gui qt6

# Run the imports
from wgpu.gui.qt import WgpuCanvas
import pygfx as gfx

# Prepare something to render
cube = gfx.Mesh(
     gfx.box_geometry(200, 200, 200),
     gfx.MeshPhongMaterial(color="#336699"),
)

# This spawns the window and the REPL is still responsive
canvas = WgpuCanvas(size=(1000, 800))

# Initialise renderer
renderer = gfx.WgpuRenderer(canvas)

# Running `gfx.show()` starts the rendering and makes the REPL unresponsive
gfx.show(cube, canvas=canvas, renderer=renderer)

I'm probably missing something obvious but right now I'm a bit stumped and I was hoping you could point me in the right direction?

Thanks in advance!

Best,
Philipp

[kushalkolar]

I use it with %gui qt all the time, you have to make your own canvas, renderer, scene, camera, controller etc. and not use pygfx.show().

simple example using https://github.com/pygfx/pygfx/edit/main/examples/feature_demo/geometry_image.py

import imageio.v3 as iio
from wgpu.gui.auto import WgpuCanvas, run
import pygfx as gfx


canvas = WgpuCanvas()
renderer = gfx.renderers.WgpuRenderer(canvas)
scene = gfx.Scene()

im = iio.imread("imageio:astronaut.png")[:, :, 1].copy()

image = gfx.Image(
    gfx.Geometry(grid=gfx.Texture(im, dim=2)),
    gfx.ImageBasicMaterial(clim=(0, 255)),
)
scene.add(image)

camera = gfx.OrthographicCamera(512, 512)
camera.local.position = (256, 256, 0)
camera.local.scale_y = -1

def animate():
    renderer.render(scene, camera)
    canvas.request_draw()

canvas.request_draw(animate)

And then change stuff:

image.material.clim = (50, 250)
image.geometry.grid.data[::2, ::2] = 255
image.geometry.grid.update_range((0, 0, 0), image.geometry.grid.size)

[almarklein]

Hi @schlegelp! Thanks for reporting this! It's definitely our intention to work well in interactive environments, but things indeed don't seem very smooth yet on IPython,

and not use pygfx.show().

This is probably because it calls the run() method. We should probably check there, whether the GUI event is already running.

I would also expect %gui asyncio to work fine (with the glfw backend), but I just tried and it looks like we need a few tweaks to our code 😄

[kushalkolar]

Actually the glfw backend is non blocking without any magic commands (at least in jupyter on my end). 🤔 If you use %gui qt I think you'll have to explicitly import and use the QWgpuCanvas.

[almarklein]

at least in jupyter on my end

It looks like this is not the case in IPython (from the terminal) though. Even when I %gui asyncio there is no running asyncio event loop 🤔 . I'll have to dig a bit deeper.

The goal is that from wgou.gui.auto import WgpuCanvas produces a a canvas for the backend that makes the most sense.

From what I can see now we need some work in these areas:

• The gui selection logic will need to check with ipython/jupyter what GUI is integrated (rather than only looking at imported modules).
• Depending on the interactive environment we're in, we may need to dynamically start an event-loop when a canvas is created, to make the canvas interactive.
• The run() methods should become non-blocking when a loop is already running.

[kushalkolar]

It looks like this is not the case in IPython (from the terminal) though. Even when I %gui asyncio there is no running asyncio event loop 🤔 . I'll have to dig a bit deeper.

Looked at this with the example I posted above:

• in jupyter, works without %gui asyncio.
• in ipython, works only with %gui asyncio, can create multiple canvases and they're all non-blocking.

If it helps my ipython package versions, I'm on latest pygfx@main, wgpu-py v0.14.1 (release)

glfw                          2.5.9
ipykernel                     6.22.0
ipython                       8.12.0

[gviejo]

Hi all,

Coming from Fastplotlib and following Kushal advice, I am posting this error I have on my Mac when trying to use glfw in ipython. The command %gui asyncio doesn't change anything.

Using python 3.11, setup is

glfw==2.7.0
ipykernel==6.29.3
ipython==8.22.2
wgpu==0.15.0

I tried to do :

import fastplotlib as fpl
plot = fpl.Plot(canvas='glfw')

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[4], line 1
----> 1 plot = fpl.Plot(canvas='glfw')

File ~/fastplotlib/fastplotlib/layouts/_plot.py:48, in Plot.__init__(self, canvas, renderer, camera, controller, size, **kwargs)
     12 def __init__(
     13     self,
     14     canvas: Union[str, WgpuCanvas] = None,
   (...)
     19     **kwargs,
     20 ):
     21     """
     22     Simple Plot object.
     23 
   (...)
     46 
     47     """
---> 48     super(Plot, self).__init__(
     49         parent=None,
     50         position=(0, 0),
     51         parent_dims=(1, 1),
     52         canvas=canvas,
     53         renderer=renderer,
     54         camera=camera,
     55         controller=controller,
     56         **kwargs,
     57     )
     58     RecordMixin.__init__(self)
     59     Frame.__init__(self)

File ~/fastplotlib/fastplotlib/layouts/_subplot.py:69, in Subplot.__init__(self, parent, position, parent_dims, camera, controller, canvas, renderer, name)
     27 """
     28 General plot object that composes a ``Gridplot``. Each ``Gridplot`` instance will have [n rows, n columns]
     29 of subplots.
   (...)
     64 
     65 """
     67 super(GraphicMethodsMixin, self).__init__()
---> 69 canvas, renderer = make_canvas_and_renderer(canvas, renderer)
     71 if position is None:
     72     position = (0, 0)

File ~/fastplotlib/fastplotlib/layouts/_utils.py:80, in make_canvas_and_renderer(canvas, renderer)
     76         raise ImportError(
     77             f"The {canvas} framework is not installed for using this canvas"
     78         )
     79     else:
---> 80         canvas = CANVAS_OPTIONS_AVAILABLE[canvas](max_fps=60)
     82 elif not isinstance(canvas, (WgpuCanvasBase, Texture)):
     83     raise ValueError(
     84         f"canvas option must either be a valid WgpuCanvas implementation, a pygfx Texture"
     85         f" or a str from the following options: {CANVAS_OPTIONS}"
     86     )

File ~/miniconda3/envs/fastplotlib/lib/python3.11/site-packages/wgpu/gui/glfw.py:112, in GlfwWgpuCanvas.__init__(self, size, title, **kwargs)
    111 def __init__(self, *, size=None, title=None, **kwargs):
--> 112     ensure_app()
    113     super().__init__(**kwargs)
    115     # Handle inputs

File ~/miniconda3/envs/fastplotlib/lib/python3.11/site-packages/wgpu/gui/glfw.py:535, in ensure_app()
    533 glfw.init()
    534 if glfw._pygfx_mainloop is None:
--> 535     loop = asyncio.get_event_loop()
    536     glfw._pygfx_mainloop = mainloop()
    537     loop.create_task(glfw._pygfx_mainloop)

File ~/miniconda3/envs/fastplotlib/lib/python3.11/asyncio/events.py:681, in BaseDefaultEventLoopPolicy.get_event_loop(self)
    678     self.set_event_loop(self.new_event_loop())
    680 if self._local._loop is None:
--> 681     raise RuntimeError('There is no current event loop in thread %r.'
    682                        % threading.current_thread().name)
    684 return self._local._loop

RuntimeError: There is no current event loop in thread 'MainThread'.

[almarklein]

The issues will be resolved in pygfx/wgpu-py#478. In particular, glfw runs as expected, and in Jupyter the %gui qt is honored.

Unfortunately, it is not possible to use the glfw backend interactively in IPython. IPython does have some sort of asyncio support, but only runs a loop briefly when calling code with await. It does not have a running asyncio loop that we need.
So on (classic) IPython you better use %gui qt (or start with ipython --gui qt).

On jupyter console the story is a bit shitty in a different way 😅. Since the code runs in a Jupyter kernel, there is no way to distinguish it from a notebook (probably because it can actually have both a notebook and a console attached to the kernel). So in jupyter console and qtconsoleyou should either use %gui qt. Since there always is a nice asyncio loop here, you can also use glfw, but have to force it with e.g. WGPU_GUI_BACKEND=glfw jupyter console.

The PR also adds docs that explain what I just said.

[almarklein]

Closing, as the pr is merged. See the docs here: https://wgpu-py.readthedocs.io/en/latest/gui.html#using-wgpu-interactively

[schlegelp]

Thanks!