Vaex process stuck in the Spyder IPython console (only)

[erwanp]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset (fixed the problem with inline graphics, see below)
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Spyder IPython console process becomes stuck when computing a mean with vaex.
Does not happen with other IDEs / Consoles.
Somehow, it may be related to the output graphics mode of the Spyder iPython console (see below)

What steps reproduce the problem?

Minimal error :

import numpy as np
import vaex
df = vaex.from_dict({"E":np.random.rand(1000)})
print(df)
print("before EQ mean")
E_bins = df.mean("E", binby="E", shape=10)
print("after EQ mean")       # 👈 I never get to here

Script works well if

• ran from an Ipython ( 7.22.0) console in the terminal
• ran from Python in the terminal
• ran from VS Code
• ran from Jupyter Lab (3.0.14)
• ran in Jupyter QtConsole

As pointed out by @minouHub , opening a new Spyder IPython console unlocks the process. If console has been "unlocked", I can even re-run the script another time before it gets stuck again. (?)

Last thing : although no graphics are plot anywhere in this example, this problem appears when my iPython console has Qt5 output graphics. It does not appear with Inline graphics (tried both consistently 3 times)

Versions

spyder
spyder-kernels 2.0.5 py38haa95532_0
ipython 7.22.0 py38hd4e2768_0

on Windows.

Spyder installed through Anaconda

• Spyder version: 5.0.5 py38haa95532_2
• Python version: 3.8.8
• Qt version: 5.9.7 vc14h73c81de_0
• PyQt version: 1.9.0
• Operating System name/version: Windows 10

[steff456]

Hi @erwanp,

Can you upgrade your Spyder to 5.1.1 and try it again? I'm in the latest version and I'm not able to reproduce this issue.

[erwanp]

Same problem with Spyder 5.1.1 and Python 3.8.8 64-bit | Qt 5.12.10 | PyQt5 5.12.3 | Windows 10

I also confirm that the process is stuck if using Qt5 graphics in iPython console; and not stuck if using Inline graphics (although no graphic is shown in the minimal code)

Same symptoms for other users : @minouHub; @BlehMaks

[steff456]

Which version of Vaex are you using? I'm activating the option for using Qt5 graphics and the minimal example is still running on my end.

This is my output for the Vaex version I installed:

{'vaex-core': '4.3.0.post1',
 'vaex-viz': '0.5.0',
 'vaex-hdf5': '0.8.0',
 'vaex-server': '0.5.0',
 'vaex-astro': '0.8.2',
 'vaex-jupyter': '0.6.0',
 'vaex-ml': '0.12.0'}

[erwanp]

We have the same version

vaex                      4.3.0              pyhd8ed1ab_0    conda-forge
vaex-astro                0.8.2              pyhd8ed1ab_0    conda-forge
vaex-core                 4.3.0.post1      py38hd61481d_0    conda-forge
vaex-hdf5                 0.8.0              pyhd8ed1ab_0    conda-forge
vaex-jupyter              0.6.0              pyhd8ed1ab_0    conda-forge
vaex-ml                   0.12.0             pyhd8ed1ab_0    conda-forge
vaex-server               0.5.0              pyhd8ed1ab_0    conda-forge
vaex-viz                  0.5.0              pyhd8ed1ab_0    conda-forge

[steff456]

I could reproduce this issue on Windows, we will investigate it because it doesn't happen on macOS or Linux. We will try to fix it for one of our next releases.

@impact27, when the console stays opened some errors start appearing,

[SpyderKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "C:\Users\andfoy-windows\spyder\external-deps\spyder-kernels\spyder_kernels\comms\frontendcomm.py", line 164, in poll_one
    asyncio.run(handler(out_stream, ident, msg))
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\site-packages\nest_asyncio.py", line 32, in run
    return loop.run_until_complete(future)
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\site-packages\nest_asyncio.py", line 60, in run_until_complete
    f = asyncio.ensure_future(future, loop=self)
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\asyncio\tasks.py", line 679, in ensure_future
    raise TypeError('An asyncio.Future, a coroutine or an awaitable is '
TypeError: An asyncio.Future, a coroutine or an awaitable is required

I don't know if this is related to the kernel.

[erwanp]

Well done for reproducing it. The 2 other users also experienced it on Windows.

If it can help , note that I also reproduce the issue on the Ubuntu I host on my Windows VMWare.

[maartenbreddels]

I wonder if this could be because Vaex is using nest_asyncio, is spyder using an event loop?

[ccordoba12]

Hey @maartenbreddels, thanks for showing up! That's a very good point: when people activate the Qt5 Matplotlib backend, a Qt event loop is created in the console to handle interactive figures created by it. That's not done by Spyder, but by IPython (we just run %matplotlib qt5 behind the scenes).

@erwanp, could you try running in a notebook

%matplotlib qt5

then creating a Vaex dataframe to see what happens there? My guess is that you should observe the same stuck behavior.

[impact27]

@impact27, when the console stays opened some errors start appearing,

looks like the handler is not an asyncio coroutine. I think the error was introduced in spyder-ide/spyder-kernels#292. Maybe we need to replace https://github.com/spyder-ide/spyder-kernels/blob/db5aebe79e8820623926df9c480b893c9692a815/spyder_kernels/comms/frontendcomm.py#L155 by inspect.iscoroutinefunction(handler)

[erwanp]

then creating a Vaex dataframe to see what happens there? My guess is that you should observe the same stuck behavior.

Indeed @ccordoba12 , I am stuck there too

[ccordoba12]

@maartenbreddels, since this is not a problem with Spyder, do you think it can be fixed in Vaex somehow?

[ccordoba12]

looks like the handler is not an asyncio coroutine

@impact27, could you help me to fix that in spyder-kernels? Thanks!

[maartenbreddels]

That's gonna be difficult to fix I'm afraid. I think the problem is the interaction between nest_asyncio and pyqt, does erdewit/nest_asyncio#8 maybe help?

[maartenbreddels]

vaexio/vaex#1546 might fix this, but I cannot use that solution as explained in the issue

[impact27]

@ccordoba12 spyder-ide/spyder-kernels#317

[ccordoba12]

does erdewit/nest_asyncio#8 maybe help?

That looks really promising! But it'd need to added to ipykernel so all projects benefit from it.

vaexio/vaex#1546 might fix this, but I cannot use that solution as explained in the issue

Ok, then I think the best solution is to make the Qt and asyncio loops work together.

[ccordoba12]

@ccordoba12 spyder-ide/spyder-kernels#317

Thanks @impact27! I don't know if that will fix this problem. Could you check that?

I mean, please follow the instructions of @erwanp above.

[erwanp]

Hello @impact27 . Did you have the chance to look at the problem above ?

[maartenbreddels]

If you install the latest alpha, you can try this trick vaexio/vaex#1546 (comment) to see if it does not get stuck (that might tell you that nest_asyncio is the problem).

[erwanp]

Hello, it worked !

import numpy as np
import vaex
df = vaex.from_dict({"E":np.random.rand(1000)})
df.executor.async_method = "awaitio"
print(df)
print("before EQ mean")
E_bins = df.mean("E", binby="E", shape=10)
print("after EQ mean")       # 👈 I never get to here

Thanks a lot. How to proceed from now ?

[ccordoba12]

We could set the VAEX_ASYNC=awaitio env var mentioned by @maartenbreddels in vaexio/vaex#1546 (comment) in our kernel, so that Vaex dataframes work out of the box in Spyder.

@maartenbreddels, I guess that can't be done on demand, right? I mean, to only set that env var when people activate an interactive Matplotlib backend?

[maartenbreddels]

That's great news. I'm actually going to test if I can make that the default, and only set it to 'nest' when we need it (when using the jupyter widgets).

[ccordoba12]

Great news @maartenbreddels! Please keep us posted about it.

[maartenbreddels]

A bigger challenge then I hoped for, and I'm pretty sure in not all case the 'awaitio' method will work currently. In any case,
I still hope to solve this in Vaex though, but it does expose an issue with spyder + nest_asyncio.

[erwanp]

As far as Radis is concerned, I understand that we can :

• wait for the problem to be fixed entirely
• or write df.executor.async_method = "awaitio" after generation of the vaex DataFrame. This is only merged in Vaex's master, so already in production ?

The 2nd option would allow us to start using Vaex as our by-default DataFrame library (almost) immediatly

[maartenbreddels]

• This is only merged in Vaex's master, so already in production ?

It's released in the latest alpha for sure, but I think there are edgecases that might not work (some groupby operation, that might not work without the nested asyncio).

[maartenbreddels]

vaexio/vaex#1783 should fix this, it's not released yet, but confirmation would be great (vaex v4.8 should be the next release)

[erwanp]

Tested right now ; everything works properly with Vaex 4.8

Vaex 4.8 is officially released so we can consider this issue fixed.

Thank you @maartenbreddels @ccordoba12

[ccordoba12]

Well, I think @maartenbreddels is the one that deserves all the credit here. Thanks for making Spyder a fully supported platform for Vaex!