Tasks that throw exceptions block IPython-Parallel queue

[martinzellner]

I am using IPython-Parallel's load_balanced_view and map to perform some simulations. I use the asynchronous version of the view so it doesn't block.

Whenever the function applied by map throws an exception, the error property of the asynchronous map is set correctly. If this happens however the errorous tasks stays in the queue forever. Is there any possibility to cancel/abort it?

As far as I understand it cancel and abort only work on jobs that are not yet running (waiting in queue).

I think there should be a possibility to clear tasks that threw an error.

[minrk]

Do you have an example that reproduces the issue? Can you show pip list or conda list? It's possible this is a bug fixed in master (about to be released as 5.2).

[martinzellner]

Sure. pip3 list outputs:

appdirs (1.4.0)
backports-abc (0.4)
calliope (0.3.7)
click (6.6)
coverage (3.7.1)
CVXcanon (0.0.23.3)
cvxopt (1.1.8)
cvxpy (0.3.3)
cycler (0.9.0)
Cython (0.23.4)
decorator (4.0.10)
ecos (2.0.4)
folium (0.1.6)
geopy (1.11.0)
geotiler (0.7.0)
GRIDOPT (1.1)
gurobipy (6.0.2)
ipykernel (4.4.1)
ipyparallel (5.1.1)
ipython (5.0.0)
ipython-genutils (0.1.0)
ipyviz (1.0)
ipywidgets (5.1.5)
Jinja2 (2.8)
jsonschema (2.5.1)
jupyter (1.0.0)
jupyter-client (4.3.0)
jupyter-console (4.1.1)
jupyter-core (4.1.1)
MarkupSafe (0.23)
matplotlib (1.5.0)
mistune (0.7.1)
mppfnet (1.0)
nbconvert (4.1.0)
nbformat (4.0.1)
networkx (1.11)
nose (1.3.7)
notebook (4.2.0)
numexpr (2.6.1)
numpy (1.10.1)
OPTALG (1.0)
palettable (2.1.1)
pandas (0.17.1)
path.py (8.1.2)
pexpect (4.2.0)
PFNET (1.3)
pfvis (1.0)
pickleshare (0.7.3)
Pillow (3.1.0)
pip (7.1.2)
ply (3.8)
prompt-toolkit (1.0.5)
ptyprocess (0.5.1)
py (1.4.31)
Pygments (2.1.3)
Pyomo (4.2.10784)
pyparsing (2.0.6)
pypvwatts (2.0.1)
pytest (2.9.2)
pytest-cov (2.1.0)
python-coveralls (2.4.1)
python-dateutil (2.4.2)
pytz (2015.7)
PyUtilib (5.3.5)
PyYAML (3.11)
pyzmq (15.4.0)
qtconsole (4.2.1)
requests (2.10.0)
runipy (0.1.5)
scipy (0.16.1)
scs (1.1.7)
seaborn (0.6.0)
setuptools (25.1.6)
sh (1.11)
simplegeneric (0.8.1)
six (1.10.0)
tables (3.2.3.1)
terminado (0.6)
toolz (0.7.4)
tornado (4.4.1)
traitlets (4.2.2)
wcwidth (0.1.7)
wheel (0.24.0)
widgetsnbextension (1.2.3)

Should I try installing the master? The issue occurs for example when I try to run a command that has not been imported on the engines.

[martinzellner]

Master fixes the issue. Thanks!