ipcluster creates one icon per engine in Dock.app

[michaelaye]

Since I upgraded to 10.11.x (currently 10.11.6), every launch of engines with ipcluster creates an icon per engine in my macOS Dock.app.
Can this be avoided? Strangely, I don't find any tips on that with Google (the google-fu is strong in this one), so I'm afraid I have to bother you guys.

1. The command:

ipcluster start -n 6

2. The env:

• ipcluster --version: 5.2.0
• python 3.5
• running via conda, using conda_forge channel for everything

3. The symptom:

[minrk]

How did you install ipyparallel? I think this could be related to launching with python.app instead of plain python.

[michaelaye]

Everything is purely via conda from the conda_forge channel.

[minrk]

Can you do conda env export?

[michaelaye]

conda_env_export.txt

[michaelaye]

With "everything" i meant all the ipython stuff. ;) Surely have a few Pypi packages there.

[michaelaye]

I have confirmed with a minimal conda env that installs only ipython and ipyparallel, that no icons appear in the Dock, so it's something with my standard (huge) set of environment dependencies.

[michaelaye]

Wow, this was complicated ...

Using conda-forge

(see below for slightly different behavior with the default channel)

First, using conda-forge this is definitely related to pyqt (both 4.11 or 5.6), because I can simply remove that one package out of my minimal env (see below) and this issue disappears.
Now the weird bit: the issue is being triggered by something during the installation of the pandas package.

1. conda create -n ipython python=3 matplotlib ipython ipyparallel pyqt -y
2. source activate ipython
3. ipcluster start
   • The issue does NOT appear
4. Ctrl-C
5. conda install pandas -y (no other dependency pops up)
6. ipcluster start
   • The issue appears (8 Python icons in the dock)
7. Ctrl-C
8. conda remove pandas -y
9. ipcluster start
   • The issue remains!
10. Ctrl-C1.
11. conda remove pyqt -y
12. ipcluster start
    • The issue is gone.
    • For completeness I installed pandas again:
13. conda install pandas -y
    • The issue remains gone and another install of pyqt brings it back:
14. conda install pyqt -y
    • The issue appears.

I have confirmed that an enviroment that does start with pandas without pyqt does not show the issue until I install pyqt, but it seems that the above error path is more relevant.

For your convenience here a script that installs up to before pandas and launches ipcluster start.
All you need to create the issue now is to Ctrl-C out of the ipcluster, conda install pandas -y and launchipcluster start again.
I used a variable in the script, in case the name ipython clashes with your local conda envs.

CONDA_ENV_NAME=ipython
# note the -y at end of next line!
conda create -n $CONDA_ENV_NAME python=3 ipython ipyparallel matplotlib pyqt -y
source activate $CONDA_ENV_NAME
ipcluster start

The buggy env using conda-forge is listed in here:
buggy_env.txt

Using only the default channel (slightly different error path)

I was then afraid that this has all something to do with python.app that only is required (why?) when using the conda-forge channel, so I redid this exercise by removing my extra conda channels, leaving only the default channel.

One difference is the different qt/pyqt versions available, on the default version 5.6.x for both are available while on conda-forge I receive 4.8.x/4.11.x, so I made sure, I start with forcing both qt's to 4:

CONDA_ENV_NAME=ipython
conda create -n $CONDA_ENV_NAME python=3 ipython ipyparallel matplotlib pyqt=4 -y

In this case, both first, and with a subsequent conda install pandas -y the issue does NOT appear.

I now remove the pandas package before I upgrade to pyqt5.
Doing now a conda update --all -y does the following changes:

┌─( ~ )──────────────────────────────────────────────────────────────────( @macd2860 )─┐
(ipython) └─❱❱❱ conda update --all                                       +7124 14:54 ❰─┘
Fetching package metadata .......
Solving package specifications: ..........

Package plan for installation in environment /Users/klay6683/miniconda3/envs/ipython:

The following NEW packages will be INSTALLED:

    icu:        54.1-0

The following packages will be UPDATED:

    matplotlib: 1.5.1-np111py35_0 --> 1.5.3-np111py35_1
    pyqt:       4.11.4-py35_4     --> 5.6.0-py35_0
    qt:         4.8.7-4           --> 5.6.0-0

Proceed ([y]/n)? y

I now check and the issue still does not appear.
After installing pandas, the issue does appear.

But now the weird difference to above:

Removing pandas now removes the issue, in contrast to above!
Reinstalling pandas brings it back.
Removing pyqt in this environment also removes matplotlib, after which the issue is gone.
Reinstalling matplotlib brings back pyqt as well, after the issue is again there.

Here is the buggy enviroment using the default channel only:
buggy_env_default_channel.txt

I'm missing some insight somewhere and am totally puzzled, it seems to be created by an intricate mix of pyqt, pandas, matplotlib, icu and pandas.app but I have no clue which way and don't think I can do more to debug this at this point. Hope this helps. :/

[minrk]

Are you loading matplotlib by default in your engines (do you do this in IPython startup files)? Can you check if matplotlib or PyQt5 are in sys.modules?

[michaelaye]

yes, I do, in profile_default/startup/00-first.py.
Is that a problem? Both matplotlib and PyQt5 are in sys.modules.
If that's the issue, is there a way to have the matplotlib Qt imported for interactive work, but use the 'Agg' backend only for the parallel engines?

[michaelaye]

I just realized that starting an ipython session with the auto-import of matplotlib also creates an icon in the Dock. So this is a PyQt5 issue then?

[minrk]

Yup, instantiating the Qt backend puts an icon in the doc. If you told it to use the inline backend, I don't think you'll get this.

[michaelaye]

In the matplotlibrc file there is no offering of the inline backend?

[michaelaye]

I switched it to MacOSX, which also does not create an icon in the doc.

[minrk]

You can specify inline in matplotlibrc, it would be:

backend : 'module://ipykernel.pylab.backend_inline'

[michaelaye]

Tried that, gives me:

backend : 'module://ipykernel.pylab.backend_inline'Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:52:12)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
Automatic calling is: Full
/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/matplotlib/__init__.py:1076: UserWarning: Bad val "'module://ipykernel.pylab.backend_inline'" on line #39
	"backend : 'module://ipykernel.pylab.backend_inline'
"
	in file "/Users/klay6683/.matplotlib/matplotlibrc"
	Key backend: Unrecognized backend string "'module://ipykernel.pylab.backend_inline'": valid strings are ['GTK', 'Qt5Agg', 'pgf', 'WXAgg', 'pdf', 'Qt4Agg', 'emf', 'GTK3Cairo', 'gdk', 'cairo', 'MacOSX', 'ps', 'GTKCairo', 'GTK3Agg', 'WX', 'GTKAgg', 'TkAgg', 'agg', 'WebAgg', 'svg', 'nbAgg', 'template', 'CocoaAgg']
  (val, error_details, msg))

Using it without the apostrophes (which I think the matplotlibrc wants), I get:

backend : 'module://ipykernel.pylab.backend_inline'Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:52:12)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
Automatic calling is: Full
/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/matplotlib/__init__.py:1076: UserWarning: Bad val "'module://ipykernel.pylab.backend_inline'" on line #39
	"backend : 'module://ipykernel.pylab.backend_inline'
"
	in file "/Users/klay6683/.matplotlib/matplotlibrc"
	Key backend: Unrecognized backend string "'module://ipykernel.pylab.backend_inline'": valid strings are ['GTK', 'Qt5Agg', 'pgf', 'WXAgg', 'pdf', 'Qt4Agg', 'emf', 'GTK3Cairo', 'gdk', 'cairo', 'MacOSX', 'ps', 'GTKCairo', 'GTK3Agg', 'WX', 'GTKAgg', 'TkAgg', 'agg', 'WebAgg', 'svg', 'nbAgg', 'template', 'CocoaAgg']
  (val, error_details, msg))object?   -> Details about 'object', use 'object??' for extra details.
Automatic calling is: Full
[TerminalIPythonApp] WARNING | Unknown error in handling startup files:
---------------------------------------------------------------------------
UnknownBackend                            Traceback (most recent call last)
/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/IPython/core/shellapp.py in _exec_file(self, fname, shell_futures)
    326                                                  self.shell.user_ns,
    327                                                  shell_futures=shell_futures,
--> 328                                                  raise_exceptions=True)
    329         finally:
    330             sys.argv = save_argv

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/IPython/core/interactiveshell.py in safe_execfile(self, fname, *where, **kw)
   2479                 py3compat.execfile(
   2480                     fname, glob, loc,
-> 2481                     self.compile if kw['shell_futures'] else None)
   2482             except SystemExit as status:
   2483                 # If the call was made with 0 or None exit status (sys.exit(0)

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/IPython/utils/py3compat.py in execfile(fname, glob, loc, compiler)
    184         with open(fname, 'rb') as f:
    185             compiler = compiler or compile
--> 186             exec(compiler(f.read(), fname, 'exec'), glob, loc)
    187
    188     # Refactor print statements in doctests.

/Users/klay6683/.ipython/profile_default/startup/00-first.py in <module>()
----> 1 import matplotlib.pyplot as plt
      2 import pandas as pd
      3 import numpy as np

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/matplotlib/pyplot.py in <module>()
   2510 # are no-ops and the registered function respect `mpl.is_interactive()`
   2511 # to determine if they should trigger a draw.
-> 2512 install_repl_displayhook()
   2513
   2514 ################# REMAINING CONTENT GENERATED BY boilerplate.py ##############

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/matplotlib/pyplot.py in install_repl_displayhook()
    163             ipython_gui_name = backend2gui.get(get_backend())
    164             if ipython_gui_name:
--> 165                 ip.enable_gui(ipython_gui_name)
    166         else:
    167             _INSTALL_FIG_OBSERVER = True

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/IPython/terminal/interactiveshell.py in enable_gui(self, gui)
    450     def enable_gui(self, gui=None):
    451         if gui:
--> 452             self._inputhook = get_inputhook_func(gui)
    453         else:
    454             self._inputhook = None

/Users/klay6683/miniconda3/envs/stable/lib/python3.5/site-packages/IPython/terminal/pt_inputhooks/__init__.py in get_inputhook_func(gui)
     36
     37     if gui not in backends:
---> 38         raise UnknownBackend(gui)
     39
     40     if gui in aliases:

UnknownBackend: No event loop integration for 'inline'. Supported event loops are: qt, qt4, qt5, gtk, gtk2, gtk3, tk, wx, pyglet, glut, osx