Cannot create a new InternalIPKernel after shutting down the first one.

[mdickinson]

I'm attempting to add a test to test_internal_ipkernel that looks something like this:

    def test_repeated_start_stop(self):
        for count in xrange(5):
            logger.warning("count: %d", count)
            kernel = InternalIPKernel()
            kernel.init_ipkernel(gui_backend=None)
            kernel.shutdown()

Currently, this test fails on the second iteration:

======================================================================
ERROR: test_repeated_start_stop (__main__.TestInternalIPKernel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mdickinson/Desktop/envisage/envisage/plugins/ipython_kernel/tests/test_internal_ipkernel.py", line 43, in test_repeated_start_stop
    kernel.init_ipkernel(gui_backend=None)
  File "/Users/mdickinson/Desktop/envisage/envisage/plugins/ipython_kernel/internal_ipkernel.py", line 62, in init_ipkernel
    self.ipkernel = mpl_kernel(gui_backend)
  File "/Users/mdickinson/Desktop/envisage/envisage/plugins/ipython_kernel/internal_ipkernel.py", line 28, in mpl_kernel
    kernel.initialize(argv)
  File "<decorator-gen-120>", line 2, in initialize
  File "/Users/mdickinson/.edm/envs/canopy-data-ci/lib/python2.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/Users/mdickinson/.edm/envs/canopy-data-ci/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 445, in initialize
    self.init_sockets()
  File "/Users/mdickinson/.edm/envs/canopy-data-ci/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 235, in init_sockets
    self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
  File "/Users/mdickinson/.edm/envs/canopy-data-ci/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 177, in _bind_socket
    s.bind("tcp://%s:%i" % (self.ip, port))
  File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:5324)
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:7916)
    raise ZMQError(errno)
ZMQError: Address already in use

This is an important use-case for application using the InternalIPKernel and its associated plugin, and particularly for the test suite for such applications, which could be starting and shutting down the plugin repeatedly.

[mdickinson]

Not a bug; this just needs an IPKernelApp.clear_instance() in each iteration.

[mdickinson]

Re-opening: doing IPKernelApp.clear_instance() is enough to go for a couple of iterations, but there are still issues if the iteration count is increased (e.g., to 10).

The specific issue right now is that we're creating 22 new open unix sockets per kernel instantiation, which means we very quickly run out of file descriptors.

[dpinte]

@mdickinson I don't remember the details but wonder if this is not the reason why I had to add the ZMQ context cleanup code in the Canopy Data test suite.

[mdickinson]

Each cycle creates 8 zmq.sugar.socket.Socket instances, and the current shutdown doesn't close any of these. Half of these are created in ipykernel.kernelapp.IPKernelApp:

• shell_socket, stdin_socket and control_socket created in init_sockets
• iopub_socket created in init_iopub

and the other half in the ipykernel.iostream.IOPubThread machinery. EDIT: not quite true; critically, one of the 8 sockets is for the heartbeat thread.

[mdickinson]

Closed by #188.