IPython.kernel.client cannot be imported inside an engine

[ipython]

Original Launchpad bug 300218: https://bugs.launchpad.net/ipython/+bug/300218
Reported by: dejan-igi (Dejan Pecevski).

Importing IPython.kernel.client in an engine doesn't work.
The twisted reactor is started for the controller and engine
automatically when they start and for the client when
IPython.kernel.client is imported. The problem is that when executing import
IPython.kernel.client script in the engine, the engine already has the reactor running,
and the import tries to start the reactor again. This causes
a the ipengine<->ipcontroller communication to fail on
subsequently executed commands.

A typical use case when one needs to import IPython.kernel.client in the
engines, is when one wants to use one of the engines as a client connected
to another ipcontroller, and it seems impossible to do it,
if one cannot import the client in one of the engines.

[ipython]

[ LP comment 1 by: Brian Granger, on 2009-04-14 21:15:00.482329+00:00 ]

Fixing this will require that the core be thread safe and that the reactor runs in a different thread from user code. This should happen this summer.

[ipython]

[ LP comment 2 by: Vishal Vatsa, on 2009-04-24 18:35:58+00:00 ]

Just wondering, if the problem is that there is already another reactor
running so can't we just use the asyncclient.

I ran this StringTask:
"from IPython.kernel import asyncclient ; doc_str = asyncclient.docformat"

I am sure using asyncclient has its own set of complications.

-vishal

2009/4/14 Brian Granger ellisonbg@gmail.com:

Fixing this will require that the core be thread safe and that the
reactor runs in a different thread from user code.  This should happen
this summer.

** Changed in: ipython
  Importance: Undecided => Medium

** Changed in: ipython
      Status: New => Confirmed

** Changed in: ipython
    Assignee: (unassigned) => Brian Granger (ellisonbg)

IPython.kernel.client cannot be imported inside an engine
https://bugs.launchpad.net/bugs/300218
You received this bug notification because you are a member of IPython
Developers, which is subscribed to IPython.

Status in IPython - Enhanced Interactive Python: Confirmed

Bug description:
Importing IPython.kernel.client in an engine doesn't work.
The twisted reactor is started for the controller and engine
automatically when they start and for the client when
IPython.kernel.client is imported. The problem is that when executing import
IPython.kernel.client script in the engine, the engine already has the reactor running,
and the import tries to start the reactor again. This causes
a the ipengine<->ipcontroller communication to fail on
subsequently executed commands.

A typical use case when one needs to import IPython.kernel.client in the
engines, is when one wants to use one of the engines as a  client connected
to another ipcontroller, and it seems impossible to do it,
if one cannot import the client in one of the engines.

[ipython]

[ LP comment 3 by: Brian Granger, on 2009-04-27 21:12:24+00:00 ]

There are a couple of issue:

• The reactor is already running and kernel.client tries to start it.
• The sync code in kernel.client has to be run in a different thread
  from the main reactor. When you use kernel.client in IPython, it
  actually starts the twisted reactor in a thread. The make the client
  work in the engine, we need to run the users code (where the client
  will be used) in a different thread from the reactor.
• To do this, parts of the engine have to be made threadsafe.

I haven't though about using the asyncclient though in the engine. It
may work, but as you guess, there may be complications.

Cheers,

Brian

On Fri, Apr 24, 2009 at 11:35 AM, Vishal Vatsa vishal.vatsa@gmail.com wrote:

Just wondering, if the problem is that there is already another reactor
running so can't we just use the asyncclient.

I ran this StringTask:
"from IPython.kernel import asyncclient ; doc_str = asyncclient.docformat"

I am sure using asyncclient has its own set of complications.

-vishal

2009/4/14 Brian Granger ellisonbg@gmail.com:

Fixing this will require that the core be thread safe and that the
reactor runs in a different thread from user code.  This should happen
this summer.

** Changed in: ipython
  Importance: Undecided => Medium

** Changed in: ipython
      Status: New => Confirmed

** Changed in: ipython
    Assignee: (unassigned) => Brian Granger (ellisonbg)

IPython.kernel.client cannot be imported inside an engine
https://bugs.launchpad.net/bugs/300218
You received this bug notification because you are a member of IPython
Developers, which is subscribed to IPython.

Status in IPython - Enhanced Interactive Python: Confirmed

Bug description:
Importing IPython.kernel.client in an engine doesn't work.
The twisted reactor is started for the controller and engine
automatically when they start and for the client when
IPython.kernel.client is imported. The problem is that when executing import
IPython.kernel.client script in the engine, the engine already has the reactor running,
and the import tries to start the reactor again. This causes
a the ipengine<->ipcontroller communication to fail on
subsequently executed commands.

A typical use case when one needs to import IPython.kernel.client in the
engines, is when one wants to use one of the engines as a  client connected
to another ipcontroller, and it seems impossible to do it,
if one cannot import the client in one of the engines.

IPython.kernel.client cannot be imported inside an engine
https://bugs.launchpad.net/bugs/300218
You received this bug notification because you are a member of IPython
Developers, which is subscribed to IPython.

Status in IPython - Enhanced Interactive Python: Confirmed

Bug description:
Importing IPython.kernel.client in an engine doesn't work.
The twisted reactor is started for the controller and engine
automatically when they start and for the client when
IPython.kernel.client is imported. The problem is that when executing import
IPython.kernel.client script in the engine, the engine already has the reactor running,
and the import tries to start the reactor again. This causes
a the ipengine<->ipcontroller communication to fail on
subsequently executed commands.

A typical use case when one needs to import IPython.kernel.client in the
engines, is when one wants to use one of the engines as a  client connected
to another ipcontroller, and it seems impossible to do it,
if one cannot import the client in one of the engines.

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu
ellisonbg@gmail.com

[ellisonbg]

This won't be fixed in the Twisted version, but works in the pyzmq version that Min is working on.