Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

print inside multiprocessing crashes Ipython kernel #2438

Closed
pv opened this Issue · 3 comments

3 participants

@pv
pv commented

Printing to stdout when in multiprocessing crashes Ipython notebook kernel.
Seems to occur both in 0.13 and 1981689 (current Git master)

# bar.py
import multiprocessing

def foo(x):
    print x  # <- this statement causes the crash, works properly if commented out
    return x + 1

def go():
    pool = multiprocessing.Pool()
    try:
        print "Got:", pool.map_async(foo, [1,2,3]).get(999)
    finally:
        pool.terminate()

And in notebook:

import bar
bar.go() # <- crashes kernel

The notebook process prints this:

Assertion failed: ok (mailbox.cpp:84)
[NotebookApp] Kernel xxx-yyy-zzz-qqq-fff failed to respond to heartbeat

mailbox.cpp seems to be inside ZMQ/PyZMQ (tried with versions zmq-2.2.0 & pyzmq-2.2.0.1 and zmq-2.1.11 & pyzmq-2.1.7).

This works properly on the Ipython & Python command-line shells.

@minrk
Owner

As discussed in #2422, the solution here is to ensure that the zmq sockets are never used in a forked process. This should be possible by checking os.getpid in the right places before touching the sockets.

@piti118

Just want to note here in case someone is looking for the same thing. It also happens with stderr too.

logger = multiprocessing.log_to_stderr()
logger.setLevel(multiprocessing.SUBDEBUG)
@piti118

I can take a shot at fixing this. Would love if you tell me where to start looking.

@minrk minrk closed this in ad7b12c
@claesenm claesenm referenced this issue in claesenm/optunity
Closed

IPython crashes with optunity.parallel.pmap #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.