Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

print inside multiprocessing crashes Ipython kernel #2438

Closed
pv opened this issue Sep 27, 2012 · 3 comments
Closed

print inside multiprocessing crashes Ipython kernel #2438

pv opened this issue Sep 27, 2012 · 3 comments
Labels
Milestone

Comments

@pv
Copy link
Contributor

pv commented Sep 27, 2012

Printing to stdout when in multiprocessing crashes Ipython notebook kernel.
Seems to occur both in 0.13 and 1981689 (current Git master)

# bar.py
import multiprocessing

def foo(x):
    print x  # <- this statement causes the crash, works properly if commented out
    return x + 1

def go():
    pool = multiprocessing.Pool()
    try:
        print "Got:", pool.map_async(foo, [1,2,3]).get(999)
    finally:
        pool.terminate()

And in notebook:

import bar
bar.go() # <- crashes kernel

The notebook process prints this:

Assertion failed: ok (mailbox.cpp:84)
[NotebookApp] Kernel xxx-yyy-zzz-qqq-fff failed to respond to heartbeat

mailbox.cpp seems to be inside ZMQ/PyZMQ (tried with versions zmq-2.2.0 & pyzmq-2.2.0.1 and zmq-2.1.11 & pyzmq-2.1.7).

This works properly on the Ipython & Python command-line shells.

@minrk
Copy link
Member

minrk commented Sep 27, 2012

As discussed in #2422, the solution here is to ensure that the zmq sockets are never used in a forked process. This should be possible by checking os.getpid in the right places before touching the sockets.

@piti118
Copy link
Contributor

piti118 commented Dec 20, 2012

Just want to note here in case someone is looking for the same thing. It also happens with stderr too.

logger = multiprocessing.log_to_stderr()
logger.setLevel(multiprocessing.SUBDEBUG)

@piti118
Copy link
Contributor

piti118 commented Dec 20, 2012

I can take a shot at fixing this. Would love if you tell me where to start looking.

@minrk minrk closed this as completed in ad7b12c Apr 11, 2013
mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014
forward stdout from forked processes

uses zmq instead of multiprocessing, because mp has too many issues.

- messages are sent via PUSH/PULL from subprocesses
- messages are sent at flush time, not at write time
- subprocess messages
- no threads, no sync events, etc.

some basic tests are included

closes ipython#2438
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants