newparallel on Windows #365

minrk opened this Issue Apr 11, 2011 · 6 comments

3 participants

IPython member

IPython.parallel has various issues on Windows, some of which may actually be pyzmq or libzmq issues.

One issue that comes up is that the Windows select does not allow events==0 (causing a crash in libzmq). This is fixed in pyzmq, with PR95: zeromq/pyzmq#95

There also appear to be some process launching issues that result from the use of multiprocessing, which is not well behaved on Windows.

@minrk minrk was assigned Apr 11, 2011
IPython member

Using Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on Win7 x64 I tested the branch:

with the latest pyzmq. ipcontroller/ipcluster combination appear to be working perfectly. ipcluster fails since CTRL_C_EVENT appears to be new in 3.2? (

C:\WINDOWS\system32>python C:\dev\bin\Python26\Scripts\ipcluster start -n 8
[IPClusterApp] Using existing cluster dir: C:\Users\dhirschfeld\.ipython\cluster_default
[IPClusterApp] Cluster directory set to: C:\Users\dhirschfeld\.ipython\cluster_default
[IPClusterApp] Starting ipcluster with [daemon=False]
[IPClusterApp] Creating pid file: C:\Users\dhirschfeld\.ipython\cluster_default\pid\
ERROR:root:Error in periodic callback
Traceback (most recent call last):
  File "C:\dev\bin\Python26\lib\site-packages\zmq\eventloop\", line 423, in _run
  File "C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\", line 375, in start_launchers
    cl_class = import_item(config.Global.controller_launcher)
  File "C:\dev\bin\Python26\lib\site-packages\IPython\utils\", line 40, in import_item
    module = __import__(package,fromlist=[obj])
  File "C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\", line 30, in 
    from signal import CTRL_C_EVENT as SIGINT
ImportError: cannot import name CTRL_C_EVENT
IPython member

New in 2.7, actually - sorry, I didn't catch that.

Fix pushed.


Confirmed that the fix works for me. Parallel computing and ipcluster working great on 32 nodes!

...however CTRL-C to exit results in a ValueError:

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\ipclusterapp.pyc in sigint_handler(self=, signum=2, frame=)
    466     def sigint_handler(self, signum, frame):
--> 467         self.stop_launchers()

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\ipclusterapp.pyc in stop_launchers(self=, r=None)
--> 459             d1 = self.stop_engines()

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\ipclusterapp.pyc in stop_engines(self=, r=None)
--> 439             d = self.engine_launcher.stop()

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\launcher.pyc in stop(self=)
--> 422         return self.interrupt_then_kill()

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\launcher.pyc in interrupt_then_kill(self=, delay=1.0)
--> 416             d = el.interrupt_then_kill(delay)

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\launcher.pyc in interrupt_then_kill(self=, delay=1.0)
    287     def interrupt_then_kill(self, delay=2.0):
    288         """Send INT, wait a delay and then send KILL."""
--> 289         self.signal(SIGINT)

C:\dev\bin\Python26\lib\site-packages\IPython\parallel\apps\launcher.pyc in signal(self=, sig=2)
--> 285             self.process.send_signal(sig)

C:\dev\bin\Python26\lib\subprocess.pyc in send_signal(self=, sig=2)
    938         def send_signal(self, sig):
    939             """Send a signal to the process
    940             """
    941             if sig == signal.SIGTERM:
    942                 self.terminate()
    943             else:
--> 944                 raise ValueError("Only SIGTERM is supported on Windows")

Not a big problem. I've emailed the full crash report.

IPython member

This Issue will be closed by PR #374

IPython member

closed by 877366d

@minrk minrk closed this Apr 21, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment