python3.3: Type error when launching SGE cluster in IPython notebook

[GeorgViehoever]

IPython13.2: Launching a cluster configured for SGE use caused a type error stating that you cannot apply a regular expression to a byte sequence (that apparently was the result of some shell level command). Unfortunately, the exact message got lost.
Workaround: Change ib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py a follows:

def parse_job_id(self, output):
"""Take the output of the submit command and return the job id."""
output=output.decode('UTF8')
...

I guess that python3.3 issues like these lurk in many places where the result of a shell command (usually a byte string) is combined somehow with a python3.3 (unicode) string.
Georg

[minrk]

try #3328

[GeorgViehoever]

Patch seems to work fine with SGE, both for starting and stopping. However, when I try to stop a cluster started with the "sh" profile, I know get the attached traceback and the message "error stopping cluster" in the web GUI.
Georg

[NotebookApp] Using existing profile dir: '/remote/de02h3/georgv/.config/ipython/profile_sh'
ERROR:tornado.application:Uncaught exception POST /clusters/sh/stop (10.96.2.63)
HTTPRequest(protocol='http', host='lenzcs47:8888', method='POST', uri='/clusters/sh/stop', version='HTTP/1.1', remote_ip='10.96.2.63', body=b'', headers={'Accept': 'application/json, text/javascript, /; q=0.01', 'Referer': 'http://lenzcs47:8888/', 'Accept-Encoding': 'gzip,deflate,sdch', 'Connection': 'keep-alive', 'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,_;q=0.3', 'Host': 'lenzcs47:8888', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31', 'Origin': 'http://lenzcs47:8888', 'Accept-Language': 'en-US,en;q=0.8', 'Content-Length': '0', 'X-Requested-With': 'XMLHttpRequest'})
Traceback (most recent call last):
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/tornado/web.py", line 1077, in _execute
*self.path_args, *_self.path_kwargs)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/tornado/web.py", line 1892, in wrapper
return method(self, _args, *_kwargs)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/frontend/html/notebook/handlers.py", line 706, in post
data = cm.stop_cluster(profile)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/frontend/html/notebook/clustermanager.py", line 160, in stop_cluster
esl.stop()
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 420, in stop
return self.interrupt_then_kill()
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 414, in interrupt_then_kill
for el in self.launchers.itervalues():
AttributeError: 'dict' object has no attribute 'itervalues'
ERROR:tornado.access:500 POST /clusters/sh/stop (10.96.2.63) 14.46ms
ERROR:tornado.application:Exception in I/O handler for fd 18
Traceback (most recent call last):
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/tornado/ioloop.py", line 653, in start
self._handlers[fd](fd, events)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/tornado/stack_context.py", line 241, in wrapped
callback(_args, *_kwargs)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 321, in handle_stdout
self.poll()
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 340, in poll
self.notify_stop(dict(exit_code=status, pid=self.process.pid))
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 201, in notify_stop
d(data)
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/frontend/html/notebook/clustermanager.py", line 132, in controller_stopped
esl.stop()
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 420, in stop
return self.interrupt_then_kill()
File "/global/cust_apps_seg2/seg-tools/development/linux64/ipython_0.13.2-python-3.3.1-gcc-4.5.2_20130506/lib/python3.3/site-packages/ipython-0.13.2-py3.3.egg/IPython/parallel/apps/launcher.py", line 414, in interrupt_then_kill
for el in self.launchers.itervalues():
AttributeError: 'dict' object has no attribute 'itervalues'

[minrk]

PR #3328 should not have any effect on that - it looks like your system has somehow managed to install the file without running 2to3 on it. Try rm -rf build before installing, or do an install from a fresh checkout.

[GeorgViehoever]

You are right, I was wrong. I installed your patch without first applying 2to3. Sorry about that!
For the moment, this issue can be closed.