Deprecation warnings and assertion failures with ipcontroller

[slishak]

Hi,

I've been using ipcontroller and ipengine commands separately to start up an IPython cluster. However, upon updating IPython to 4.0, I'm now getting some errors and warnings printed in the console:

C:\Users\sam.lishak>ipcontroller --cluster-id=cluster
c:\python27\lib\site-packages\ipyparallel\controller\hub.py:261: UserWarning: get_default_value is deprecated: use the .default_value attribute
  scheme = TaskScheduler.scheme_name.get_default_value()
c:\python27\lib\site-packages\ipyparallel\apps\ipcontrollerapp.py:406: UserWarning: get_default_value is deprecated: use the .default_value attribute
  scheme = TaskScheduler.scheme_name.get_default_value()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\python27\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "c:\python27\lib\multiprocessing\forking.py", line 488, in prepare
    assert main_name not in sys.modules, main_name
AssertionError: __main__
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\python27\lib\multiprocessing\forking.py", line 380, in main
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\python27\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "c:\python27\lib\multiprocessing\forking.py", line 488, in prepare
    prepare(preparation_data)
  File "c:\python27\lib\multiprocessing\forking.py", line 488, in prepare
    assert main_name not in sys.modules, main_name
AssertionError: __main__
    assert main_name not in sys.modules, main_name
Traceback (most recent call last):
AssertionError  File "<string>", line 1, in <module>
: __main__
  File "c:\python27\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "c:\python27\lib\multiprocessing\forking.py", line 488, in prepare
    assert main_name not in sys.modules, main_name
AssertionError: __main__

I don't get the AssertionErrors when I also add --usethreads to the command. I don't get any errors or warnings when using ipcluster.

I'm using Windows 7 with Python 2.7.9 (32-bit) and IPython 4.0/ipyparallel 4.0.2.

Cheers,

Sam

[minrk]

Hm, multiprocessing is always a mess on Windows. I'll see if I can figure out what's going on.

[dhirschfeld]

FWIW ipcontroller --cluster-id=cluster works fine for me in Win10 x64 with Python 3.4

C:\bin\ConsoleZ>ipcontroller --cluster-id=cluster
C:\Anaconda3\lib\site-packages\ipyparallel\controller\hub.py:261: UserWarning: get_default_value is deprecated: use the .default_value attribute
  scheme = TaskScheduler.scheme_name.get_default_value()
2015-10-08 08:09:03.656 [IPControllerApp] Hub listening on tcp://127.0.0.1:53473 for registration.
2015-10-08 08:09:03.701 [IPControllerApp] Hub using DB backend: 'NoDB'
2015-10-08 08:09:03.980 [IPControllerApp] hub::created hub
2015-10-08 08:09:03.994 [IPControllerApp] writing connection info to C:\Users\Dave\.ipython\profile_default\security\ipcontroller-cluster-client.json
2015-10-08 08:09:04.023 [IPControllerApp] writing connection info to C:\Users\Dave\.ipython\profile_default\security\ipcontroller-cluster-engine.json
C:\Anaconda3\lib\site-packages\ipyparallel\apps\ipcontrollerapp.py:406: UserWarning: get_default_value is deprecated: use the .default_value attribute
  scheme = TaskScheduler.scheme_name.get_default_value()
2015-10-08 08:09:04.233 [IPControllerApp] task::using Python leastload Task scheduler
2015-10-08 08:09:04.276 [IPControllerApp] Heartmonitor started
2015-10-08 08:09:04.610 [IPControllerApp] Creating pid file: C:\Users\Dave\.ipython\profile_default\pid\ipcontroller-cluster.pid
2015-10-08 08:09:09.912 [scheduler] Scheduler started [leastload]
2015-10-08 08:09:09.919 [IPControllerApp] client::client b'\x00\x80\x00\x00)' requested 'connection_request'
2015-10-08 08:09:09.925 [IPControllerApp] client::client [b'\x00\x80\x00\x00)'] connected
2015-10-08 08:09:58.277 [IPControllerApp] CRITICAL | Received signal 2, shutting down
2015-10-08 08:09:58.279 [IPControllerApp] CRITICAL | terminating children...

C:\bin\ConsoleZ>python -c "from IPython import sys_info; print(sys_info())"
{'commit_hash': 'f534027',
 'commit_source': 'installation',
 'default_encoding': 'cp850',
 'ipython_path': 'C:\\Anaconda3\\lib\\site-packages\\IPython',
 'ipython_version': '4.0.0',
 'os_name': 'nt',
 'platform': 'Windows-8-6.2.9200',
 'sys_executable': 'C:\\Anaconda3\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '3.4.3 |Anaconda 2.3.0 (64-bit)| (default, Mar  6 2015, '
                '12:06:10) [MSC v.1600 64 bit (AMD64)]'}

[minrk]

@slishak what do you get from pip list?

[slishak]

Assimulo (2.8)
Cython (0.23.1)
decorator (4.0.2)
ecos (2.0.2)
futures (3.0.3)
gpy (0.6.0)
ipykernel (4.0.3)
ipyparallel (4.0.2)
ipython (4.0.0)
ipython-genutils (0.1.0)
jupyter-client (4.0.0)
jupyter-core (4.0.4)
lxml (3.4.4)
matplotlib (1.4.3)
menuinst (1.0.4)
numpy (1.9.2)
optimizeoptimize (0.0.0)
pandas (0.16.2)
path.py (8.1)
pickleshare (0.5)
pip (7.1.2)
psutil (3.1.1)
pycosat (0.6.1)
pyDOE (0.3.8)
pymssql (2.1.1)
pyparsing (2.0.3)
pyreadline (2.0)
python-dateutil (2.4.2)
pythonnet (2.0.0)
pytz (2015.4)
PyYAML (3.11)
pyzmq (14.7.0)
requests (2.7.0)
scandir (1.1)
scikit-learn (0.16.1)
scipy (0.16.0)
setuptools (18.2)
simplegeneric (0.8.1)
six (1.9.0)
sympy (0.7.6)
traitlets (4.0.0)
wheel (0.24.0)

After installing decorator-4.0.4, ipykernel-4.1.0, jupyter-client-4.1.1, jupyter-core-4.0.6 and path.
py-8.1.2, there is no change in behaviour. Note that the --cluster-id argument is not needed to reproduce, I just had that in my call because of some issues I was having with a previous release when it wasn't specified. I can just type "ipcontroller" (deleting the profile_default profile to make sure no other configuration settings are being used) and I will see the errors above.

[slishak]

Also a bonus bug - if I start an IPython cluster, and then do

client = ipyparallel.Client()
client.shutdown(hub=True)

I get the following printed in the ipcontroller window:

2015-10-08 15:53:59.905 [IPControllerApp] WARNING | unhandled iopub msg_type: u'shutdown_reply'
2015-10-08 15:53:59.907 [IPControllerApp] WARNING | unhandled iopub msg_type: u'shutdown_reply'
2015-10-08 15:54:00.155 [IPControllerApp] client::client '\x00\x80\x00\x00*' requested u'shutdown_request'
2015-10-08 15:54:00.157 [IPControllerApp] ERROR | Unhandled message type: u'shutdown_notice'
2015-10-08 15:54:01.157 [IPControllerApp] hub::hub shutting down.

[minrk]

Hi! I’m going through and cleaning up old/stale issues on this repo. It looks like this issue is resolved. Feel free to open a new Issue if you are still having this trouble.