Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TimeourError after spawning container #59

Closed
MarcoMeter opened this issue Jan 4, 2018 · 2 comments
Closed

TimeourError after spawning container #59

MarcoMeter opened this issue Jan 4, 2018 · 2 comments

Comments

@MarcoMeter
Copy link

MarcoMeter commented Jan 4, 2018

This issue might be related to #55

Hello everybody,

I basically followed your instructions, but made a few adjustments. The issue is that the notebook container doesn't respond.

jhub      | [I 2018-01-04 14:00:46.393 JupyterHub dockerspawner:452] Container 'jupyter-leckofunny' is gone
jhub      | [I 2018-01-04 14:00:46.520 JupyterHub dockerspawner:540] Created container 'jupyter-leckofunny' (id: c0ee899) from image jupyter/tensorflow-notebook:2c80cf3537ca
jhub      | [I 2018-01-04 14:00:46.521 JupyterHub dockerspawner:557] Starting container 'jupyter-leckofunny' (id: c0ee899)
jhub      | [W 2018-01-04 14:00:56.389 JupyterHub base:499] User leckofunny is slow to become responsive (timeout=10)
jhub      | [I 2018-01-04 14:00:56.395 JupyterHub base:748] leckofunny is pending spawn
jhub      | [I 2018-01-04 14:00:56.403 JupyterHub log:122] 200 GET /hub/user/leckofunny/ (leckofunny@::ffff:172.17.1.1) 10110.44ms
jhub      | [W 2018-01-04 14:00:56.422 JupyterHub log:122] 404 GET /hub/logo (@::ffff:172.17.1.1) 1.08ms
jhub      | [I 2018-01-04 14:01:11.638 JupyterHub base:719] Pending spawn for leckofunny didn't finish in 10.0 seconds
jhub      | [I 2018-01-04 14:01:11.638 JupyterHub base:724] leckofunny is pending spawn
jhub      | [W 2018-01-04 14:01:11.658 JupyterHub log:122] 404 GET /hub/logo (@::ffff:172.17.1.1) 1.00ms
jhub      | [W 2018-01-04 14:01:14.223 JupyterHub user:458] leckofunny's server never showed up at http://127.0.0.1:32771/user/leckofunny/ after 30 seconds. Giving up
jhub      | [I 2018-01-04 14:01:14.230 JupyterHub dockerspawner:628] Stopping container jupyter-leckofunny (id: c0ee899)
jhub      | [E 2018-01-04 14:01:14.767 JupyterHub gen:914] Exception in Future <tornado.concurrent.Future object at 0x7f3e5058c908> after timeout
jhub      |     Traceback (most recent call last):
jhub      |       File "/opt/conda/lib/python3.5/site-packages/tornado/gen.py", line 910, in error_callback
jhub      |         future.result()
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 444, in finish_user_spawn
jhub      |         yield spawn_future
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 476, in spawn
jhub      |         raise e
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 450, in spawn
jhub      |         resp = yield server.wait_up(http=True, timeout=spawner.http_timeout)
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/utils.py", line 180, in wait_for_http_server
jhub      |         timeout=timeout
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/utils.py", line 135, in exponential_backoff
jhub      |         raise TimeoutError(fail_message)
jhub      |     TimeoutError: Server at http://127.0.0.1:32771/user/leckofunny/ didn't respond in 30 seconds
jhub      |
jhub      | [E 2018-01-04 14:01:14.771 JupyterHub gen:914] Exception in Future <tornado.concurrent.Future object at 0x7f3e5058c908> after timeout
jhub      |     Traceback (most recent call last):
jhub      |       File "/opt/conda/lib/python3.5/site-packages/tornado/gen.py", line 910, in error_callback
jhub      |         future.result()
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 444, in finish_user_spawn
jhub      |         yield spawn_future
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 476, in spawn
jhub      |         raise e
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 450, in spawn
jhub      |         resp = yield server.wait_up(http=True, timeout=spawner.http_timeout)
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/utils.py", line 180, in wait_for_http_server
jhub      |         timeout=timeout
jhub      |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/utils.py", line 135, in exponential_backoff
jhub      |         raise TimeoutError(fail_message)
jhub      |     TimeoutError: Server at http://127.0.0.1:32771/user/leckofunny/ didn't respond in 30 seconds

I tried these notebook images and called make nobteook_image:
jupyter/minimal-notebook:e1677043235c and jupyter/tensorflow-notebook:2c80cf3537ca (which is currently latest)

Maybe its due to some of my modifications:

  • removed https stuff
  • set port to 8080 (port 8000 is already in use by my gitlab server which is on the same host)
  • set hub_port to 8081
  • oauth via custom gitlab (works)
  • assigned a subnet to the docker network docker network create --subnet=10.180.112.1/24 $(DOCKER_NETWORK_NAME) (otherwise it causes a conflict within my university's network)

I'm wondering if the issue is caused by the network setup.
jupyterhub-deploy-docker.zip

@MarcoMeter
Copy link
Author

This is the log of the spawned container

Executing the command: jupyterhub-singleuser --ip="0.0.0.0" --port=8888
[W 2018-01-09 12:40:02.986 SingleUserNotebookApp configurable:168] Config option                                                                                                                                                                                                                                              `open_browser` not recognized by `SingleUserNotebookApp`.  Did you mean `browse                                                                                                                                                                                                                                             r`?
[I 2018-01-09 12:40:03.405 SingleUserNotebookApp extension:48] JupyterLab alpha                                                                                                                                                                                                                                              preview extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
[I 2018-01-09 12:40:03.405 SingleUserNotebookApp extension:49] JupyterLab applic                                                                                                                                                                                                                                             ation directory is /opt/conda/share/jupyter/lab
[I 2018-01-09 12:40:03.409 SingleUserNotebookApp singleuser:365] Starting jupyte                                                                                                                                                                                                                                             rhub-singleuser server version 0.8.1
[E 2018-01-09 12:40:03.410 SingleUserNotebookApp singleuser:354] Failed to conne                                                                                                                                                                                                                                             ct to my Hub at http://127.0.0.1:8081/hub/api (attempt 1/5). Is it running?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", li                                                                                                                                                                                                                                             ne 351, in check_hub_version
        resp = yield client.fetch(self.hub_api_url)
      File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 1055, i                                                                                                                                                                                                                                             n run
        value = future.result()
      File "/opt/conda/lib/python3.6/site-packages/tornado/concurrent.py", line                                                                                                                                                                                                                                              238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2018-01-09 12:40:05.414 SingleUserNotebookApp singleuser:354] Failed to conne                                                                                                                                                                                                                                             ct to my Hub at http://127.0.0.1:8081/hub/api (attempt 2/5). Is it running?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", li                                                                                                                                                                                                                                             ne 351, in check_hub_version
        resp = yield client.fetch(self.hub_api_url)
      File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 1055, i                                                                                                                                                                                                                                             n run
        value = future.result()
      File "/opt/conda/lib/python3.6/site-packages/tornado/concurrent.py", line                                                                                                                                                                                                                                              238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2018-01-09 12:40:09.419 SingleUserNotebookApp singleuser:354] Failed to conne                                                                                                                                                                                                                                             ct to my Hub at http://127.0.0.1:8081/hub/api (attempt 3/5). Is it running?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", li                                                                                                                                                                                                                                             ne 351, in check_hub_version
        resp = yield client.fetch(self.hub_api_url)
      File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 1055, i                                                                                                                                                                                                                                             n run
        value = future.result()
      File "/opt/conda/lib/python3.6/site-packages/tornado/concurrent.py", line                                                                                                                                                                                                                                              238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2018-01-09 12:40:17.427 SingleUserNotebookApp singleuser:354] Failed to conne                                                                                                                                                                                                                                             ct to my Hub at http://127.0.0.1:8081/hub/api (attempt 4/5). Is it running?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", li                                                                                                                                                                                                                                             ne 351, in check_hub_version
        resp = yield client.fetch(self.hub_api_url)
      File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 1055, i                                                                                                                                                                                                                                             n run
        value = future.result()
      File "/opt/conda/lib/python3.6/site-packages/tornado/concurrent.py", line                                                                                                                                                                                                                                              238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
    ConnectionRefusedError: [Errno 111] Connection refused
[C 2018-01-09 12:40:32.712 SingleUserNotebookApp notebookapp:1294] received sign                                                                                                                                                                                                                                             al 15, stopping
Traceback (most recent call last):
  File "/opt/conda/bin/jupyterhub-singleuser", line 6, in <module>
    main()
  File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", line 4                                                                                                                                                                                                                                             55, in main
    return SingleUserNotebookApp.launch_instance(argv)
  File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", lin                                                                                                                                                                                                                                             e 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py",                                                                                                                                                                                                                                              line 658, in launch_instance
    app.start()
  File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", line 3                                                                                                                                                                                                                                             67, in start
    ioloop.IOLoop.current().run_sync(self.check_hub_version)
  File "/opt/conda/lib/python3.6/site-packages/tornado/ioloop.py", line 453, in                                                                                                                                                                                                                                              run_sync
    self.start()
  File "/opt/conda/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 17                                                                                                                                                                                                                                             7, in start
    super(ZMQIOLoop, self).start()
  File "/opt/conda/lib/python3.6/site-packages/tornado/ioloop.py", line 863, in                                                                                                                                                                                                                                              start
    event_pairs = self._impl.poll(poll_timeout)
  File "/opt/conda/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 12                                                                                                                                                                                                                                             2, in poll
    z_events = self._poller.poll(1000*timeout)
  File "/opt/conda/lib/python3.6/site-packages/zmq/sugar/poll.py", line 99, in p                                                                                                                                                                                                                                             oll
    return zmq_poll(self.sockets, timeout=timeout)
  File "zmq/backend/cython/_poll.pyx", line 123, in zmq.backend.cython._poll.zmq                                                                                                                                                                                                                                             _poll (zmq/backend/cython/_poll.c:2186)
  File "zmq/backend/cython/_poll.pyx", line 116, in zmq.backend.cython._poll.zmq                                                                                                                                                                                                                                             _poll (zmq/backend/cython/_poll.c:2036)
  File "zmq/backend/cython/checkrc.pxd", line 12, in zmq.backend.cython.checkrc.                                                                                                                                                                                                                                             _check_rc (zmq/backend/cython/_poll.c:2418)
  File "/opt/conda/lib/python3.6/site-packages/notebook/notebookapp.py", line 12                                                                                                                                                                                                                                             95, in _signal_stop
    self.io_loop.add_callback_from_signal(self.io_loop.stop)
AttributeError: 'SingleUserNotebookApp' object has no attribute 'io_loop'

@manics
Copy link
Member

manics commented Dec 5, 2022

Hi 👋 This is an old issue and the repository has been updated. If you have more questions please use the Jupyter community forum https://discourse.jupyter.org/.

@manics manics closed this as completed Dec 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants