Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to run ray.init on windows linux subsystem, help please #5018

Open
michael0521 opened this issue Jun 22, 2019 · 2 comments
Open

Failed to run ray.init on windows linux subsystem, help please #5018

michael0521 opened this issue Jun 22, 2019 · 2 comments

Comments

@michael0521
Copy link

@michael0521 michael0521 commented Jun 22, 2019

System information

  • **OS Platform and Distribution: Windows Linux subsystem Ubuntu
  • Ray installed from (source or binary): binary
  • Ray version: 0.8.0
  • Python version: 3.7
  • Exact command to reproduce: ray.init()

Describe the problem

This is my first trial with ray, I'm a dummy, I followed tutorial but failed when I run ray.init()
but failed to start it.

I tried adding 'vm.overcommit_memory = 1' to /etc/sysctl.conf and run 'sysctl vm.overcommit_memory=1' but not help.

Please help out, thanks!

pip3 install ray
pip3 install psutil

Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import ray
ray.init()
2019-06-21 22:40:43,515 WARNING worker.py:1331 -- WARNING: Not updating worker name since setproctitle is not installed. Install this with pip install setproctitle (or ray[debug]) to enable monitoring of worker processes.
2019-06-21 22:40:43,520 INFO node.py:498 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-06-21_22-40-43_519125_1536/logs.
2019-06-21 22:40:43,683 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:18024 to respond...
2019-06-21 22:40:43,851 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:64585 to respond...
2019-06-21 22:40:43,855 INFO services.py:806 -- Starting Redis shard with 6.83 GB max memory.
2019-06-21 22:40:43,969 INFO node.py:512 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-06-21_22-40-43_519125_1536/logs.
2019-06-21 22:40:43,973 INFO services.py:1442 -- Starting the Plasma object store with 10.25 GB memory using /dev/shm.
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0621 22:40:44.737254 1536 raylet_client.cc:113] Retrying to connect to socket for pathname /tmp/ray/session_2019-06-21_22-40-43_519125_1536/sockets/raylet (num_attempts = 1, num_retries = 5)
E0621 22:40:45.239096 1536 raylet_client.cc:113] Retrying to connect to socket for pathname /tmp/ray/session_2019-06-21_22-40-43_519125_1536/sockets/raylet (num_attempts = 2, num_retries = 5)
E0621 22:40:45.740525 1536 raylet_client.cc:113] Retrying to connect to socket for pathname /tmp/ray/session_2019-06-21_22-40-43_519125_1536/sockets/raylet (num_attempts = 3, num_retries = 5)
E0621 22:40:46.242228 1536 raylet_client.cc:113] Retrying to connect to socket for pathname /tmp/ray/session_2019-06-21_22-40-43_519125_1536/sockets/raylet (num_attempts = 4, num_retries = 5)
F0621 22:40:46.745719 1536 raylet_client.cc:122] Could not connect to socket /tmp/ray/session_2019-06-21_22-40-43_519125_1536/sockets/raylet
*** Check failure stack trace: ***
Fatal Python error: Aborted

Stack (most recent call first):
File "/home/michael/.local/lib/python3.6/site-packages/ray/worker.py", line 1889 in connect
File "/home/michael/.local/lib/python3.6/site-packages/ray/worker.py", line 1443 in init
File "", line 1 in
Aborted (core dumped)

Source code / logs

I found four logs with output below:

raylet.err

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0621 23:07:15.833678 2105 stats.h:48] Succeeded to initialize stats: exporter address is 127.0.0.1:8888
W0621 23:07:15.835813 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:15.936827 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.037441 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.138187 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.239048 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.339941 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.440994 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.541756 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.642078 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.742915 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.843641 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:16.944412 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.045114 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.145862 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.246727 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.347720 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.448597 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.549438 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.649876 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.750186 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.850605 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:17.951361 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.052026 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.152767 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.253532 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.354274 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.455360 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.555773 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.656281 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.756831 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.857595 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:18.958155 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.058992 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.159663 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.260383 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.361197 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.461926 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.562868 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.664078 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.764802 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.865753 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:19.966158 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.067158 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.168246 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.269014 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.369814 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.470813 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.571750 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.672900 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
W0621 23:07:20.773736 2105 redis_context.cc:190] Failed to connect to Redis, retrying.
F0621 23:07:20.874476 2105 redis_context.cc:185] Could not establish connection to redis 172.21.4.50:14887 (context.err = 1)
*** Check failure stack trace: ***
*** Aborted at 1561183640 (unix time) try "date -d @1561183640" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000839) received by PID 2105 (TID 0x7f76c12c0740) from PID 2105; stack trace: ***
@ 0x7f76c0df2890 (unknown)
@ 0x7f76bfebee97 (unknown)
@ 0x7f76bfec0801 (unknown)
@ 0x4ed429 (unknown)
@ 0x4ef15a (unknown)
@ 0x4f0463 (unknown)
@ 0x4eee82 (unknown)
@ 0x4ef071 (unknown)
@ 0x4ec712 (unknown)
@ 0x4d71db (unknown)
@ 0x4d755a (unknown)
@ 0x4a39ed (unknown)
@ 0x4a5f1e (unknown)
@ 0x40dec1 (unknown)
@ 0x7f76bfea1b97 (unknown)
@ 0x41f971 (unknown)

plasma_store.err

external/plasma/cpp/src/plasma/store.cc:1114: Allowing the Plasma store to use up to 10.2497GB of memory.
external/plasma/cpp/src/plasma/store.cc:1141: Starting object store with directory /dev/shm and huge page support disabled
external/plasma/cpp/src/plasma/store.cc:714: Disconnecting client on fd 6

redis.out

2094:C 21 Jun 2019 23:07:15.323 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2094:C 21 Jun 2019 23:07:15.323 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2094, just started
2094:C 21 Jun 2019 23:07:15.323 # Configuration loaded
2094:M 21 Jun 2019 23:07:15.325 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2094:M 21 Jun 2019 23:07:15.326 # Server initialized
2094:M 21 Jun 2019 23:07:15.326 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

redis-shard_0.out

2098:C 21 Jun 2019 23:07:15.488 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2098:C 21 Jun 2019 23:07:15.488 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2098, just started
2098:C 21 Jun 2019 23:07:15.488 # Configuration loaded
2098:M 21 Jun 2019 23:07:15.490 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2098:M 21 Jun 2019 23:07:15.491 # Server initialized
2098:M 21 Jun 2019 23:07:15.491 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

@michael0521 michael0521 changed the title Failed to run ray.init on windows linux subsystem, please help Failed to run ray.init on windows linux subsystem, help please Jun 22, 2019
@honnibal

This comment has been minimized.

Copy link

@honnibal honnibal commented Aug 21, 2019

The solution here worked for me: #3059

pip install psutil setproctitle
@mkarbo

This comment has been minimized.

Copy link

@mkarbo mkarbo commented Nov 27, 2019

You might need to install python-dev, python3-dev or (for me) python3.7-dev before the above, especially if you want to use ray in WSL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.