[autoscaler] Submit command

[richardliaw]

As requested from users.

TODO:

• Try it out
• Docs

Changes

This allows users to submit scripts and args to a cluster (that might not even exist yet). Closes #2970.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9327/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9333/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9334/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9335/
Test FAILed.

[ericl]

Since this is just sugar for rsync_up followed by an exec, why not just call those two functions instead of making this new one?

[richardliaw]

Hm, rsync_up doesn't have start cluster functionality but exec has start cluster functionality.

Ideally, you'd start, rsync, and exec.

One alternative is to do create_or_update_cluster, rsync, and then exec but it wasn't clear to me that was the cleanest way of doing things (it would create 3 different NodeUpdaters)

[ericl]

I think doing that is fine, even if it's a bit slower. The code will be hard to maintain otherwise.

…

On Tue, Nov 13, 2018, 6:11 PM Richard Liaw ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In python/ray/autoscaler/commands.py <#3312 (comment)>: > @@ -236,6 +236,55 @@ def attach_cluster(config_file, start, use_tmux, override_cluster_name, new): override_cluster_name, None) +def submit_cluster(config_file, screen, tmux, stop, start, Hm, rsync_up doesn't have start cluster functionality but exec has start cluster functionality. Ideally, you'd start, rsync, and exec. One alternative is to do create_or_update_cluster, rsync, and then exec but it wasn't clear to me that was the cleanest way of doing things (it would create 3 different NodeUpdaters) — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3312 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA6Soo1MLIFiCv-jZRhih5oJmBSA3snks5uu3vQgaJpZM4Ya2JN> .

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9352/
Test PASSed.