incremental improvements to SSH launchers #1634
Conversation
* add to_send/fetch steps for moving connection files around. * add SSHProxyEngineSetLauncher, corresponding to `ipcluster engines` on a remote host. This can be used to start a set of engines via PBS/SGE/MPI *remotely*.
| check = check_output(self.ssh_cmd + self.ssh_args + \ | ||
| [self.location, 'test -e', remote, "&& echo 'yes' || echo 'no'"]) | ||
| check = check.strip() | ||
| if check.strip() == 'no': |
There was a problem hiding this comment.
No need to call strip() again here, since it was just done in the line above.
|
Looks good to me, other than minor comments. Thanks! |
|
edits made. Thanks! I won't do any Windows testing, and just go with: SSH Launchers are not supported on Windows. I won't assert, in case some advanced user has a weird setup that actually works, but it's not worth anyone else's time to support it. |
|
Did you forget to push? I don't see the commit with the new edits. Agreed on Windows support: when an enterprising Windows user shows up who can make this work, we'll be happy to take his/her PRs. Given how most of the time I wasted yesterday was dealing with the quirks of a Windows release, I have no problem with you punting on Win/SSH. |
Oops, could be. I'll make sure it's pushed in the morning. |
fix a few docstrings, names, and extra strips
|
yep, I pushed to trunk instead of my own branch. Should be up to date now. |
|
Great, thanks. Merging in. |
Incremental improvements to SSH launchers: * add to_send/fetch steps for moving connection files around. * add SSHProxyEngineSetLauncher, for invoking to `ipcluster engines` on a remote host. This can be used to start a set of engines via PBS/SGE/MPI *remotely*. This makes the SSHLauncher usable on machines without shared filesystems. When sending files, the destination directory must *already exist* - that is, `ipython profile create` may be necessary on the remote system, before the security dir exists for putting the connection file the first time.
|
Oops, I just pushed one more commit with associated doc updates. Sorry, I'll do a tiny PR for that. |
|
PR #1650 has just the doc commit |
|
done, merged. thanks! |
Incremental improvements to SSH launchers: * add to_send/fetch steps for moving connection files around. * add SSHProxyEngineSetLauncher, for invoking to `ipcluster engines` on a remote host. This can be used to start a set of engines via PBS/SGE/MPI *remotely*. This makes the SSHLauncher usable on machines without shared filesystems. When sending files, the destination directory must *already exist* - that is, `ipython profile create` may be necessary on the remote system, before the security dir exists for putting the connection file the first time.
ipcluster engineson a remote host.This can be used to start a set of engines via PBS/SGE/MPI remotely.
This makes the SSHLauncher usable on machines without shared filesystems, and might.
When sending files, the destination directory must already exist - that is,
ipython profile createmay be necessary on the remote system, before the security dir exists for putting the connection file the first time.I still think the SSHLaunchers are pretty crummy, but this addresses one of their shortcomings.