Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy tunnel support, including subclassing SSHCommandRunner #85

Merged
merged 2 commits into from
Aug 23, 2023

Conversation

dongreenberg
Copy link
Contributor

@dongreenberg dongreenberg commented Jul 13, 2023

  • Add ssh_proxy support, except for folders (due to sshfs issues). cluster.run, cluster.rsync, and forming tunnels all work through SSHCommandRunner.
  • Create a subclass of SSHCommandRunner for cleanliness and richer options.
  • Tweak working_dir behavior slightly more

Pending - allowing more ssh options, replace cluster.ssh with CommandRunner.

max_retry: int = 1,
) -> None:
# temp update rsync filters to exclude docs, when syncing over runhouse folder
org_rsync_filter = cr.RSYNC_FILTER_OPTION
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could also be pretty easily adapted to handle --delete option if we want to do that. it does get a little messy/confusing against other methods like fsspec folder copying though, where certain options will support these features and others may not

- Create a subclass of SSHCommandRunner for cleanliness and richer options.
- Tweak working_dir behavior slightly more
@carolineechen carolineechen merged commit 765edce into main Aug 23, 2023
5 checks passed
@jlewitt1 jlewitt1 deleted the proxy_tunneling branch September 1, 2023 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants