use libssh instead of shelling out

[sckott]

@jeroen may be able to hook us up with an R libssh client so we don't have to shell out with system/etc., using one of https://www.libssh2.org/ or https://www.libssh.org/

We do a variety of operations in analogsea, all funneling into this https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/droplet-ssh.R#L164 system() call.

For all system calls, we first do two things:

1. check if users have ssh and scp installed with this https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/zzz.R#L25-L31 (essentially Sys.which)
2. check if port 22 is open or not with https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/droplet-ssh.R#L160-L162 (if not stop with message)

The different use cases include (all sent to the remote droplet):

• update the remote droplet e.g, sending apt-get update
• remove libs e.g., sending apt-get autoremove -y
• install R itself, e.g. apt-get install r-base-core r-base-dev --yes --force-yes
• upload files with scp, see https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/droplet-ssh.R#L70-L77
• download files with scp, see https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/droplet-ssh.R#L84-L137
• running docker commands - if a user has docker installed on their remote droplet, there's a suite of functions in analogsea for running docker commands, see https://github.com/sckott/analogsea/blob/db3c0ec407ea0c7a7cf1112d58de3fdbaa19b2e1/R/docklet.R

[jeroen]

How do users typically authenticate? Do you assume they have their keypair setup and everything? Because this will definitely not be the case on Windows systems.

[sckott]

We strongly recommend that users use ssh keys - and assume they have them in the right place. i guess it'd be nice to fail well with useful message if key not in right place

[jeroen]

The problem is that because ssh is not native to windows, there is no right place. So we'll have to make something in R to let the user provide a keyfile or password.

[sckott]

okay, makes sense

[jeroen]

Gonna work on a ssh client this week :)

[sckott]

thanks @jeroen

[sckott]

pkg by jeroen https://github.com/ropensci/ssh getting there , looking most bueno

[sckott]

@jeroen a question for you on scp_download - to try, insall devtools::install_github("sckott/analogsea@ssh")

library(analogsea)
d <- droplet_create() %>% droplet_wait()

uploading works great

tmp <- tempfile()
saveRDS(mtcars, tmp)
d %>% droplet_upload(tmp, "mtcars2.rds")
#> [100%] /private/var/folders/fc/n7g_vrvn0sx_st0p8lxb3ts40000gn/T/Rtmpg8C0Ru/file10cda7b26f4c3
#> mtcars2.rds

But when I download, the behavior isn't what I would expect, where mtcars2.rds ends up being a directory, under which the file actual file exists as file10cda7b26f4c3

(tmp2 <- tempdir())
#> [1] "/var/folders/fc/n7g_vrvn0sx_st0p8lxb3ts40000gn/T//Rtmpg8C0Ru"
d %>% droplet_download("mtcars2.rds", tmp2)
#>          0 /var/folders/fc/n7g_vrvn0sx_st0p8lxb3ts40000gn/T//Rtmpg8C0Ru/mtcars2.rds
#>       1218 /var/folders/fc/n7g_vrvn0sx_st0p8lxb3ts40000gn/T//Rtmpg8C0Ru/mtcars2.rds/file107b067ff446c
#>       1218 /var/folders/fc/n7g_vrvn0sx_st0p8lxb3ts40000gn/T//Rtmpg8C0Ru/mtcars2.rds/file10cda7b26f4c3

Then reading fails

mtcars2 <- readRDS(tmp2)
#> Error in readRDS(tmp2) : error reading from connection
#> In addition: Warning message:
#> In readRDS(tmp2) : error reading the file

[jeroen]

I think you are passing "mtcars2.rds" to the to parameter, which indicates a directory on the server where you want to download into. The directory will be created if it does not exist.

Just upload to "." to copy the file to the working directory.

[sckott]

okay, i think it makes sense now

[sckott]

@amoeba @Thercast @trestletech @studerus @nicksmithr @berkorbay

Thanks to @jeroen pkg ssh we have a new and better way to ssh to droplets. If you have time can you try out a dev version with

devtools::install_github("sckott/analogsea@ssh")

or switch to that branch, etc.

changes so far in droplet_ssh, see examples there.

see changes here (master vs. ssh branches): https://github.com/sckott/analogsea/compare/master...ssh?w=1

Let me know of any problems encountered

[amoeba]

Cool! Just tested it out and it works great!

[sckott]

great, thanks @amoeba

[rpodcast]

@sckott Thank you very much for implementing this! I ran a quick test and found a minor problem. I always use ssh keys from my local machines to connect to my droplets, but it appears that the droplet_ssh() does not allow for extra parameters to be supplied and passed on to the ssh::ssh_connect()(see line 110) . Here's a minimal example:

d <- droplet_create(region = "nyc3")
droplet_ssh(d, "ls")
#Error: libssh failure at 'connect': Failed to resolve hostname droplet likely not up yet (Temporary failure in name resolution)
#In addition: Warning message:
#In x$network : partial match of 'network' to 'networks'

# suppose IP is 555.555.5.555
library(ssh)

# this works
ssh_connect(host = "root@555.555.5.555", keyfile = "~/.ssh/id_rsa_digitalocean")

It seems the easiest approach would be to add a keyfile parameter (with default NULL) to the droplet_ssh() arguments and simply pass that along to ssh_connect(). I'll give that a go and if it works without hassle I'll send a PR.

EDIT: PR filed #153

[sckott]

thanks @Thercast looking at the PR