Use SFTP internally for SCP command

[Jakuje]

Based on the discussion on mailing list [1], this is initial implementation of SCP mode using SFTP. Comments, suggestions, reviews, feedback welcomed.

[1] https://lists.mindrot.org/pipermail/openssh-unix-dev/2020-June/038594.html

[djmdjm]

This is a good start and is less intrusive than I though it would be.

My comments are mostly nitpicks because this is clearly on a start of a path and not the end. I think to properly replace the scp protocol we'd need a few more things

1. automatic sftp-scp fallback without need to reconnect if the remote end doesn't support sftp. This is probably possible using mux (perhaps mux-proxy) mode.
2. some degree of bug-compatibility with scp's horrid file quoting (perhaps optional?)
3. (eventually) inter-file pipelining of transfers, though this would require a fair amount of work on sftp-client.c

@mfriedl @daztucker can you take a look too?

[Jakuje]

Thank you for the comments. I addressed your suggestions in the current version.

I will be able to follow-up no earlier than in August on the fallback to scp (thanks for suggesting the mux -- it could actually work that way).

[mfriedl]

I had a quick first look and I think this looks great!
It also looks like we could put this in soon and keep defaulting to the scp protocol for a while, but
get people to use it and figure out problems.
I'll have a detailed look at the code soon!
thanks!

[Jakuje]

Updated to resolve conflicts.

I tried also to play with the fallback to scp from sftp, but the current code in sftp-client is really not ready for this as it calls fatal() in send_msg() from do_init(), which is the place we should just figure out that sftp subsystem request failed so it would require some more tweaking. Putting this aside for now.

[Jakuje]

I rebased on current master and change default to SFTP to give it more testing. Any update from your side?

[mfriedl]

I'd like to work on this if spare time permits.

[DemiMarie]

Can we please make quoting bug-compatibility optional? I really really would like to be able to avoid it in scripts.

[djmdjm]

All but the switch-default change have been committed (in af5d809 and 197e29f).

My plan is to leave the protocol as scp for at least one release and then look at switching.

Thanks Jakub for writing this!