ssh: add ability to enable compression (for SCP/SFTP)

[vszakats]

Add the ability to enable compression for SCP/SFTP.

The required low-level logic was already available as part of libssh2 (via LIBSSH2_FLAG_COMPRESS libssh2_session_flag() option.)

This patch adds the new libcurl option CURLOPT_SSH_COMPRESSION (boolean) and the new curl command-line option --compressed-ssh to request this libssh2 feature. To have compression enabled, it is required that the SSH server supports a (zlib) compatible compression method and that libssh2 was built with zlib support enabled.

Ref: #1732

---

Use case: When transferring well-compressible (f.e. plaintext) data which is either large and/or the bandwidth is relatively low. For small transfers and/or fast connections, it may even result in slightly worse performance. See also: https://askubuntu.com/questions/6770/does-ssh-use-any-compression

[mention-bot]

@vszakats, thanks for your PR! By analyzing the history of the files in this pull request, we identified @Lekensteyn, @bagder and @yangtse to be potential reviewers.

[vszakats]

I've created the patch assuming it would make it only into 7.56.0. That is to say I'd be quite glad if it could be included in 7.55.0 next week. Given, of course that all eyes find it alright.

[bagder]

We only merge features when the feature window is open. That's 4 weeks following a release, so too late for 7.55.0

[vszakats]

Ok, it's clearly too late for that, thanks for clarifying!

[vszakats]

My real world tests were conclusively positive. Here are test binaries (for Windows) with this patch applied including a libssh2 library with zlib enabled: https://bintray.com/vszakats/generic/curl-test/7.55.1

(NOTE: these binaries are not for production use or distribution, their sole purpose is testing this specific feature on that platform.)

[vszakats]

Any suggestions or anything missing to have this approved?

[bagder]

Looks great!

[jay]

this should be my_setopt and you have to pass it a long like if(config->ssh_compression) my_setopt(curl, CURLOPT_SSH_COMPRESSION, 1L). Also the example should use 1L

[vszakats]

Thanks!

[vszakats]

Fixed both.

[jay]

the test doesn't do anything to see if the compression is enabled, is that right?

[vszakats]

That's right. It tests if download is still (transparently) working.

[vszakats]

I've found no obvious way to verify if compression is indeed used in a libcurl transfer. It also requires support from the server-side, not sure if the test server supports it. For local, manual tests, I was timing the transfers and watching the process with a network monitor.

[vszakats]

Squashed and rebased on current master.

[vszakats]

Slightly off, but the Travis CI queue may be eased a bit by enabling both 'Auto cancel' options on https://travis-ci.org/curl/curl/settings. This would drop any pending commits from the queue if a newer commit lands.

There is a similar option on AppVeyor CI, called 'Rolling builds', which could be enabled for a similar effect, but in this case, by default it will even cancel running builds on the same branch on any new commit (this can be disabled to match Travis CI.)

With these set, there would be less strain on the CI servers and the queue could finish faster.

What do you think about it?

[coveralls]

Coverage increased (+0.01%) to 73.158% when pulling fa7b3a1 on vszakats:ssh_compress into 870d849 on curl:master.

[jay]

Thanks. I landed this with a slight change so that the error message will be shown even if libssh2_session_flag isn't available. Also note infof should end in a linefeed

  if(data->set.ssh_compression) {
#if LIBSSH2_VERSION_NUM >= 0x010208
    if(libssh2_session_flag(ssh->ssh_session, LIBSSH2_FLAG_COMPRESS, 1) < 0)
#endif
      infof(data, "Failed to enable compression for ssh session\n");
  }

[vszakats]

Good idea, thank you @jay! (Noted the infof() issue – what happened is that I copied that line from the preceding failf() which doesn't need the \n)