SNI: Update SNI related log messages

[JDepooter]

Update debug log messages related to SNI:
schannel: Turning off host validation does not turn off SNI
darwinssl: Turning off host validation does turn off SNI

I recently came across a case which worked on Windows using SChannel but not on a Mac using DarwinSSL. It turns out the request was accessing a machine on AWS which required SNI in the TLS handshake. The request was setup to turn off host validation.

Apparently, turning off host validation on DarwinSSL also turns off SNI, which is why the request failed on the Mac. I have added a debug message when this happens, as I think it will be useful for future debugging.

With SChannel , the debug messages claimed that turning off host validation also turned off SNI, but this is not the case. I have updated that debug message as well. See the attached screenshots for verification. It is possible that this behaviour differs between the various windows versions. I only have access to Windows 10.

Both OpenSSL and GnuTLS have the same behaviour as SChannel, SNI is still used when the verify host option is disabled.

[mention-bot]

@JDepooter, thanks for your PR! By analyzing the history of the files in this pull request, we identified @mback2k, @nickzman and @bagder to be potential reviewers.

[jay]

With SChannel , the debug messages claimed that turning off host validation also turned off SNI, but this is not the case. I have updated that debug message as well. See the attached screenshots for verification. It is possible that this behaviour differs between the various windows versions. I only have access to Windows 10.

Git blame puts the "Also disables SNI" comment in schannel at 1394cad, which has a commit message that says in part:

As a side effect of switching off the RFC2818 related servername checks with SCH_CRED_NO_SERVERNAME_CHECK (http://msdn.microsoft.com/en-us/library/aa923430.aspx) the SNI feature is being disabled. This effect is not documented in MSDN, but Wireshark output clearly shows the effect (details on the libcurl maillist).

I found a bunch of e-mails from Nov 2012 written by the author but not the one that discusses this issue.

I can confirm Windows 7 SP1 SNI is still sent when SCH_CRED_NO_SERVERNAME_CHECK is added to the flags. Test command curld -vk https://www.test.com

/cc @okoeroo

[okoeroo]

Hi,
At the time I had all the libraries on my machines to test with, except the Windows API.

On the maillist archive I dug up this message:
https://curl.haxx.se/mail/lib-2012-11/0036.html

i was amazed about the ill documentation if this side effect really happens. Now you mention that this effect seems to be in line with the MSDN documentation on the Schannel API. I would recommend to follow this and have the message be corrected in libcurl to match the documentation and observation made by @JDepooter and pulled up by @jay.

I did not verify the SNI (side-) effects at the time. I focussed on the peer-jacking attack only exclusively based on the paper which only tested libcurl with the OpenSSL backend. :-)

[jay]

Thanks for your input @okoeroo. I tested in Windows XP and the SNI is not sent regardless of whether the server name check is enabled. IE which also uses WinSSL shows the same behavior. We could make an <= XP specific message by Curl_verify_windows_version and then if verbose. (Also ref 6842afb)

schannel: WinSSL version is too old to connect to some servers (missing SNI, algorithms)
schannel: WinSSL version is old and may not be able to connect to some servers due to lack of SNI, algorithms, etc.
or something like that and brief. @JDepooter do you want to take a crack at it

Also can we hear from someone with ability to test on Mac versions like @nickzman to know if there is similar inconsistency there.

[JDepooter]

@jay I have updated the pull request with a Windows XP specific message which is logged at the beginning of schannel_connect_step1.

[jay]

Thanks. I've split your schannel changes off since it's sort of a different issue and landed it in 18495ec. The darwinssl change I think we should wait for someone who can test new and old Mac versions to check for inconsistency, like maybe old versions disabled SNI and new versions don't? Also typo s/validatation/validation

[jay]

ping. Anyone with Mac experience can give this a review so I can close it out?

I found a modified version of darwinssl used by the maintainer of Git OS X Installer where he says:

TLS SNI is only available under SecureTransport when running on OS X 10.5.6 or later and SSLSetPeerDomainName has been called.

@mackyle would you mind weighing in on the darwinssl part of this PR, ie that as long as SSLSetPeerDomainName is disabled SNI is disabled as well, that this holds true even in the latest Mac versions?

[nickzman]

Yeah, that's still true; there's no way to disable the peer domain check without also disabling SNI.

[jay]

Ok. Thanks guys

[mackyle]

@jay sorry, just saw this. But yeah, if you never call SSLSetPeerDomainName (I haven't checked the very latest API to see if there are any new functions to set an SNI name independently, but there weren't the last time I looked), then SecureTransport never sends SNI -- you can see this in the code on opensource.apple.com if you dig around in it.

The whole section discussing this can be found in the comments at:

https://github.com/mackyle/git-osx-installer/blob/master/patches/curl/curl_darwinssl_macosx.c#L34

There's a table in the comments:

VERIFYPEER  VERIFYHOST  Result
----------  ----------  ----------------------------------------------------
disabled    disabled    SNI still sent for DNS host names
disabled    enabled     Peer's first cert must match the host DNS or IP name
enabled     disabled    DNS host name matching still occurs, IP does not
enabled     enabled     Peer's first cert must match the host DNS or IP name

It's only the third entry in the table above that ends up with surprising behavior.

Just because you call SSLSetPeerDomainName does not mean you have to verify the peer. The darwinssl version I include in the Git OS X Installer always calls SSLSetPeerDomainName for non-IPv4/IPv6 names but never for IPv4/IPv6 (because SecureTransport is stupid and will send an IPv4 as an SNI if you do and that's standards violating behavior). It also supports CURLOPT_PINNEDPUBLICKEY, works with 10.4 or later and dynamically adjusts to the available SecureTransport features at runtime based on what's actually present in the MacOSX version it runs on (this support was crucial in order to create a universal, single-version installer).

Other than for literal IPv4/IPv6 names, the SSLSetPeerDomainName call should never be skipped because you can end up fetching the wrong site for name-based TLS virtual hosts and that, to me, is far more surprising than having VERIFYPEER force VERIFYHOST.

TLDR; Always call SSLSetPeerDomainName for non-IPv4/IPv6 literal names (otherwise SNI will not be sent) which makes VERIFYPEER force VERIFYHOST for non-IPv4/IPv6 names but avoids fetching the wrong TLS name-based virtual host.