WinSSL - No TLS handshake when specifying ciphers (encryption algorithms)

[georgeok]

When specifying encryption algorithms on build with Schannel, the call to AcquireCredentialsHandle fails and no client hello is sent at all. I have tried various combinations of CALG_ options and they all seems to fail with the same error.

curl/lib/vtls/schannel.c

Line 666 in f097669

s_pSecFn->AcquireCredentialsHandle(NULL, (TCHAR *)UNISP_NAME,

I did this

curl.exe --ciphers "CALG_MD2:CALG_MD4:CALG_MD5:CALG_SHA:CALG_SHA1:CALG_MAC:CALG_RSA_SIGN:CALG_DSS_SIGN:CALG_NO_SIGN:CALG_RSA_KEYX:CALG_DES:CALG_3DES_112:CALG
_3DES:CALG_DESX:CALG_RC2:CALG_RC4:CALG_SEAL:CALG_DH_SF:CALG_DH_EPHEM:CALG_AGREEDKEY_ANY:CALG_HUGHES_MD5:CALG_SKIPJACK:CALG_TEK:CALG_CYLINK_MEK:CALG_SSL3_SHAMD5:CALG_SSL3_MASTER:CALG_SCHANNEL
_MASTER_HASH:CALG_SCHANNEL_MAC_KEY:CALG_SCHANNEL_ENC_KEY:CALG_PCT1_MASTER:CALG_SSL2_MASTER:CALG_TLS1_MASTER:CALG_RC5:CALG_HMAC:CALG_TLS1PRF:CALG_HASH_REPLACE_OWF:CALG_AES_128:CALG_AES_192:CA
LG_AES_256:CALG_AES:CALG_SHA_256:CALG_SHA_384:CALG_SHA_512:CALG_ECDH:CALG_ECMQV:CALG_ECDSA" https://www.github.com

and the command fails with
curl: (35) schannel: AcquireCredentialsHandle failed: SEC_E_ALGORITHM_MISMATCH (0x80090331) - The client and server cannot communicate, because they do not possess a common algorithm.

curl/libcurl version

curl 7.63.0-DEV (i386-pc-win32) libcurl/7.63.0-DEV WinSSL
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS Debug IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL

The same happens for older versions. I don't think it's a regression thing.

operating system

Windows 7, 8, 10 ans server 2012. I haven't tried others.

[gvanem]

I have been trying to verify this. Yes, I get the same issue w/o knowing what the problem is
(partly thanks to the IMHO awful code style).

But, did you note this in schannel.c:

static ALG_ID algIds[45]; /*There are 45 listed in the MS headers*/

The 45 could be correct at some time. But for (NTDDI_VERSION >= NTDDI_WIN10_RS1), my version of wincrypt.h lists 53. Your command-line do have exactly 45 ciphers listed.

Besides, shouldn't the algIds[] be zeroed before filling it?

[georgeok]

I don't think the array length plays a role here. In

set_ssl_ciphers(SCHANNEL_CRED *schannel_cred, char *ciphers)
{
  char *startCur = ciphers;
  int algCount = 0;
. . . . . 
  schannel_cred->palgSupportedAlgs = algIds;
  schannel_cred->cSupportedAlgs = algCount;
  return CURLE_OK;
}

we set the algCount so the Schannel knows exactly the number of algorithms.

The problem seems unclear to me. Some options like CALG_ECDSA seem to work fine, while others like CALG_ECDH return the same error.

Is there a chance that we can only set algorithms that are enabled in the windows registry?

[jay]

I think this is schannel's fault that it does not document which individual algorithm ids it supports for TLS other than to say "the algorithm identifiers CALG_AES, CALG_AES_128, and CALG_AES_256are not supported." It is more groups that bundle them like CALG_RSA_SIGN or CALG_DH_EPHEM should work. CALG_ECDH is "is supported only through Cryptography API: Next Generation" so maybe we don't support that?

[georgeok]

I agree. Shall we add something in the documentation to indicate to people how to use this feature?

[jay]

I think that's a good idea. I'll write up a script to go through each id and see which ones work.

[jay]

I'll write up a script to go through each id and see which ones work.

Script is at https://gist.github.com/jay/51f8511e411fd6bf9ea0d531f30f27d6

The script tests each CALG and looks for common errors SEC_E_ALGORITHM_MISMATCH and SEC_E_ILLEGAL_MESSAGE. The illegal message results differ for example test.com CALG_DH_EPHEM illegal but example.com success. I'm not sure if it's right for us to specify which algorithms work when different hosts return different results. The algorithm mismatches probably are going to remain fairly constant because that error seems to be returned acquiring the handle, but who's to say a different version of Windows won't behave differently (I did this test in Windows 7).

Host/SSL-URL: test.com

Testing 47 schannel ciphers from CIPHERS.md...............................................

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ALGORITHM_MISMATCH: 
CALG_MD2, CALG_MD4, CALG_MD5, CALG_MAC, CALG_NO_SIGN, CALG_DES, CALG_3DES_112, CALG_DESX, CALG_RC2, CALG_RC4, CALG_SEAL, CALG_DH_SF, CALG_AGREEDKEY_ANY, CALG_HUGHES_MD5, CALG_SKIPJACK, CALG_TEK, CALG_CYLINK_MEK, CALG_SSL3_SHAMD5, CALG_SSL3_MASTER, CALG_SCHANNEL_MASTER_HASH, CALG_SCHANNEL_MAC_KEY, CALG_SCHANNEL_ENC_KEY, CALG_PCT1_MASTER, CALG_SSL2_MASTER, CALG_TLS1_MASTER, CALG_RC5, CALG_HMAC, CALG_TLS1PRF, CALG_HASH_REPLACE_OWF, CALG_AES_192, CALG_AES, CALG_ECDH, CALG_ECMQV

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ILLEGAL_MESSAGE: 
CALG_DSS_SIGN, CALG_3DES, CALG_DH_EPHEM, CALG_ECDSA

CURLE_SSL_CIPHER (59): 
CALG_SHA_384, CALG_SHA_512, CALG_ECDH_EPHEM

No error: 
CALG_SHA, CALG_SHA1, CALG_RSA_SIGN, CALG_RSA_KEYX, CALG_AES_128, CALG_AES_256, CALG_SHA_256

Host/SSL-URL: example.com

Testing 47 schannel ciphers from CIPHERS.md...............................................

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ALGORITHM_MISMATCH: 
CALG_MD2, CALG_MD4, CALG_MD5, CALG_MAC, CALG_NO_SIGN, CALG_DES, CALG_3DES_112, CALG_DESX, CALG_RC2, CALG_RC4, CALG_SEAL, CALG_DH_SF, CALG_AGREEDKEY_ANY, CALG_HUGHES_MD5, CALG_SKIPJACK, CALG_TEK, CALG_CYLINK_MEK, CALG_SSL3_SHAMD5, CALG_SSL3_MASTER, CALG_SCHANNEL_MASTER_HASH, CALG_SCHANNEL_MAC_KEY, CALG_SCHANNEL_ENC_KEY, CALG_PCT1_MASTER, CALG_SSL2_MASTER, CALG_TLS1_MASTER, CALG_RC5, CALG_HMAC, CALG_TLS1PRF, CALG_HASH_REPLACE_OWF, CALG_AES_192, CALG_AES, CALG_ECDH, CALG_ECMQV

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ILLEGAL_MESSAGE: 
CALG_DSS_SIGN, CALG_3DES, CALG_ECDSA

CURLE_SSL_CIPHER (59): 
CALG_SHA_384, CALG_SHA_512, CALG_ECDH_EPHEM

No error: 
CALG_SHA, CALG_SHA1, CALG_RSA_SIGN, CALG_RSA_KEYX, CALG_DH_EPHEM, CALG_AES_128, CALG_AES_256, CALG_SHA_256

[georgeok]

Thanks @jay. I agree, Windows is dictating the rules for Schannel. What I found strange though is that with some algorithms don't even attempt to connect. For example CALG_MD4 doesn't even send a client hello message. For the SEC_E_ALGORITHM_MISMATCH error it's clear that the server and the client cannot agree. I'll make a PR with a few extra words in the documentation to help users navigate around these issues.

I am posting results for the same test on Windows 8.1 and 10 and 2012 for reference.
Windows 8.1:

Host/SSL-URL: https://example.com

Testing 47 schannel ciphers from CIPHERS.md...............................................

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ALGORITHM_MISMATCH: 
CALG_MD2, CALG_MD4, CALG_MAC, CALG_NO_SIGN, CALG_DES, CALG_3DES_112, CALG_DESX, CALG_RC2, CALG_SEAL, CALG_DH_SF, CALG_AGREEDKEY_ANY, CALG_HUGHES_MD5, CALG_SKIPJACK, CALG_TEK, CALG_CYLINK_MEK, CALG_SSL3_SHAMD5, CALG_SSL3_MASTER, CALG_SCHANNEL_MASTER_HASH, CALG_SCHANNEL_MAC_KEY, CALG_SCHANNEL_ENC_KEY, CALG_PCT1_MASTER, CALG_SSL2_MASTER, CALG_TLS1_MASTER, CALG_RC5, CALG_HMAC, CALG_TLS1PRF, CALG_HASH_REPLACE_OWF, CALG_AES_192, CALG_AES, CALG_SHA_512, CALG_ECDH, CALG_ECMQV

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ILLEGAL_MESSAGE: 
CALG_MD5, CALG_DSS_SIGN, CALG_3DES, CALG_RC4, CALG_ECDSA

No error: 
CALG_SHA, CALG_SHA1, CALG_RSA_SIGN, CALG_RSA_KEYX, CALG_DH_EPHEM, CALG_AES_128, CALG_AES_256, CALG_SHA_256, CALG_SHA_384, CALG_ECDH_EPHEM

Windows 10:

Host/SSL-URL: https://example.com

Testing 47 schannel ciphers from CIPHERS.md...............................................

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ALGORITHM_MISMATCH: 
CALG_MD2, CALG_MD4, CALG_MD5, CALG_MAC, CALG_DSS_SIGN, CALG_NO_SIGN, CALG_DES, CALG_3DES_112, CALG_DESX, CALG_RC2, CALG_RC4, CALG_SEAL, CALG_DH_SF, CALG_AGREEDKEY_ANY, CALG_HUGHES_MD5, CALG_SKIPJACK, CALG_TEK, CALG_CYLINK_MEK, CALG_SSL3_SHAMD5, CALG_SSL3_MASTER, CALG_SCHANNEL_MASTER_HASH, CALG_SCHANNEL_MAC_KEY, CALG_SCHANNEL_ENC_KEY, CALG_PCT1_MASTER, CALG_SSL2_MASTER, CALG_TLS1_MASTER, CALG_RC5, CALG_HMAC, CALG_TLS1PRF, CALG_HASH_REPLACE_OWF, CALG_AES_192, CALG_AES, CALG_SHA_512, CALG_ECDH, CALG_ECMQV

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ILLEGAL_MESSAGE: 
CALG_3DES, CALG_ECDSA

No error: 
CALG_SHA, CALG_SHA1, CALG_RSA_SIGN, CALG_RSA_KEYX, CALG_DH_EPHEM, CALG_AES_128, CALG_AES_256, CALG_SHA_256, CALG_SHA_384, CALG_ECDH_EPHEM

Windows 2012 r2:

Host/SSL-URL: https://www.example.com

Testing 47 schannel ciphers from CIPHERS.md...............................................

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ALGORITHM_MISMATCH: 
CALG_MD2, CALG_MD4, CALG_MAC, CALG_NO_SIGN, CALG_DES, CALG_3DES_112, CALG_DESX, CALG_RC2, CALG_SEAL, CALG_DH_SF, CALG_AGREEDKEY_ANY, CALG_HUGHES_MD5, CALG_SKIPJACK, CALG_TEK, CALG_CYLINK_MEK, CALG_SSL3_SHAMD5, CALG_SSL3_MASTER, CALG_SCHANNEL_MASTER_HASH, CALG_SCHANNEL_MAC_KEY, CALG_SCHANNEL_ENC_KEY, CALG_PCT1_MASTER, CALG_SSL2_MASTER, CALG_TLS1_MASTER, CALG_RC5, CALG_HMAC, CALG_TLS1PRF, CALG_HASH_REPLACE_OWF, CALG_AES_192, CALG_AES, CALG_SHA_512, CALG_ECDH, CALG_ECMQV

CURLE_SSL_CONNECT_ERROR (35) + SEC_E_ILLEGAL_MESSAGE: 
CALG_MD5, CALG_DSS_SIGN, CALG_3DES, CALG_RC4, CALG_ECDSA

No error: 
CALG_SHA, CALG_SHA1, CALG_RSA_SIGN, CALG_RSA_KEYX, CALG_DH_EPHEM, CALG_AES_128, CALG_AES_256, CALG_SHA_256, CALG_SHA_384, CALG_ECDH_EPHEM

[bagder]

(let me channel in relevant comments I got over twitter)

Eric Lawrence (@ericlaw1979) commented:

Historically, you got algorithm_mismatch if you attempted to use a cipher in a protocol for which it is not defined. Most famously this was a huge problem with Vista, when IBM WebSphere picked SSL3+AES, illegal because AES wasn't defined until TLS1.0. imagine this is similar.

Nasko Oskov (@naskooskov) commented:

I haven't touched this area in a long time, however if AcquireCredentialsHandle is failing, it is expected that no ClientHello is sent, as it fails in the config part. Likely an algorithm is supplied to ACH which is disabled through the registry configuration.

InitializeSecurityContext is the call that will fill in the ClientHello into the supplied buffer for the code to send to the server. Failing before that is likely a config issue. One could read the registry to ensure no conflicting ciphers are specified on the command line.

[jay]

let me channel in relevant comments I got over twitter

Thanks for that. I don't think we should be responsible for disabled/unexpected ciphers that were specified to curl by the user. In other words I think what we are currently doing is correct and the burden should be on the user. I don't think we should be pre-emptive about it and read the registry. Perhaps we should put something in the CIPHERS doc that explains it though.

[georgeok]

I created the #4106 MR to add some extra information in the docs regarding to this.