Can't use Secure Transport backend with Crypto Token Kit

[kaisq]

I did this

In the past, I have been able to run this command on macOS successfully, when a key has been generated in the keychain and the related signed client certificate has been loaded into the keychain.

CURL_SSL_BACKEND=secure-transport curl -E <name of client cert in keychain> https://<domain requiring client cert>

However, after switching to an identity provided via Crypto Token Kit, i.e. where the key has been loaded from a smart card, this command no longer works and reports that it was unable to find the identity in the keychain. I know the identity is present and available for use, because I have used it successfully in both the Safari and Chrome browsers.

I expected the following

Since the identity is still loaded using the same keychain APIs, I expected that curl would be able to load the identity and successfully complete the request.

curl/libcurl version

I've tried with both the version of curl that ships with macOS:

$ curl -V
curl 7.64.1 (x86_64-apple-darwin20.0) libcurl/7.64.1 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.41.0
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets

And also one installed via brew:

$ /usr/local/opt/curl/bin/curl -V
curl 7.76.1 (x86_64-apple-darwin20.3.0) libcurl/7.76.1 (SecureTransport) OpenSSL/1.1.1k zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.43.0 librtmp/2.3
Release-Date: 2021-04-14
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink MultiSSL NTLM NTLM_WB SPNEGO SSL TLS-SRP UnixSockets zstd

operating system

macOS/Darwin
Big Sur 11.3.1

[kaisq]

After some investigation, I think I've narrowed this down to how the keychain query is generated in the code. Currently, the query is built using these parameters:

curl/lib/vtls/sectransp.c

Lines 1125 to 1138 in fa050ff

	/* Set up our search criteria and expected results: */
	values[0] = kSecClassIdentity; /* we want a certificate and a key */
	keys[0] = kSecClass;
	values[1] = kCFBooleanTrue; /* we want a reference */
	keys[1] = kSecReturnRef;
	values[2] = kSecMatchLimitAll; /* kSecMatchLimitOne would be better if the
	* label matching below worked correctly */
	keys[2] = kSecMatchLimit;
	/* identity searches need a SecPolicyRef in order to work */
	values[3] = SecPolicyCreateSSL(false, NULL);
	keys[3] = kSecMatchPolicy;
	/* match the name of the certificate (doesn't work in macOS 10.12.1) */
	values[4] = label_cf;
	keys[4] = kSecAttrLabel;

When I was writing code to load the identity in my own program, I had to also include the key/value kSecAttrAccessGroup: kSecAttrAccessGroupToken in my query. I can't see any way to modify this query when running curl. Are there any potential workarounds for this? I'm even open to ideas like writing my own SSL backend in a shared library and loading that into curl at runtime; I'm not sure whether curl supports any sorts of plugins like that.

[nickzman]

You could add the key-value to the dictionary, recompile, and see if that solves the problem for you. If it does, then please create a PR with the change, since I don't have a smart card for development or testing. Thanks.

[kaisq]

Ok, in my free time the past couple days, I've tried out a couple different things here, but I haven't had any complete success.
I built my dev builds using just ./configure --with-secure-transport (please let me know if I'm missing any required flags). The closest I got was this:

diff --git a/lib/vtls/sectransp.c b/lib/vtls/sectransp.c
index 4276b89cf..05379bd29 100644
--- a/lib/vtls/sectransp.c
+++ b/lib/vtls/sectransp.c
@@ -1116,8 +1116,8 @@ static OSStatus CopyIdentityWithLabel(char *label,
      kSecClassIdentity was introduced in Lion. If both exist, let's use them
      to find the certificate. */
   if(SecItemCopyMatching != NULL && kSecClassIdentity != NULL) {
-    CFTypeRef keys[5];
-    CFTypeRef values[5];
+    CFTypeRef keys[4];
+    CFTypeRef values[4];
     CFDictionaryRef query_dict;
     CFStringRef label_cf = CFStringCreateWithCString(NULL, label,
       kCFStringEncodingUTF8);
@@ -1130,17 +1130,13 @@ static OSStatus CopyIdentityWithLabel(char *label,
     values[2] = kSecMatchLimitAll; /* kSecMatchLimitOne would be better if the
                                     * label matching below worked correctly */
     keys[2] = kSecMatchLimit;
-    /* identity searches need a SecPolicyRef in order to work */
-    values[3] = SecPolicyCreateSSL(false, NULL);
-    keys[3] = kSecMatchPolicy;
-    /* match the name of the certificate (doesn't work in macOS 10.12.1) */
-    values[4] = label_cf;
-    keys[4] = kSecAttrLabel;
+    /* match tokens in CTK */
+    values[3] = kSecAttrAccessGroupToken;
+    keys[3] = kSecAttrAccessGroup;
     query_dict = CFDictionaryCreate(NULL, (const void **)keys,
-                                    (const void **)values, 5L,
+                                    (const void **)values, 4L,
                                     &kCFCopyStringDictionaryKeyCallBacks,
                                     &kCFTypeDictionaryValueCallBacks);
-    CFRelease(values[3]);
     /* Do we have a match? */
     status = SecItemCopyMatching(query_dict, (CFTypeRef *) &keys_list);

Which matches exactly the query I use in other software, and does successfully find the identity. However, trying to use that in the subsequent SSLHandshake() call fails.

CURL_SSL_BACKEND=secure-transport src/curl -v -E <name of client cert> https://<domain requiring client cert>
*   Trying <target IP>:443...
* Connected to <domain> (<target IP>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Client certificate: <name of client cert>
* Internal SSL engine error encountered during the SSL handshake
* Closing connection 0
curl: (35) Internal SSL engine error encountered during the SSL handshake

Any thoughts what might be the issue here? This is still just a SecIdentityRef object, the same as it was before I modified the query. In other programs, I am able to use this to perform encryption operations, although I have never used it in a full Secure Transport SSL/TLS connection before.

[eabalea]

Being concerned by the same KeyChain/CryptoTokenKit problem, I'm glad I'm not alone (and that's why I'm still using Mojave).
I tested the proposed diff on a Catalina machine, this works with my Gemalto MD940 token (and a certificate loaded in the Digital Signature slot).

[eabalea]

The patch works on Big Sur as well (latest release), with the same token type (tested this time on the normal slot).
One strange behaviour is that the PIN code needs to be entered after each call to curl. That was not the case before (KeyChain unlocked the token once, and every app could use it), and it's not the case with other apps using CTK (Safari, Chrome, Acrobat Reader, ...).