libcurl with Secure Transport and --enable-optimize crashes when settings CURLOPT_SSLCERT to an inexistant cert

[guillaumealgis]

I did this

• Compiled libcurl with Secure Transport on Darwin platform (tested on macOS and iOS).
• Used --enable-optimize during configuration (this bellow for the full options list).
• Tried to make a request while using a non-existent client SSL certificate (eg. curl_easy_setopt(handle, CURLOPT_SSLCERT, "thisdoesnotexistinkeychain");).

I expected the following

• The request fails and curl_easy_perform returns an error.

What I got

• curl_easy_perform crashes on a CFRelease call.

curl/libcurl version

curl 7.84.0 (arm-apple-darwin) libcurl/7.84.0 SecureTransport zlib/1.2.11 nghttp2/1.47.0
Release-Date: 2022-06-27
Protocols: http https mqtt 
Features: alt-svc AsynchDNS HSTS HTTP2 IPv6 Largefile libz SSL threadsafe UnixSockets

Operating system

macOS Monterey 12.4 (21F79)
Xcode 13.4.1 (13F100)

Darwin 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64

Details

libcurl ./configure options

./configure \
    --disable-dependency-tracking \
    --disable-shared \
    --enable-static \
    \
    --disable-debug \
    --disable-curldebug \
    --enable-optimize \
    --enable-warnings \
    --enable-symbol-hiding \
    \
    --disable-ares \
    \
    --enable-http \
    --disable-ftp \
    --disable-file \
    --disable-ldap \
    --disable-ldaps \
    --disable-rtsp \
    --disable-proxy \
    --disable-dict \
    --disable-telnet \
    --disable-tftp \
    --disable-pop3 \
    --disable-imap \
    --disable-smb \
    --disable-smtp \
    --disable-gopher \
    --disable-manual \
    --disable-libcurl-option \
    --enable-ipv6 \
    \
    --enable-threaded-resolver \
    --disable-sspi \
    --disable-crypto-auth \
    --disable-tls-srp \
    \
    --without-schannel \
    --with-secure-transport \
    \
    --without-libidn2 \
    \
    --host="arm-apple-darwin" --prefix="dist"

Minimal sample reproducing the issue

#include <stdlib.h>
#include <printf.h>

#include "curl-7.84.0/dist/include/curl/curl.h"

int main() {
    CURL *handle = curl_easy_init();

    if (!handle) {
        exit(1);
    }

    char errbuf[CURL_ERROR_SIZE];
    curl_easy_setopt(handle, CURLOPT_ERRORBUFFER, errbuf);
    errbuf[0] = 0;

    curl_easy_setopt(handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

    curl_easy_setopt(handle, CURLOPT_SSLCERTTYPE, "P12");
    curl_easy_setopt(handle, CURLOPT_SSLCERT, "thisdoesnotexistinkeychain");
    curl_easy_setopt(handle, CURLOPT_KEYPASSWD, "foobar");

    curl_easy_setopt(handle, CURLOPT_ACCEPT_ENCODING, "gzip,deflate");
    curl_easy_setopt(handle, CURLOPT_URL, "https://example.com");

    CURLcode res = curl_easy_perform(handle);

    long http_code = 0;
    curl_easy_getinfo(handle, CURLINFO_RESPONSE_CODE, &http_code);

    if (res != CURLE_OK) {
        size_t len = strlen(errbuf);
        printf("\nlibcurl: (%d) ", res);
        if(len)
            printf("%s%s", errbuf, ((errbuf[len - 1] != '\n') ? "\n" : ""));
        else
            printf("%s\n", curl_easy_strerror(res));
    } else {
        printf("OK");
    }
}

xcrun clang -g test.c -I curl-7.84.0/dist/include -L curl-7.84.0/dist/lib -lcurl -lnghttp2 -lz -framework Security -framework CoreFoundation -framework SystemConfiguration

The program will crash on :

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1b925544c)
  * frame #0: 0x00000001b925544c CoreFoundation`CFRelease.cold.1 + 16
    frame #1: 0x00000001b908838c CoreFoundation`CFRelease + 268
    frame #2: 0x0000000100060574 a.out`CopyIdentityWithLabel(label="thisdoesnotexistinkeychain", out_cert_and_key=0x000000016fdfea00) at sectransp.c:1185:11
    frame #3: 0x000000010005de80 a.out`sectransp_connect_step1(data=0x0000000101808208, conn=0x000000010180ec08, sockindex=0) at sectransp.c:1890:13
    frame #4: 0x000000010005d440 a.out`sectransp_connect_common(data=0x0000000101808208, conn=0x000000010180ec08, sockindex=0, nonblocking=true, done=0x000000016fdfef15) at sectransp.c:3068:14
    frame #5: 0x000000010005d114 a.out`sectransp_connect_nonblocking(data=0x0000000101808208, conn=0x000000010180ec08, sockindex=0, done=0x000000016fdfef15) at sectransp.c:3157:10
    frame #6: 0x0000000100063a94 a.out`Curl_ssl_connect_nonblocking(data=0x0000000101808208, conn=0x000000010180ec08, isproxy=false, sockindex=0, done=0x000000016fdfef15) at vtls.c:371:12
    frame #7: 0x00000001000200d0 a.out`https_connecting(data=0x0000000101808208, done=0x000000016fdfef15) at http.c:1597:12
    frame #8: 0x000000010001ffbc a.out`Curl_http_connect(data=0x0000000101808208, done=0x000000016fdfef15) at http.c:1523:14
    frame #9: 0x000000010003e49c a.out`protocol_connect(data=0x0000000101808208, protocol_done=0x000000016fdfef15) at multi.c:1731:16
    frame #10: 0x000000010003b274 a.out`multi_runsingle(multi=0x0000000100504088, nowp=0x000000016fdfef70, data=0x0000000101808208) at multi.c:2047:16
    frame #11: 0x000000010003aa8c a.out`curl_multi_perform(multi=0x0000000100504088, running_handles=0x000000016fdfefd8) at multi.c:2640:14
    frame #12: 0x0000000100014c04 a.out`easy_transfer(multi=0x0000000100504088) at easy.c:662:15
    frame #13: 0x0000000100013b10 a.out`easy_perform(data=0x0000000101808208, events=false) at easy.c:752:42
    frame #14: 0x0000000100013970 a.out`curl_easy_perform(data=0x0000000101808208) at easy.c:771:10
    frame #15: 0x0000000100000e84 a.out`main at test.c:27:20
    frame #16: 0x00000001000e108c dyld`start + 520

Apparently curl is releasing the keys_list pointer even if nothing has been found by SecItemCopyMatching, which causes the crash. I spent a good few hours on this trying to figure out why keys_list was not NULL, but apparently my C is too rusty to solve this :(

Any help would be greatly appreciated !

[bagder]

@nickzman can you see if you can make anything from this?

[nickzman]

Copy that; I'll see if I can reproduce this.

[nickzman]

I can't reproduce the problem with the example provided; it properly exits without finding the certificate. I'm scanning the code, and nothing really looks like it would obviously cause the problem. Can you still reproduce the problem with the latest code?

[guillaumealgis]

Thanks for taking a look @nickzman !

Here's a more "plug and play" sample which reproduces the crash on for me. You should only have to run repro.sh, and it will fetch the latest versions of curl and nghttp2, build the sample program and run it.

repro-curl-9194.zip

(So to answer your question, I can reproduce using libcurl 7.84.0 and nghttp2 1.48.0)

Here's a video of me running it on my machine :

Enregistrement.de.l.ecran.2022-07-25.a.17.41.44.mov

[nickzman]

I ran the script, but still could not reproduce the problem on my M1 Mac running the latest macOS release.

What version of Clang are you using? I'm using Apple Clang 13.1.6, which comes with Xcode 13.4.1.

[guillaumealgis]

@nickzman

❯ xcrun clang --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Applications/Xcode-13.4.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Also here's the full log of my repro.sh script running on my machine :

out.log

[guillaumealgis]

Ok so I have 2 coworkers (Intel machines) who are crashing like me, and a coworker (Apple Silicon) not crashing and getting the error message like you are.

I'll compare our configs / build logs and report back.

[guillaumealgis]

Some news :

I could not find any meaningful differences between my coworker's build logs and mine.

After a bit more debugging in lldb, it appears the bug is caused by SecCertificateCopyCommonName() (sectransp.c:1174) returning a code -26276, which I could not find a mention of in the official doc, but is said to be errSecInternal "An internal error occured in the Security framework." by osstatus.com.

Apparently in that case common_name will be set (kept) to NULL, and thus the subsequent call to CFRelease(common_name) will crash.

That doesn't looks too good to me. However that doesn't explain why it works on some machines on not others (mine 😭).

It's starting to look more and more like an Apple bug, though, so I'm not sure if I should continue to bother you guys about it. If you have any pointer on how I could possibly debug this further I'd appreciate it, but either way thank you already for the time you have spent on this issue 👍🏻

I'll probably file a radar if I can reduce the scope of the issue further (not having Apple engineers build nghttp2 and curl to reproduce would be nice), but I think I already know how this will end...

[DemiMarie]

Could this be worked around in curl by checking if common_name is NULL, and if it is, skipping the CFRelease call?

[bagder]

Does this still happen with a current curl?