Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSA key file leak in C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 #9300

Closed
ShadowZzj opened this issue Aug 12, 2022 · 25 comments
Closed

RSA key file leak in C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 #9300

ShadowZzj opened this issue Aug 12, 2022 · 25 comments
Labels
TLS Windows Windows-specific

Comments

@ShadowZzj
Copy link

ShadowZzj commented Aug 12, 2022

I use this code to perform tls mutual authentication. It works. However, anytime curl_easy_perform returns, there is a file generated in C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18. And curl_easy_cleanup doesn't delete the file. Is this a bug or there is a attribute that I can set to auto delete the file ?

   CURL *curl = curl_easy_init();
    CURLcode res;
    int result = 0;
    char errBuf[CURL_ERROR_SIZE]{0};
    char *version = curl_version();

    struct curl_slist *http_headers = NULL;
    ret                             = "";
    http_headers = curl_slist_append(http_headers, "Accept: application/json");
    http_headers = curl_slist_append(http_headers, "Content-Type: application/json");
    http_headers = curl_slist_append(http_headers, "charsets: utf-8");
    curl_easy_setopt(curl, CURLOPT_HTTPHEADER, http_headers);

    curl_easy_setopt(curl, CURLOPT_URL, apiPath);
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, str);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteData);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ret);
    curl_easy_setopt(curl, CURLOPT_POST, 1);
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errBuf);

    curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_TIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, true);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, true);
    curl_easy_setopt(curl, CURLOPT_SSLCERTTYPE, "P12");
    curl_easy_setopt(curl, CURLOPT_SSLCERT, personalCertificateFile.c_str());
    curl_easy_setopt(curl, CURLOPT_KEYPASSWD, passwd.c_str());
    res = curl_easy_perform(curl);
    curl_slist_free_all(http_headers);
    curl_easy_cleanup(curl);
    return result;

I expected the following

curl/libcurl version

7.85.0-DEV

operating system

Windows 10

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 12, 2022

It has generated tons of files in my machine, approximately 6GB as my service call this function every 15 seconds

@bagder
Copy link
Member

bagder commented Aug 12, 2022

libcurl does internet transfers for you. You decide what to download and how/if to store it. I can't see how curl makes any such decision for you.

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 12, 2022

libcurl does internet transfers for you. You decide what to download and how/if to store it. I can't see how curl makes any such decision for you.

I just test the same code in Adnimistrator process and Service process. It seems that in adnimistrator process, it will not write the file. In service process, it will write the file into C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18

I have searched on the internet that the folder C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18 is most likely relative with tls certificate authentication (use rsa key). I imagine that libcurl use windows crypto api and in system process, the api behaves different with in admin process

@bagder bagder added the Windows Windows-specific label Aug 12, 2022
@bagder
Copy link
Member

bagder commented Aug 12, 2022

Does your libcurl build use Schannel? Is that the reason? So that file is created no matter what you do in your transfer?

It seems that in administrator process, it will not write the file. In service process, it will write the file

What is the difference between those processes?

@bagder
Copy link
Member

bagder commented Aug 12, 2022

According to links like this, this what some windows crypto API does.

@bagder
Copy link
Member

bagder commented Aug 12, 2022

Sounds like a Windows bug to me.

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 12, 2022

Does your libcurl build use Schannel? Is that the reason? So that file is created no matter what you do in your transfer?

It seems that in administrator process, it will not write the file. In service process, it will write the file

What is the difference between those processes?

I use this command line to build libcurl. Maybe it uses crypto api as default?
nmake /f Makefile.vc mode=dllVC=15 MACHINE=x64 DEBUG=no

The difference can be quite a lot and I don't know how it affects the crypto api.

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 12, 2022

Sounds like a Windows bug to me.

Well, I don't know if windows crypto provides apis like CryptoCreateKeyFile(it's a imaginary name) or CryptoDeleteKeyFile or something. Maybe the defualt implementation of libcurl forget to call CryptoDeleteKeyFile or something to delete the file? For now, I will test with Schannel instead.

@bagder
Copy link
Member

bagder commented Aug 12, 2022

Saving a file on disk behind our back is insane, even more so if it is crypto related. I cannot see how that could be a mistake in libcurl code. But I know very little about Schannel/windows crypto.

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 12, 2022

Well, if you guys find the exact reason causing the problem, please comment in this issue. It will help a lot.

@bagder
Copy link
Member

bagder commented Aug 18, 2022

Can anyone else running on Windows confirm/reproduce this?

@bagder
Copy link
Member

bagder commented Aug 24, 2022

I don't think we can do much more than to document this issue, and maybe recommend not using schannel if this is a problem for the application.

@chunhualiu
Copy link

chunhualiu commented Aug 24, 2022

Try use Process Monitor to find out which process created this file.
Maybe some IDS/IPS software tried to examine the https server and saved the file.

@bagder
Copy link
Member

bagder commented Aug 24, 2022

This Windows documentation page on Key Storage and Retrieval seems to include details on how these files are made.

Mentioned by @sleevi on twitter

@bagder bagder added the TLS label Aug 24, 2022
@danielsand
Copy link

danielsand commented Aug 24, 2022

holy cow - Windows Crypto API
@ShadowZzj
https://social.msdn.microsoft.com/Forums/en-US/c75d4013-f732-4721-9c8e-fbb35e03b1c5/crypto-library-api?forum=windowssecurity

just a recommendation - switch to WSL2 and run your code inside a Container.
Seems like you do a lot of curling - thats why your C:\ProgramData\Microsoft\Crypto\ folder is stuffed.
Every new SSL/https connection will generate new data under the folder....

or you clean the folder in your code after x requests.

and yeah - windows problem....

DHowett added a commit to DHowett/curl that referenced this issue Aug 25, 2022
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)

The documentation specifies that keys that are not to be persisted
should be imported with the flag `PKCS12_NO_PERSIST_KEY`.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.

Closes curl#9300
@bagder
Copy link
Member

bagder commented Aug 25, 2022

@ShadowZzj any chance you can try out #9363 and see if it fixes the problem for you?

@bagder bagder closed this as completed in 70d010d Aug 25, 2022
@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 29, 2022

I apply the commit: schannel: when importing PFX, disable key persistence. curl_easy_perform returns CURLE_SSL_CONNECT_ERROR and the error message is schannel: next InitializeSecurityContext failed: SEC_E_INTERNAL_ERROR (0x80090304).

If I don't apply the commit, the code works. The reason I imagine is that some steps in tls handshake needs to read the key, if the key is not written to the local file, these steps fail?

@DHowett
Copy link
Contributor

DHowett commented Aug 29, 2022

Thanks for testing this! I validated it with curl as a stand-alone tool, and will try on Monday to reproduce this issue with libcurl.

What specific version of Windows are you on?

@bagder since you’re so close to the next release of curl, it may be a good idea to revert this commit in case it causes a broad issue on Windows until we can get to the root cause.

@ShadowZzj that’s a fair observation! It’s supposed to keep the key resident in memory, but if we reference it later by its ID instead of its crypto handle we might be running into some trouble.

@ShadowZzj
Copy link
Author

ShadowZzj commented Aug 29, 2022

@DHowett Thanks for the quick reply. The windows version is Windows 10 Professional 21H1 (19043.1889).

@bagder
Copy link
Member

bagder commented Aug 29, 2022

Reopened due to the issues

@bagder bagder reopened this Aug 29, 2022
bagder added a commit that referenced this issue Aug 29, 2022
This reverts commit 70d010d.

Due to further reports in #9300 that indicate this commit might
introduce problems.
@bagder
Copy link
Member

bagder commented Aug 29, 2022

thanks @DHowett, it is now reverted in aec8d30 while researching. I say we aim for landing any fix post this release.

@DHowett
Copy link
Contributor

DHowett commented Sep 8, 2022

I'm having trouble reproducing this with the snippet that you included. On Windows 10 (17763, not 19043) and Windows 11, a simple client certificate exchange with client.badssl.com (using their provided client certificate) using this code reports success.

#include <curl/curl.h>
int main(int argc, char *argv[])
{
    CURL *curl = curl_easy_init();
    CURLcode res;
    int result = 0;
    char errBuf[CURL_ERROR_SIZE]{0};
    char *version = curl_version();

    struct curl_slist *http_headers = NULL;
    char* ret                       = NULL;
    http_headers = curl_slist_append(http_headers, "Accept: application/json");
    http_headers = curl_slist_append(http_headers, "Content-Type: application/json");
    http_headers = curl_slist_append(http_headers, "charsets: utf-8");
    curl_easy_setopt(curl, CURLOPT_HTTPHEADER, http_headers);

    curl_easy_setopt(curl, CURLOPT_URL, "https://client.badssl.com");
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errBuf);

    curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_TIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, true);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, true);
    curl_easy_setopt(curl, CURLOPT_SSL_OPTIONS, CURLSSLOPT_AUTO_CLIENT_CERT);
    curl_easy_setopt(curl, CURLOPT_SSLCERTTYPE, "P12");
    curl_easy_setopt(curl, CURLOPT_SSLCERT, "badssl.com-client.p12");
    curl_easy_setopt(curl, CURLOPT_KEYPASSWD, "badssl.com");
    res = curl_easy_perform(curl);
    curl_slist_free_all(http_headers);
    curl_easy_cleanup(curl);

    fprintf(stderr, "RES: %d\n", res);
    fprintf(stderr, "RET: %s\n", ret);
    fprintf(stderr, "ERR: %s\n", errBuf);
    return result;	
}

Is there anything about your specific use case that I may be missing?

I'll try a build of 19041/19043 in a little bit.

@ShadowZzj
Copy link
Author

ShadowZzj commented Sep 8, 2022

I use the code you provide and It shows the same error. Besides, I can't run the libcurl in master. It has another error. I use the libcurl tag 7_74_0(commit hash e052859) which I have been using 1 year ago. And I only change the code in schannel.c to the code below(
image

The command I use to compile libcurl is "nmake /f Makefile.vc mode=dll VC=15 MACHINE=x64 DEBUG=yes RTLIBCFG=static"

int main(int argc, char* argv[])
{
    CURL* curl = curl_easy_init();
    CURLcode res;
    int result = 0;
    char errBuf[CURL_ERROR_SIZE]{ 0 };
    char* version = curl_version();

    struct curl_slist* http_headers = NULL;
    char* ret = NULL;
    std::string retString = "";
    http_headers = curl_slist_append(http_headers, "Accept: application/json");
    http_headers = curl_slist_append(http_headers, "Content-Type: application/json");
    http_headers = curl_slist_append(http_headers, "charsets: utf-8");
    curl_easy_setopt(curl, CURLOPT_HTTPHEADER, http_headers);

    curl_easy_setopt(curl, CURLOPT_URL, "https://testurl.com");
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errBuf);
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "{\"test\":\"123\" }");
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteData);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &retString);
    curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_TIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, true);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, true);
    curl_easy_setopt(curl, CURLOPT_SSLCERTTYPE, "P12");
    curl_easy_setopt(curl, CURLOPT_SSLCERT, R"(C:\personal.p12)");
    curl_easy_setopt(curl, CURLOPT_KEYPASSWD, "da773a19e2f24037");
    res = curl_easy_perform(curl);
    curl_slist_free_all(http_headers);
    curl_easy_cleanup(curl);

    fprintf(stderr, "RES: %d\n", res);
    fprintf(stderr, "RET: %s\n", ret);
    fprintf(stderr, "ERR: %s\n", errBuf);
    return result;
}

DHowett added a commit to DHowett/curl that referenced this issue Sep 8, 2022
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)

The documentation specifies that keys that are not to be persisted
should be imported with the flag `PKCS12_NO_PERSIST_KEY`.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.

Closes curl#9300
@DHowett
Copy link
Contributor

DHowett commented Sep 8, 2022

Thanks for the additional info! I think I've figured this out.

I couldn't reproduce your issue because I had checked out the wrong branch. 😁

In addition to the original fix, we need to keep the certificate store handle open until we're done with the certificates inside it.

Can you test this patch against the version of curl you're using?

@ShadowZzj
Copy link
Author

ShadowZzj commented Sep 9, 2022

I have tested the patch. It works! No more files in C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18. And the curl_easy_perform returns good result. It seems the problem has been solved. I think then you should check if there is memory leak or something. Maybe some execution path will not close the cert store eventually.

ps: pr is required as soon. Thanks for your help!

jay pushed a commit to jay/curl that referenced this issue Sep 14, 2022
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)

The documentation specifies that keys that are not to be persisted
should be imported with the flag `PKCS12_NO_PERSIST_KEY`.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.

---

This is take 2 of the original fix. It extends the lifetime of the
client certificate store to that of the credential handle. The original
fix which landed in 70d010d and was later reverted in aec8d30 failed to
work properly because it did not do that.

Fixes curl#9300
Closes curl#9460
@jay jay closed this as completed in 1027d52 Oct 11, 2022
obonaventure pushed a commit to mptcp-apps/curl that referenced this issue Oct 12, 2022
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)

The documentation specifies that keys that are not to be persisted
should be imported with the flag PKCS12_NO_PERSIST_KEY.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.

--

This is take 2 of the original fix. It extends the lifetime of the
client certificate store to that of the credential handle. The original
fix which landed in 70d010d and was later reverted in aec8d30 failed to
work properly because it did not do that.

Minor changes were made to the schannel credential context to support
closing the client certificate store handle at the end of an SSL session.

--

Reported-by: ShadowZzj@users.noreply.github.com

Fixes curl#9300
Supersedes curl#9363
Closes curl#9460
jquepi pushed a commit to jquepi/curl.1.555 that referenced this issue Oct 24, 2022
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)

The documentation specifies that keys that are not to be persisted
should be imported with the flag PKCS12_NO_PERSIST_KEY.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.

--

This is take 2 of the original fix. It extends the lifetime of the
client certificate store to that of the credential handle. The original
fix which landed in 70d010d and was later reverted in aec8d30 failed to
work properly because it did not do that.

Minor changes were made to the schannel credential context to support
closing the client certificate store handle at the end of an SSL session.

--

Reported-by: ShadowZzj@users.noreply.github.com

Fixes curl/curl#9300
Supersedes curl/curl#9363
Closes curl/curl#9460
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TLS Windows Windows-specific
Projects
None yet
5 participants