New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl with nss doesn't send intermediate client certificates? #851

Closed
tomfitzhenry opened this Issue Jun 1, 2016 · 20 comments

Comments

Projects
None yet
5 participants
@tomfitzhenry

tomfitzhenry commented Jun 1, 2016

I did this

curl --cert client.crt --key client.crt https://server.that.prompts.for.client.certs.example.com/

where client.crt contains a client certificate, an intermediate certificate, and the corresponding private key.

I expected the following

Successful connection!

I actually got

...
* SSL read: errno -12195 (SSL_ERROR_UNKNOWN_CA_ALERT)
* Peer does not recognize and trust the CA that issued your certificate.

curl/libcurl version

$ curl --version
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

operating system

$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

More details

When I execute the same command, with the same certificates, against a curl/openssl:

curl 7.48.0 (x86_64-apple-darwin13.4.0) libcurl/7.48.0 OpenSSL/1.0.2h zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

then it successfully connects.

When I execute the same command, with the same version of curl, against a host that has the full CA chain, then it successfully connects.

@tomfitzhenry

This comment has been minimized.

Show comment
Hide comment
@tomfitzhenry

tomfitzhenry Jun 1, 2016

If that's not enough to spot the problem, please say, and I can spend time building a proper reproducible bug case.

tomfitzhenry commented Jun 1, 2016

If that's not enough to spot the problem, please say, and I can spend time building a proper reproducible bug case.

@bagder bagder added the SSL/TLS label Jun 1, 2016

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Jun 1, 2016

Member

Are you even using the same CA cert store with NSS as with OpenSSL ?

And you say "curl with nss doesn't send intermediate client certificates" while the NSS error message say SSL_ERROR_UNKNOWN_CA_ALERT which to me sounds like NSS didn't like the server's certificate. Are you sure it isn't?

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/SSL_functions/sslerr.html lists the error codes and for the above code it explains:

"Peer does not recognize and trust the CA that issued your certificate."

Member

bagder commented Jun 1, 2016

Are you even using the same CA cert store with NSS as with OpenSSL ?

And you say "curl with nss doesn't send intermediate client certificates" while the NSS error message say SSL_ERROR_UNKNOWN_CA_ALERT which to me sounds like NSS didn't like the server's certificate. Are you sure it isn't?

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/SSL_functions/sslerr.html lists the error codes and for the above code it explains:

"Peer does not recognize and trust the CA that issued your certificate."

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Jun 2, 2016

Collaborator

On Wednesday, June 01, 2016 10:55:47 Tom Fitzhenry wrote:

If that's not enough to spot the problem, please say, and I can spend time
building a proper reproducible bug case.

That would definitely help! I have never used such a client certificate
myself. If you prepare a test-case that works with OpenSSL but not with NSS,
I will have a look at the implementation to see how we can improve it.

Kamil

Collaborator

kdudka commented Jun 2, 2016

On Wednesday, June 01, 2016 10:55:47 Tom Fitzhenry wrote:

If that's not enough to spot the problem, please say, and I can spend time
building a proper reproducible bug case.

That would definitely help! I have never used such a client certificate
myself. If you prepare a test-case that works with OpenSSL but not with NSS,
I will have a look at the implementation to see how we can improve it.

Kamil

@tomfitzhenry

This comment has been minimized.

Show comment
Hide comment
@tomfitzhenry

tomfitzhenry Jun 2, 2016

@bagder I just checked with -k, and I observed the same behaviour, suggesting it isn't that NSS doesn't like the server's certificate.

I think "Peer does not recognize and trust the CA that issued your certificate." means the server (NSS's peer) is the one having trust issues. That this error is under the section "All the error codes in the following block indicate that the local socket received an SSL3 or TLS alert record from the remote peer, reporting some issue that it had with an SSL record or handshake message it received." supports this too.

I will produce a reproducible test case, however.

tomfitzhenry commented Jun 2, 2016

@bagder I just checked with -k, and I observed the same behaviour, suggesting it isn't that NSS doesn't like the server's certificate.

I think "Peer does not recognize and trust the CA that issued your certificate." means the server (NSS's peer) is the one having trust issues. That this error is under the section "All the error codes in the following block indicate that the local socket received an SSL3 or TLS alert record from the remote peer, reporting some issue that it had with an SSL record or handshake message it received." supports this too.

I will produce a reproducible test case, however.

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Jun 2, 2016

Collaborator

On Thursday, June 02, 2016 02:22:23 Tom Fitzhenry wrote:

I think "Peer does not recognize and trust the CA that issued your
certificate." means the server (NSS's peer) is the one having trust issues.

Exactly. That is also my understanding of the issue. I suspect the reason
that peer does not trust your certificate is that the intermediate certificate
is not presented to the peer at all.

Collaborator

kdudka commented Jun 2, 2016

On Thursday, June 02, 2016 02:22:23 Tom Fitzhenry wrote:

I think "Peer does not recognize and trust the CA that issued your
certificate." means the server (NSS's peer) is the one having trust issues.

Exactly. That is also my understanding of the issue. I suspect the reason
that peer does not trust your certificate is that the intermediate certificate
is not presented to the peer at all.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jun 2, 2016

Oh, and when I tried the same scenario, but with Firefox (which uses NSS), it was able to connect, suggesting the problem is not NSS, but how curl uses NSS.

But anyway, I need to talk less, and produce a test case. :)

ghost commented Jun 2, 2016

Oh, and when I tried the same scenario, but with Firefox (which uses NSS), it was able to connect, suggesting the problem is not NSS, but how curl uses NSS.

But anyway, I need to talk less, and produce a test case. :)

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Jun 2, 2016

Collaborator

On Thursday, June 02, 2016 02:49:34 bbc-tomfitzhenry wrote:

Oh, and when I tried the same scenario, but with Firefox (which uses NSS),
it was able to connect, suggesting the problem is not NSS, but how curl
uses NSS.

Sort of. curl uses nss-pem to load certificates and keys from files, which
is a PKCS #11 module and definitely not something that Firefox would use.

Actually, you can try to use the SSL_DIR environment variable to make curl
use the certificate database from Firefox.

Collaborator

kdudka commented Jun 2, 2016

On Thursday, June 02, 2016 02:49:34 bbc-tomfitzhenry wrote:

Oh, and when I tried the same scenario, but with Firefox (which uses NSS),
it was able to connect, suggesting the problem is not NSS, but how curl
uses NSS.

Sort of. curl uses nss-pem to load certificates and keys from files, which
is a PKCS #11 module and definitely not something that Firefox would use.

Actually, you can try to use the SSL_DIR environment variable to make curl
use the certificate database from Firefox.

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Aug 2, 2016

Member

Out of date, this might be fixed if nss-pem is used. Feel free to (request a) reopen.

Member

bagder commented Aug 2, 2016

Out of date, this might be fixed if nss-pem is used. Feel free to (request a) reopen.

@bagder bagder closed this Aug 2, 2016

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 1, 2017

I just ran into the same issue. Here is my setup:

  • self-signed server, accepting a private root-CA
  • client with client-certificate, signed by intermediate CA, signed by private root-CA
    Using this version of curl provided by CentOS 7:
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

It seems, that the intermediate CA is not sent in the SSL handshake, so the server cannot verify the client certificate. If the server knows about the intermediate certificate or I use openssl s_client with the same certificates, everything works fine. Neither using --cert ./full-chain.crt nor --CAfile ./root-and-intermediate-ca.crt works with curl.

thraidh commented Mar 1, 2017

I just ran into the same issue. Here is my setup:

  • self-signed server, accepting a private root-CA
  • client with client-certificate, signed by intermediate CA, signed by private root-CA
    Using this version of curl provided by CentOS 7:
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

It seems, that the intermediate CA is not sent in the SSL handshake, so the server cannot verify the client certificate. If the server knows about the intermediate certificate or I use openssl s_client with the same certificates, everything works fine. Neither using --cert ./full-chain.crt nor --CAfile ./root-and-intermediate-ca.crt works with curl.

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Mar 1, 2017

Collaborator

Yes, nss-pem currently does not support client certificates signed by intermediate certificates. Please open a pull request at https://github.com/kdudka/nss-pem or at least help us with setting up a testing environment needed for development of the feature.

As a workaround, you can import the client certificate to the native NSS database and make libcurl use it by setting the SSL_DIR environment variable and selecting the client certificate by its nickname.

Collaborator

kdudka commented Mar 1, 2017

Yes, nss-pem currently does not support client certificates signed by intermediate certificates. Please open a pull request at https://github.com/kdudka/nss-pem or at least help us with setting up a testing environment needed for development of the feature.

As a workaround, you can import the client certificate to the native NSS database and make libcurl use it by setting the SSL_DIR environment variable and selecting the client certificate by its nickname.

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 1, 2017

I'm working on a script setting up a test environment and perform a test case.

thraidh commented Mar 1, 2017

I'm working on a script setting up a test environment and perform a test case.

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 2, 2017

Here is a script that produces a self-signed server certificate, a root CA, a client CA and a client certificate signed by the intermediate CA. After that it will run openssl s_server and openssl s_client using those certificates to show the good case and then openssl s_server and curl to show the difference.
It was tested on Fedora 22 with curl 7.40.0 using NSS/3.21 and openssl 1.0.1k to create the certificates.

setup-testcase.zip

thraidh commented Mar 2, 2017

Here is a script that produces a self-signed server certificate, a root CA, a client CA and a client certificate signed by the intermediate CA. After that it will run openssl s_server and openssl s_client using those certificates to show the good case and then openssl s_server and curl to show the difference.
It was tested on Fedora 22 with curl 7.40.0 using NSS/3.21 and openssl 1.0.1k to create the certificates.

setup-testcase.zip

kdudka added a commit to kdudka/curl that referenced this issue Mar 6, 2017

nss: load CA certificates even with --insecure
... because they may include an intermediate certificate for a _client_
certificate and the intermediate certificate needs to be presented to
the server, no matter if we verify the peer or not.

Reported-by: thraidh
Closes curl#851

@kdudka kdudka self-assigned this Mar 6, 2017

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Mar 6, 2017

Collaborator

Thank you very much for providing the setup script!

It looks like the only problem is that curl completely ignores the --cacert option if the -k option is used. Could you please check whether the following patch makes it work as expected?

master...kdudka:insecure-cacert

Collaborator

kdudka commented Mar 6, 2017

Thank you very much for providing the setup script!

It looks like the only problem is that curl completely ignores the --cacert option if the -k option is used. Could you please check whether the following patch makes it work as expected?

master...kdudka:insecure-cacert

@kdudka kdudka reopened this Mar 6, 2017

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 6, 2017

Thanks for your work. I'll check.

thraidh commented Mar 6, 2017

Thanks for your work. I'll check.

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 6, 2017

Just added the server.crt to the ca-bundle.crt and removed -k. This confirms that intermediate certificates are handled correctly in general.
Now I'll have to build curl from source to check if your change works for me. May take a while.

thraidh commented Mar 6, 2017

Just added the server.crt to the ca-bundle.crt and removed -k. This confirms that intermediate certificates are handled correctly in general.
Now I'll have to build curl from source to check if your change works for me. May take a while.

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Mar 6, 2017

Collaborator

Perfect. I can prepare a Copr repository for testing in case it helped. Just let me know which version of curl for which Fedora/EPEL release you prefer. If you are still using Fedora 22, it is no longer available as a choice in Copr but I can at least prepare a SRPM that you can easily rebuild locally.

Collaborator

kdudka commented Mar 6, 2017

Perfect. I can prepare a Copr repository for testing in case it helped. Just let me know which version of curl for which Fedora/EPEL release you prefer. If you are still using Fedora 22, it is no longer available as a choice in Copr but I can at least prepare a SRPM that you can easily rebuild locally.

@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 6, 2017

Fedora is just running on my desktop (for which I somehow don't find the time to upgrade) . The actual problem occurred on a CentOS 7, curl 7.29.0. I tried to apply your patch to the source RPM, but apparently the directory layout changed and then I got distracted by real life. I should be able to fix that tomorrow.

thraidh commented Mar 6, 2017

Fedora is just running on my desktop (for which I somehow don't find the time to upgrade) . The actual problem occurred on a CentOS 7, curl 7.29.0. I tried to apply your patch to the source RPM, but apparently the directory layout changed and then I got distracted by real life. I should be able to fix that tomorrow.

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Mar 7, 2017

Collaborator

Please try the packages of RHEL-7.3 curl with the above patch included from the following Copr:
https://copr.fedorainfracloud.org/coprs/kdudka/curl-testing/

Collaborator

kdudka commented Mar 7, 2017

Please try the packages of RHEL-7.3 curl with the above patch included from the following Copr:
https://copr.fedorainfracloud.org/coprs/kdudka/curl-testing/

kdudka added a commit to kdudka/curl that referenced this issue Mar 8, 2017

nss: load CA certificates even with --insecure
... because they may include an intermediate certificate for a client
certificate and the intermediate certificate needs to be presented to
the server, no matter if we verify the peer or not.

Reported-by: thraidh
Closes curl#851
@thraidh

This comment has been minimized.

Show comment
Hide comment
@thraidh

thraidh Mar 8, 2017

I'll test as soon as possible. Unfortunately I got sick, so it may take some time.

thraidh commented Mar 8, 2017

I'll test as soon as possible. Unfortunately I got sick, so it may take some time.

@kdudka

This comment has been minimized.

Show comment
Hide comment
@kdudka

kdudka Mar 9, 2017

Collaborator

No worries. This is broken for 8+ years, so a few more days or weeks to get it fixed is fine. I wish you to get well soon!

Collaborator

kdudka commented Mar 9, 2017

No worries. This is broken for 8+ years, so a few more days or weeks to get it fixed is fine. I wish you to get well soon!

@kdudka kdudka closed this in d29e9de Apr 10, 2017

@lock lock bot locked as resolved and limited conversation to collaborators May 6, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.