SSL Error on macOS, when --insecure flag is given. #1184

fchandur · 2016-12-30T23:03:32Z

I did this

./curl --insecure https://www.google.com

I expected the following

html document from google:
<!doctype html><html itemscope="" itemtype="...

Instead, I got this message

curl: (35) SSL: CA certificate set, but certificate verification is disabled

curl/libcurl version

$ ./curl --version
curl 7.52.1 (x86_64-apple-darwin16.3.0) libcurl/7.52.1 SecureTransport zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS Debug TrackMemory IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets

operating system

System Version: macOS 10.12.2 (16C67)
Kernel Version: Darwin 16.3.0

configuration

$ ./curl-config --configure
 '--with-darwinssl' '--enable-threaded-resolver' '--enable-debug' '--without-libssh2' '--prefix=/Users/user/Downloads/curl-7.52.1/build'

verbose

$ ./curl --insecure --verbose https://www.google.com
* STATE: INIT => CONNECT handle 0x7fd920801a08; line 1413 (connection #-5000)
* Rebuilt URL to: https://www.google.com/
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x7fd920801a08; line 1450 (connection #0)
*   Trying 172.217.6.100...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x7fd920801a08; line 1531 (connection #0)
* Connected to www.google.com (172.217.6.100) port 443 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x7fd920801a08; line 1583 (connection #0)
* Marked for [keep alive]: HTTP default
* SSL: CA certificate set, but certificate verification is disabled
* Marked for [closure]: Failed HTTPS connection
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x7fd920801a08; line 1597 (connection #0)
* Expire cleared
* multi_done
* Curl_http_done: called premature == 0
* Closing connection 0
* The cache now contains 0 members
curl: (35) SSL: CA certificate set, but certificate verification is disabled

On an older osx, with the same version of libcurl as above, and same configuration, I do get the expected behavior. Following are some of the details:

curl/libcurl version

./curl --version
curl 7.52.1 (x86_64-apple-darwin15.6.0) libcurl/7.52.1 SecureTransport zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets

configuration

$ ./curl-config --configure
'--without-libssh2' '--enable-threaded-resolver' '--with-darwinssl' '--prefix=/Users/user/Downloads/curl-7.52.1/build'

I did this

$ ./curl --insecure https://www.google.com

I got this

html document from google:
<!doctype html><html itemscope="" itemtype="...

The text was updated successfully, but these errors were encountered:

nickzman · 2016-12-30T23:40:51Z

Not an issue. You get that error if you put curl into insecure mode while simultaneously specifying a server certificate file or bundle.

In general, you shouldn't specify a server certificate file or bundle using curl on macOS unless you really know what you're doing. The option is there, and is intended for backward compatibility with scripts that also have to run on other operating systems where using the Keychain is not an option.

If that does not describe your use case, then just let curl use the Keychain for trust evaluation. And if you have any self-signed certificates, then use certtool or Keychain Access to add them to your Keychain.

jay · 2016-12-31T00:00:13Z

If insecure mode is specified why don't we just ignore the bundle? I can see how it might be beneficial to put --cacert FILE into a curlrc and then let's say on the command line you want to go insecure, why does it matter if the bundle was specified?

nickzman · 2016-12-31T00:34:54Z

The --cacert/--capath and --insecure options contradict each other. That would be okay on back-ends that wholly rely on a third-party certificate database, e.g. OpenSSL. But Secure Transport is intended to be used with macOS' Keychain as a certificate database, with a third-party option only in place for backward compatibility. Since they're overriding the default behavior in contradictory ways, I think it makes sense to tell the user to pick one way or the other.

We do also say in the documentation that, when Secure Transport is in use, the --cacert option is only provided for backward compatibility reasons.

jay · 2016-12-31T02:52:54Z

I see your point about the contradiction but I disagree. I'm saying there are reasonable scenarios where it may happen there are contradictory options. Someone may have in their script or their program an option that adds the cacert, and then later on in some other function in their program or point in their script they decide to add insecure for that transfer. AFAICT with every other backend insecure will work regardless of cacert. Is there any reason why we can't do that for Secure Transport as well, and wouldn't it also be helpful for backwards compatibility and consistency?

fchandur · 2016-12-31T03:31:35Z

Not an issue. You get that error if you put curl into insecure mode while simultaneously specifying a server certificate file or bundle.

I am not specifying a server certificate file or bundle, not explicitly, at least. Also, I am pointing out that same version of curl, built with same configuration options, behaves differently on different OSX versions.

Here is the code that was generated.

/********* Sample code generated by the curl command line tool **********
 * All curl_easy_setopt() options are documented at:
 * https://curl.haxx.se/libcurl/c/curl_easy_setopt.html
 ************************************************************************/
#include <curl/curl.h>

int main(int argc, char *argv[])
{
  CURLcode ret;
  CURL *hnd;

  hnd = curl_easy_init();
  curl_easy_setopt(hnd, CURLOPT_URL, "https://www.google.com");
  curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L);
  curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.52.1");
  curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L);
  curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYPEER, 0L);
  curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYHOST, 0L);
  curl_easy_setopt(hnd, CURLOPT_VERBOSE, 1L);
  curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L);

  /* Here is a list of options the curl code used that cannot get generated
     as source easily. You may select to either not use them or implement
     them yourself.

  CURLOPT_WRITEDATA set to a objectpointer
  CURLOPT_INTERLEAVEDATA set to a objectpointer
  CURLOPT_WRITEFUNCTION set to a functionpointer
  CURLOPT_READDATA set to a objectpointer
  CURLOPT_READFUNCTION set to a functionpointer
  CURLOPT_SEEKDATA set to a objectpointer
  CURLOPT_SEEKFUNCTION set to a functionpointer
  CURLOPT_ERRORBUFFER set to a objectpointer
  CURLOPT_STDERR set to a objectpointer
  CURLOPT_DEBUGFUNCTION set to a functionpointer
  CURLOPT_DEBUGDATA set to a objectpointer
  CURLOPT_HEADERFUNCTION set to a functionpointer
  CURLOPT_HEADERDATA set to a objectpointer

  */

  ret = curl_easy_perform(hnd);

  curl_easy_cleanup(hnd);
  hnd = NULL;

  return (int)ret;
}
/**** End of sample code ****/

bagder · 2017-01-02T11:26:16Z

I don't see how @fchandur specifed an additional cacert file and I agree with @jay that if --insecure is asked for, we should be able to totally ignore all ca cert files and options since the client asked to not verify the server at all.

nickzman · 2017-01-03T06:37:50Z

Then the attached patch ought to allow the contradictory commands. Does it solve the original report?

Also, it is possible to specify the certificate bundle in the curlrc, isn't it?
allow-cafile-and-verifypeer.txt

fchandur · 2017-01-03T16:00:25Z

It does. Thank you.

Also, when invoking the configure script with the argument --with-darwinssl, by default, the the flag --with-ca-bundle=/etc/ssl/cert.pem is also set.

I think the default, especially when using SecureTransport, should be --without-ca-bundle, no?

configure:

./configure  --with-darwinssl

relevant output:

  ....
  SSL support:      enabled (Apple OS-native)
  ...
  ca cert bundle:   /etc/ssl/cert.pem
  ca cert path:     no
  ca fallback:      no

nickzman · 2017-01-03T23:52:21Z

Thanks. Daniel might know more, but I think that certificate bundle is always listed there but not actually used in the ST engine. I still suspect you have something in your curlrc file that is using a certificate bundle somewhere if you're not using the --cacert option.

nickzman closed this as completed Dec 30, 2016

jay added the TLS label Dec 31, 2016

bagder reopened this Jan 2, 2017

nickzman closed this as completed in ffbb0f0 Jan 3, 2017

lock bot locked as resolved and limited conversation to collaborators May 7, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SSL Error on macOS, when --insecure flag is given. #1184

SSL Error on macOS, when --insecure flag is given. #1184

fchandur commented Dec 30, 2016

nickzman commented Dec 30, 2016

jay commented Dec 31, 2016 •

edited

Loading

nickzman commented Dec 31, 2016

jay commented Dec 31, 2016

fchandur commented Dec 31, 2016

bagder commented Jan 2, 2017

nickzman commented Jan 3, 2017

fchandur commented Jan 3, 2017

nickzman commented Jan 3, 2017

SSL Error on macOS, when --insecure flag is given. #1184

SSL Error on macOS, when --insecure flag is given. #1184

Comments

fchandur commented Dec 30, 2016

I did this

I expected the following

Instead, I got this message

curl/libcurl version

operating system

configuration

verbose

curl/libcurl version

configuration

I did this

I got this

nickzman commented Dec 30, 2016

jay commented Dec 31, 2016 • edited Loading

nickzman commented Dec 31, 2016

jay commented Dec 31, 2016

fchandur commented Dec 31, 2016

bagder commented Jan 2, 2017

nickzman commented Jan 3, 2017

fchandur commented Jan 3, 2017

configure:

relevant output:

nickzman commented Jan 3, 2017

jay commented Dec 31, 2016 •

edited

Loading