curl 7.60.0 can list & work with openssl engines while curl 7.61.1 can't

[jeanfabrice]

In the following, openssl is openssl @1.0.2p_0. The same openssl.cnf is used for both test

I did this

$ sudo port installed curl
The following ports are currently installed:
  curl @7.59.0_0+ssl
  curl @7.60.0_0+ssl (active)
  curl @7.61.0_0+ssl 
$ curl --version
curl 7.60.0 (x86_64-apple-darwin17.5.0) libcurl/7.60.0 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5)
Release-Date: 2018-05-16
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy PSL 
$ curl --engine list
Build-time engines:
  rdrand
  dynamic
  pkcs11

$ sudo port activate curl @7.61.0_0+ssl
--->  Computing dependencies for curl
--->  Deactivating curl @7.60.0_0+ssl
--->  Cleaning curl
--->  Activating curl @7.61.0_0+ssl
--->  Cleaning curl
$ curl --version
curl 7.61.0 (x86_64-apple-darwin17.6.0) libcurl/7.61.0 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5)
Release-Date: 2018-07-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy PSL
$ curl --engine list
Build-time engines:
  <none>

I expected the following

$ curl --version
curl 7.61.0 (x86_64-apple-darwin17.6.0) libcurl/7.61.0 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5)
Release-Date: 2018-07-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy PSL
$ curl --engine list
Build-time engines:
  rdrand
  dynamic
  pkcs11

operating system

Mac OS 10.13.6 & MacPorts 2.5.3

[bagder]

Thanks, but this issue is already fixed.

Fixed in d6417f6 (7.61.1) which made it work with OpenSSL 1.0.0+

and then again in 1599dfc (not released yet) which made it work with OpenSSL 0.9.8+

Duplicate of #2732

[jeanfabrice]

Well I tried with 7.61.1 too. No more success :

$ sudo port installed curl
The following ports are currently installed:
  curl @7.59.0_0+ssl
  curl @7.60.0_0+ssl
  curl @7.61.0_0+ssl
  curl @7.61.1_0+ssl (active)
$ curl -V
curl 7.61.1 (x86_64-apple-darwin17.7.0) libcurl/7.61.1 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5)
Release-Date: 2018-09-05
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy PSL 
$ curl --engine list
Build-time engines:
  <none>

[bagder]

Then there's a different problem...

[jeanfabrice]

No luck in bisecting the 158 commits between 7_60_0 and 7_61_0. Lot of unrelated errors at compile time.

[bagder]

That sounds even weirder. Most commits we merge have been verified to build cleanly on numerous platforms and build combos. So even if there's an occasional build failure, they should be rare.

[jeanfabrice]

You are right, sorry. Mistake was on my side because I did not run buildconf between each bisect.

So, bisecting properly between 7_60_0 and 7_61_0 confirms that 38203f1 is the first bad commit.

Unfortunately, d6417f6 does not solve the issue; SSL engine list is still empty after this commit. No more success after checking out 1599dfc.

[bagder]

Is this the missing piece?

diff --git a/lib/vtls/openssl.c b/lib/vtls/openssl.c
index 230eea2d0..551298926 100644
--- a/lib/vtls/openssl.c
+++ b/lib/vtls/openssl.c
@@ -976,11 +976,11 @@ static int Curl_ossl_init(void)
   char *keylog_file_name;
 #endif
 
   OPENSSL_load_builtin_modules();
 
-#ifdef HAVE_ENGINE_LOAD_BUILTIN_ENGINES
+#ifdef USE_OPENSSL_ENGINE
   ENGINE_load_builtin_engines();
 #endif
 
   /* OPENSSL_config(NULL); is "strongly recommended" to use but unfortunately
      that function makes an exit() call on wrongly formatted config files

[jeanfabrice]

That is the perfect missing piece, thank you!
Works exactly as expected on both curl-7_61_0 and curl-7_61_1 after patching them

[bagder]

excellent! 👍