Cannot build curl 7.81.0 with --with-wolfssl

[harrysarson]

I did this

CURL_VERSION=curl-7.81.0

curl https://curl.se/download/$CURL_VERSION.tar.gz | tar -xz
curl -L https://github.com/wolfSSL/wolfssl/archive/refs/tags/v4.8.1-stable.tar.gz | tar -xz

mkdir wolfssl-install
mkdir wolfssl-dir

ls

pushd wolfssl-4.8.1-stable
    
autoreconf -fi
./configure --prefix=$(realpath ../wolfssl-install)  --enable-opensslextra
make -j$(nproc) 
make test 
make install

popd
pushd $CURL_VERSION

autoreconf -fi
./configure --with-wolfssl=$(realpath ../wolfssl-install) 
make -j$(nproc)

popd

I expected the following

Curl can build with wolfssl (all the commands run successfully), as it does with CURL_VERSION=curl-7.79.1. Instead it fails to build with error:

md5.c:85:10: fatal error: openssl/md5.h: No such file or directory
   85 | #include <openssl/md5.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
In file included from http_aws_sigv4.c:33:
curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:2474: libcurl_la-md5.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from sha256.c:29:
curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:2404: libcurl_la-http_aws_sigv4.lo] Error 1
make[2]: *** [Makefile:2621: libcurl_la-sha256.lo] Error 1
In file included from vauth/digest.c:38:
../lib/curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.

(Note CURL_VERSION=curl-7.80.0 also fails).

curl/libcurl version

7.81.0

operating system

WSL: Linux D-00076 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Notes

I suspect #7806 is at fault here. It expects that the wolfssl openssl compatability layer uses openssl/*.h include paths. However, (atleast as my reading of https://www.wolfssl.com/docs/wolfssl-manual/ch13/) the wolfssl openssl compatability layer actually puts the header files at wolfssl/openssl/*.h:

I have a patch that fixes this for me locally that I can submit if the above conclusion is correct.

[jay]

I get a bunch of errors with master and less with 7.79.1. The errors I get in 7.79.1 are because of # include <openssl/ssl.h>. If the compatibility layer is enabled is it supposed to include the parent wolfssl include directory? I'm using VS project files which include only the wolfssl source directory and then one can write #include <wolfssl/... to include from include directory wolfssl/wolfssl but not #include <openssl/... to include wolfssl/wolfssl/openssl.

[bagder]

--enable-opensslextra

You probably need more openssl capability enabled. Try with --enable-all first and see if that doesn't get you going.

It expects that the wolfssl openssl compatability layer uses openssl/.h include paths. However, (atleast as my reading of https://www.wolfssl.com/docs/wolfssl-manual/ch13/) the wolfssl openssl compatability layer actually puts the header files at wolfssl/openssl/.h:

The configure script adapts to that. We successfully build with wolfSSL in our CI builds for every commit and for every PR after all...

[bagder]

If the compatibility layer is enabled is it supposed to include the parent wolfssl include directory

The compatibility layer needs to be enabled for curl to build with wolfSSL.

[bagder]

Try with --enable-all first

I should mention that wolfSSL will soon ship with an --enable-curl option for their configure which will make this process a little easier...

[jay]

Ref: https://github.com/curl/curl/blob/curl-7_81_0/m4/curl-wolfssl.m4

What does pkg-config --cflags-only-I wolfssl show? It looks like we don't add wolfssl/openssl directly so the only way it would be added is if it came from pkg-config.

You probably need more openssl capability enabled. Try with --enable-all first and see if that doesn't get you going

I don't think it is practical to enable-all. There may be some flag that does it, but I can't find it. (edit: Fair enough, as a diagnostic --enable-all may be helpful here.)

[bagder]

It looks like we don't add wolfssl/openssl directly

We add the wolfssl part only, as the openssl part is what we use in the include names:

curl/m4/curl-wolfssl.m4

Lines 137 to 152 in fde0925

	AC_DEFINE(HAVE_WOLFSSL_DES_ECB_ENCRYPT, 1,
	[if you have wolfSSL_DES_ecb_encrypt])
	if test -n "$addcflags"; then
	dnl use a for loop to strip off whitespace
	for f in $addcflags; do
	CPPFLAGS="$f/wolfssl $CPPFLAGS"
	AC_MSG_NOTICE([Add $f/wolfssl to CPPFLAGS])
	break
	done
	else
	dnl user didn't give a path, so guess/hope they installed wolfssl
	dnl headers to system default location
	CPPFLAGS="-I/usr/include/wolfssl $CPPFLAGS"
	AC_MSG_NOTICE([Add /usr/include/wolfssl to CPPFLAGS])
	fi
	WOLFSSL_NTLM=1

[jay]

We add the wolfssl part only, as the openssl part is what we use in the include names:

Ok I missed that was added when wolfSSL_DES_ecb_encrypt.

[bagder]

When I added that logic, curl would still build without that function and just have less functionality. Since then we broke that flexibility. We should probably make configure check for the functions we need instead of causing link errors much later.

[harrysarson]

hey, I confirmed that the build issue I was facing seems resolved by configuring wolfssl with --enable-all, thanks!

Would there be interest in keeping support for building curl against wolfssl configured with just --enable-opensslextra? I don't think the changes needed to restore support would be too great.

[bagder]

I think it would be cool to make curl adapt to whatever features wolfSSL was built with, instead of us insisting on a particular feature set: yes.

[jay]

Couldn't there also be a real openssl include directory (like say in /usr/local/include/openssl) that would end up taking precedence before wolfssl/wolfssl/openssl?