./configure'ing with built but not installed SSL breaks some features

[unlightable]

Maybe that's something I actually do wrong and it's not a legit problem, but still.

I did this

I'm trying to build libcurl as a shared library that depends on prebuilt libssl that differs from installed in the system.
That libssl is bundled with another libraries (including libcurl) and deployed with an executable that is built with $ORIGIN in rpath. This is not the issue, but that usage pattern leads to an actuall one.

./configure --prefix=/home/user --enable-ares=/home/user --with-ssl=/home/user does what it's suppose to do, but with a twist.

I experienced the following

./configure executes fine, but it fails some 'seems to work' tests like this:

...
configure:48929: checking if poll is prototyped
configure:48944: result: yes
configure:48960: checking if poll is compilable
configure:48989: gcc -c -Werror-implicit-function-declaration -O2 -Wno-system-headers    -I/home/user/include   conftest.c >&5
configure:48996: $? = 0
configure:49002: result: yes
configure:49022: checking if poll seems to work
configure:49079: gcc -o conftest -Werror-implicit-function-declaration -O2 -Wno-system-headers    -I/home/user/include      -L/home/user/lib   conftest.c -lssl -lcrypto -lssl -lcrypto   -lz -lrt >&5
configure:49083: $? = 0
configure:49089: ./conftest
./conftest: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
configure:49093: $? = 127
configure: program exited with status 127
...

There are about 8 test that fail in a similar fashion and disable some features, although for me disabling poll was critical.

The issue here, if I'm seeing it right, is that it can't load .so's that are not in ld search path, but still links against them.

I'm not really sure if

• those test do need to link against all libraries libcurl will be linked against
• LD_LIBRARY_PATH, modifying rpath of test executable or similar tools should be used

My dirty fix is to just put prebuilt libraries in ./ before running configure, but it seems... bad?

curl/libcurl version

$ ~/bin/curl --version
curl 7.61.1-DEV (x86_64-unknown-linux-gnu) libcurl/7.61.1-DEV OpenSSL/1.1.0h zlib/1.2.3 c-ares/1.14.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy

operating system

Centos 6

[bagder]

I'm not exactly sure what you're asking or saying. configure does checks of libraries, most of them by linking and some of them by actually running compiled code. Link-time is different than run-time. The configure script tries to make sure that what is found at its link-time checks is also used for its run-time checks.

Post configure, or post make even, when you run curl or use libcurl, making sure that the correct 3rd party libraries is used is your concern and you need to make sure that your ld.so links with the correct libs with LD_LIBRARY_PATH, editing /etc/ld.so.conf or similar.

[unlightable]

Post-configure is not an issue, it's my concern indeed.

I'm talking ./configure itself. It should check for available OS features and libraries, and it does what it should.
But applying --with-ssl=[path-to-libssl] breaks a few checks that are absolutely unrelated to SSL.
This leads to libcurl being built with those OS features disabled, although they are readily available.

[bagder]

applying --with-ssl=[path-to-libssl] breaks a few checks that are absolutely unrelated to SSL

It does? I build with that option very frequently and I have not experienced that. Which checks do you see it breaking?

[unlightable]

It does if you have no SSL of similar version installed system-wide.

A check I've been burnt by is a poll one. I've posted a cutout from config.log in original post.

But I think all of those are affected:

• getaddrinfo
• getifaddrs
• gmtime_r
• inet_ntop
• inet_pton
• localtime_r
• poll
• strerror_r

It all happens with an error like this:

configure:45268: checking if getifaddrs seems to work
configure:45313: gcc -o conftest -Werror-implicit-function-declaration -O2 -Wno-system-headers    -I/home/user/include      -L/home/user/lib   conftest.c -lssl -lcrypto -lssl -lcrypto   -lz -lrt >&5
configure:45317: $? = 0
configure:45323: ./conftest
./conftest: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

It CAN link test binary, but CAN'T run it.

[bagder]

./conftest: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

That's not what typically happens and looks like something is broken for you. It looks like it fails to set the correct path to load the library from and sure, then everything breaks from that point on.

[unlightable]

Well yeah, that's what I'm saying right from the start.

It links its tests to 'correct' SSL version, but tries to run in w/o including the path from --with-ssl=..., where those libs reside, in neither LD_LIBRARY_PATH nor rpath or any other ld-affecting mechanism. And consequently fails to run the test.

If you ./configure with --with-ssl (no path specification), it will use libssl installed system-wide and will both link and run all tests fine.

I'm not really savvy in linux builds, autotools and all that magic, but that behavior seems wrong for me.

[bagder]

It links its tests to 'correct' SSL version, but tries to run in w/o including the path from --with-ssl=..., where those libs reside, in neither LD_LIBRARY_PATH nor rpath or any other ld-affecting mechanism

That's not true. The configure.ac script uses the CURL_RUN_IFELSE macro when it needs to run something to be tested by executing code in the configure script, and that macro is defined here:

curl/m4/curl-functions.m4

Lines 7012 to 7027 in 4058cf2

	dnl CURL_RUN_IFELSE
	dnl -------------------------------------------------
	dnl Wrapper macro to use instead of AC_RUN_IFELSE. It
	dnl sets LD_LIBRARY_PATH locally for this run only, from the
	dnl CURL_LIBRARY_PATH variable. It keeps the LD_LIBRARY_PATH
	dnl changes contained within this macro.
	AC_DEFUN([CURL_RUN_IFELSE], [
	AC_REQUIRE([AC_RUN_IFELSE])dnl
	old=$LD_LIBRARY_PATH
	LD_LIBRARY_PATH=$CURL_LIBRARY_PATH:$old
	export LD_LIBRARY_PATH
	AC_RUN_IFELSE([AC_LANG_SOURCE([$1])], $2, $3, $4)
	LD_LIBRARY_PATH=$old # restore
	])

As you can see here, that macro sets LD_LIBRARY_PATH.

So, I understand that something breaks when you run configure but your analysis is directed to the wrong thing - the problem is not the lack of using the variable, the problem is either that it passes on the wrong directory or that you've found a case where it doesn't use this macro to execute the check. Also remember that countless of people already run and use this configure script without these problems.

If you would attach your config.log file, we might be able to help you narrow this search faster.

[jay]

We don't support "not installed" OpenSSL, you have to actually install it to some location first and then point configure to that location. However it sounds like you may have OpenSSL installed just independent and in a path separate from the packaged one. In other words there's actually a /home/user/lib with the import library for libssl.so. Try setting the rpath before invoking configure, for example assume I installed OpenSSL to /usr/local/ssl and the imports are in /usr/local/ssl/lib then:

LDFLAGS="-Wl,-rpath,/usr/local/ssl" ./configure --with-ssl=/usr/local/ssl ...

[jay]

oops I meant
LDFLAGS="-Wl,-rpath,/usr/local/ssl/lib" ./configure --with-ssl=/usr/local/ssl ...
edit button isn't working or I would edit my previous post

[unlightable]

If you would attach your config.log file, we might be able to help you narrow this search faster.

Sure!
config.log.zip

As you can see here, that macro sets LD_LIBRARY_PATH.

As I said, I'm really bad at Autotools, so pardon me for being wrong there.
But I'm also curious enough. So I've hijacked that poll test (while forcing it to pass by applying that dirty workaround from my first post):

...
#include <unistd.h>
#include <stdio.h>

extern char **environ;

int main (void)
{

        FILE* f = fopen("/home/user/conftest_env","w");
        char** p;
        for( p = environ; *p != NULL; p++ )
                fprintf(f,"%s\n",*p);
...

and contents of conftest_env are:

CURL_LIBRARY_PATH=:/home/user/lib
HOSTNAME=cloud_linuxbldr
SELINUX_ROLE_REQUESTED=
SHELL=/bin/sh
TERM=xterm
HISTSIZE=1000
SSH_CLIENT=192.168.20.155 60676 22
SELINUX_USE_CURRENT_RANGE=
OLDPWD=/home/user/curl_remote_build/curl-curl-7_61_1/conftest
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
DUALCASE=1
SSH_TTY=/dev/pts/0
LC_ALL=C
USER=user
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
LD_LIBRARY_PATH=/home/user/gcc/rtf/lib:/home/user/gcc/rtf/lib64:
PATH=/home/user/gcc/rtf/bin/:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/user/bin
PWD=/home/user/curl_remote_build/curl-curl-7_61_1
LANG=C
SELINUX_LEVEL_REQUESTED=
HISTCONTROL=ignoredups
HOME=/home/user
SHLVL=2
LANGUAGE=C
LOGNAME=user
CVS_RSH=ssh
QTLIB=/usr/lib64/qt-3.3/lib
as_nl=

SSH_CONNECTION=192.168.20.155 60676 192.168.20.219 22
LESSOPEN=||/usr/bin/lesspipe.sh %s
wl=-Wl,
G_BROKEN_FILENAMES=1
HISTTIMEFORMAT=%d/%m/%y %T
_=./conftest

There is LD_LIBRARY_PATH indeed, but it's the one ./configure is run with, w/o added $CURL_LIBRARY_PATH

I've failed to mention that gcc that curl is built with is installed locally too, thats why LD_LIBRARY_PATH isn't empty from the start.
And it's set from ~/.bashrc:

export LD_LIBRARY_PATH="/home/builduser/gcc/rtf/lib:/home/builduser/gcc/rtf/lib64:$LD_LIBRARY_PATH"

Could it be the culprit?

We don't support "not installed" OpenSSL, you have to actually install it to some location first and then point configure to that location.

Erm, I probably named it wrong. OpenSSL was built and, well, make install'ed into ~/, just not system-wide.

LDFLAGS="-Wl,-rpath,/usr/local/ssl/lib"

This will probably help, but it also will make a final library have that rpath embedded into it, right?
It's not very harmful, I think, but still seems like a black magic trick, instead of just using --with-ssl=...?

[bagder]

Thanks! I'm now pretty sure this is because of a few places where AC_RUN_IFELSE is still used instead of CURL_RUN_IFELSE. I'll write up a PR for it...