Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgconfig: Wrong libdir? #23569

Closed
hlandau opened this issue Feb 13, 2024 · 23 comments
Closed

pkgconfig: Wrong libdir? #23569

hlandau opened this issue Feb 13, 2024 · 23 comments
Labels
triaged: bug The issue/pr is/fixes a bug

Comments

@hlandau
Copy link
Member

hlandau commented Feb 13, 2024

@icing reported a minor issue with the pkgconfig files here: #23339 (reply in thread)

fyi: had some trouble building, since openssl puts the wrong libdir in the pkgconfig files (just the top level dir and not the /lib underneath).

@levitte Any ideas?

@hlandau hlandau added the triaged: bug The issue/pr is/fixes a bug label Feb 13, 2024
@levitte
Copy link
Member

levitte commented Feb 13, 2024

Er no... not for the moment. I'll have a look.

@levitte
Copy link
Member

levitte commented Feb 13, 2024

I think this is a user error, actually. Let me ask in the discussion thread

@icing
Copy link

icing commented Feb 13, 2024

@levitte I ran

./config --prefix=/opt/quic/openssl-master --libdir=/opt/quic/openssl-master/lib

and /opt/quic/openssl-master/lib/pkgconfig/openssl.pc showed:

libdir=/opt/quic/openssl-master
includedir=/opt/quic/openssl-master/include

Name: OpenSSL
Description: Secure Sockets Layer and cryptography libraries and tools
Version: 3.3.0-dev
Requires: libssl libcrypto

@levitte
Copy link
Member

levitte commented Feb 13, 2024

So, we do have an issue, that it's pretty unclear what's the difference between $BLDDIR/libcrypto.pc and $BLDDIR/exporters/libcrypto.pc. I'll arrange some commentary

@icing
Copy link

icing commented Feb 13, 2024

fyi: same result when I omitted the --libdir=, but we use that regularly due to having seen lib64 being used on some system...

@levitte
Copy link
Member

levitte commented Feb 13, 2024

Oh, er, you're using --libdir incorrectly. It shouldn't be a full path, and leaving it empty is a really bad idea

openssl/INSTALL.md

Lines 338 to 350 in 0969375

### libdir
--libdir=DIR
The name of the directory under the top of the installation directory tree
(see the `--prefix` option) where libraries will be installed. By default
this is `lib`. Note that on Windows only static libraries (`*.lib`) will
be stored in this location. Shared libraries (`*.dll`) will always be
installed to the `bin` directory.
Some build targets have a multilib postfix set in the build configuration.
For these targets the default libdir is `lib<multilib-postfix>`. Please use
`--libdir=lib` to override the libdir if adding the postfix is undesirable.

@icing
Copy link

icing commented Feb 13, 2024

Sorry, I expressed myself confusingly. I wanted to say: the same resulting pkgconfig files are generated when I do not specify the libdir arg to configure.

@levitte
Copy link
Member

levitte commented Feb 13, 2024

Hmmm, that's curious. I'll double check, but I'm pretty sure I don't get that

@icing
Copy link

icing commented Feb 13, 2024

@levitte send your machine over to me immediately!

I run on macOS 14.2.1 (23C71).

@levitte
Copy link
Member

levitte commented Feb 13, 2024

Heh 😉

Awright, I'm getting what I expected (with /lib64 in the libdir paths). Maybe you're running into some odd path corner case I haven't encountered. I could try reproduce your circumstances, which is essentially an in-source-tree build. Do I get it right that you're observing this oddity with the following config command?

./config --prefix=/opt/quic/openssl-master

@levitte
Copy link
Member

levitte commented Feb 13, 2024

Still not.

$ cat /opt/quic/openssl-master/lib64/pkgconfig/libcrypto.pc 
libdir=/opt/quic/openssl-master/lib64
includedir=/opt/quic/openssl-master/include
enginesdir=/opt/quic/openssl-master/lib64/engines-3
modulesdir=/opt/quic/openssl-master/lib64/ossl-modules

Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 3.3.0-dev
Libs: -L${libdir} -lcrypto
Libs.private: -ldl -pthread
Cflags: -I${includedir}
$ cat /opt/quic/openssl-master/lib64/pkgconfig/libssl.pc 
libdir=/opt/quic/openssl-master/lib64
includedir=/opt/quic/openssl-master/include

Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 3.3.0-dev
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Cflags: -I${includedir}

The way this is done is that Makefile creates installdata.pm, and what's there is then used to generate exporters/libcrypto.pc from exporters/pkg-config/libcrypto.pc.in.

Here's how it's done in my setup:

$ grep mkinstallvars Makefile
builddata.pm: util/mkinstallvars.pl 
	$(PERL) util/mkinstallvars.pl PREFIX=. BINDIR=apps LIBDIR= INCLUDEDIR=include APPLINKDIR=ms ENGINESDIR=engines MODULESDIR=providers "VERSION=$(VERSION)" "LDLIBS=$(LIB_EX_LIBS)" > $@
installdata.pm: util/mkinstallvars.pl 
	$(PERL) util/mkinstallvars.pl "PREFIX=$(INSTALLTOP)" BINDIR=bin "LIBDIR=$(LIBDIR)" INCLUDEDIR=include APPLINKDIR=include/openssl "ENGINESDIR=$(ENGINESDIR)" "MODULESDIR=$(MODULESDIR)" "PKGCONFIGDIR=$(PKGCONFIGDIR)" "CMAKECONFIGDIR=$(CMAKECONFIGDIR)" "LDLIBS=$(LIB_EX_LIBS)" "VERSION=$(VERSION)" > $@

The LIBDIR var in those command lines is the variable we're trying to figure out. In my Makefile, it looks like this:

$ grep ^LIBDIR= Makefile
LIBDIR=lib64

Could you have a look to see what this looks like on your end?

@icing
Copy link

icing commented Feb 13, 2024

Ran:

>  rm exporters/openssl.pc
> ./config --prefix=/opt/quic/openssl-master
> make install_sw
...
> cat /opt/quic/openssl-master/lib/pkgconfig/libcrypto.pc 
libdir=/opt/quic/openssl-master
includedir=/opt/quic/openssl-master/include
enginesdir=/opt/quic/openssl-master/lib/engines-3
modulesdir=/opt/quic/openssl-master/lib/ossl-modules

Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 3.3.0-dev
Libs: -L${libdir} -lcrypto
Libs.private:
Cflags: -I${includedir}
> cat /opt/quic/openssl-master/lib/pkgconfig/libssl.pc
libdir=/opt/quic/openssl-master
includedir=/opt/quic/openssl-master/include

Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 3.3.0-dev
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Cflags: -I${includedir}

Then

> grep mkinstallvars Makefile
builddata.pm: util/mkinstallvars.pl
	$(PERL) util/mkinstallvars.pl PREFIX=. BINDIR=apps LIBDIR= INCLUDEDIR=include APPLINKDIR=ms ENGINESDIR=engines MODULESDIR=providers "VERSION=$(VERSION)" "LDLIBS=$(LIB_EX_LIBS)" > $@
installdata.pm: util/mkinstallvars.pl
	$(PERL) util/mkinstallvars.pl "PREFIX=$(INSTALLTOP)" BINDIR=bin "LIBDIR=$(LIBDIR)" INCLUDEDIR=include APPLINKDIR=include/openssl "ENGINESDIR=$(ENGINESDIR)" "MODULESDIR=$(MODULESDIR)" "PKGCONFIGDIR=$(PKGCONFIGDIR)" "CMAKECONFIGDIR=$(CMAKECONFIGDIR)" "LDLIBS=$(LIB_EX_LIBS)" "VERSION=$(VERSION)" > $@
> grep ^LIBDIR= Makefile
LIBDIR=lib
> cat exporters/openssl.pc
libdir=/opt/quic/openssl-master
includedir=/opt/quic/openssl-master/include

Name: OpenSSL
Description: Secure Sockets Layer and cryptography libraries and tools
Version: 3.3.0-dev
Requires: libssl libcrypto

@levitte
Copy link
Member

levitte commented Feb 13, 2024

That's infuriating...

Can you help figure this out? I'm guessing that something odd is going on in util/mkinstallvars.pl. This patch should help showing what it sees:

diff --git a/util/mkinstallvars.pl b/util/mkinstallvars.pl
index 59a432d28c..3ba88b5c38 100644
--- a/util/mkinstallvars.pl
+++ b/util/mkinstallvars.pl
@@ -21,6 +21,7 @@ my @subdirs = qw(BINDIR LIBDIR INCLUDEDIR APPLINKDIR ENGINESDIR MODULESDIR
 
 my %keys = ();
 foreach (@ARGV) {
+    print STDERR "[DEBUG] ARG: $_\n";
     (my $k, my $v) = m|^([^=]*)=(.*)$|;
     $keys{$k} = 1;
     $ENV{$k} = $v;
@@ -47,8 +48,10 @@ foreach my $k (sort keys %keys) {
             $ENV{"${k}_REL"} = $v;
             $v = File::Spec->rel2abs($v, $ENV{PREFIX});
         }
+        print STDERR "[DEBUG] \$ENV{${k}_REL} = $ENV{'${k}_REL'}\n";
     }
     $ENV{$k} = $v;
+    print STDERR "[DEBUG] \$ENV{$k} = $ENV{$k}\n";
 }
 
 print <<_____;

And then just do this:

$ rm installdata.pm
$ make exporters/libcrypto.pc

Hopefully, the output will tell us something.

It may turn out that this is a bug in perl's File::Spec we've encountered before, where one of rel2abs or abs2rel doesn't work quite right.

Speaking of which, perl version?

@icing
Copy link

icing commented Feb 13, 2024

> make exporters/libcrypto.pc
perl util/mkinstallvars.pl "PREFIX=/opt/quic/openssl-master" BINDIR=bin "LIBDIR=lib" INCLUDEDIR=include APPLINKDIR=include/openssl "ENGINESDIR=/opt/quic/openssl-master/lib/engines-3" "MODULESDIR=/opt/quic/openssl-master/lib/ossl-modules" "PKGCONFIGDIR=/opt/quic/openssl-master/lib/pkgconfig" "CMAKECONFIGDIR=/opt/quic/openssl-master/lib/cmake/OpenSSL" "LDLIBS= " "VERSION=3.3.0-dev" > installdata.pm
[DEBUG] ARG: PREFIX=/opt/quic/openssl-master
[DEBUG] ARG: BINDIR=bin
[DEBUG] ARG: LIBDIR=lib
[DEBUG] ARG: INCLUDEDIR=include
[DEBUG] ARG: APPLINKDIR=include/openssl
[DEBUG] ARG: ENGINESDIR=/opt/quic/openssl-master/lib/engines-3
[DEBUG] ARG: MODULESDIR=/opt/quic/openssl-master/lib/ossl-modules
[DEBUG] ARG: PKGCONFIGDIR=/opt/quic/openssl-master/lib/pkgconfig
[DEBUG] ARG: CMAKECONFIGDIR=/opt/quic/openssl-master/lib/cmake/OpenSSL
[DEBUG] ARG: LDLIBS=
[DEBUG] ARG: VERSION=3.3.0-dev
[DEBUG] $ENV{APPLINKDIR_REL} =
[DEBUG] $ENV{APPLINKDIR} = /opt/quic/openssl-master/include/openssl
[DEBUG] $ENV{BINDIR_REL} =
[DEBUG] $ENV{BINDIR} = /opt/quic/openssl-master/bin
[DEBUG] $ENV{CMAKECONFIGDIR_REL} =
[DEBUG] $ENV{CMAKECONFIGDIR} = /opt/quic/openssl-master/lib/cmake/OpenSSL
[DEBUG] $ENV{ENGINESDIR_REL} =
[DEBUG] $ENV{ENGINESDIR} = /opt/quic/openssl-master/lib/engines-3
[DEBUG] $ENV{INCLUDEDIR_REL} =
[DEBUG] $ENV{INCLUDEDIR} = /opt/quic/openssl-master/include
[DEBUG] $ENV{LDLIBS} =
[DEBUG] $ENV{LIBDIR_REL} =
[DEBUG] $ENV{LIBDIR} = /opt/quic/openssl-master/lib
[DEBUG] $ENV{MODULESDIR_REL} =
[DEBUG] $ENV{MODULESDIR} = /opt/quic/openssl-master/lib/ossl-modules
[DEBUG] $ENV{PKGCONFIGDIR_REL} =
[DEBUG] $ENV{PKGCONFIGDIR} = /opt/quic/openssl-master/lib/pkgconfig
[DEBUG] $ENV{PREFIX} = /opt/quic/openssl-master
[DEBUG] $ENV{VERSION} = 3.3.0-dev
perl "-I." "-Mconfigdata" "-Minstalldata" "util/dofile.pl" "-oMakefile" exporters/pkg-config/libcrypto.pc.in > exporters/libcrypto.pc

Standard macOS /usr/bin/perl:

> perl -V
Summary of my perl5 (revision 5 version 30 subversion 3) configuration:
...

@levitte
Copy link
Member

levitte commented Feb 13, 2024

Er... that looks like what I expected. Not entirely, the _REL variables look strangely empty. That doesn't matter for .pc files.

Last check to do, show and tell style:

$ make exporters/openssl.pc
$ grep LIBDIR builddata.pm installdata.pm
builddata.pm:                  $LIBDIR $LIBDIR_REL
builddata.pm:our $LIBDIR             = '/home/levitte/tmp/openssl-master';
builddata.pm:our $LIBDIR_REL         = '.';
installdata.pm:                  $LIBDIR $LIBDIR_REL
installdata.pm:our $LIBDIR             = '/opt/quic/openssl-master/lib64';
installdata.pm:our $LIBDIR_REL         = 'lib64';
$ grep libdir exporters/*.pc
exporters/libcrypto.pc:libdir=/opt/quic/openssl-master/lib64
exporters/libcrypto.pc:Libs: -L${libdir} -lcrypto
exporters/libssl.pc:libdir=/opt/quic/openssl-master/lib64
exporters/libssl.pc:Libs: -L${libdir} -lssl
exporters/openssl.pc:libdir=/opt/quic/openssl-master/lib64

@levitte
Copy link
Member

levitte commented Feb 13, 2024

(note that the libdir in openssl.pc doesn't really matter)

@gdiscord
Copy link

can I please get some clarification on this.
Is it possible to "force" openssl to install to an already existing lib directory?
I've already tried the libdir=lib aproach without any success; at the end of the automated build process the files always get installed to lib64 directory.
Doing the experiment with make build system.

@Zenju
Copy link

Zenju commented Apr 11, 2024

Oh, er, you're using --libdir incorrectly. It shouldn't be a full path, and leaving it empty is a really bad idea

FWIW: I had the same issue: the "lib" ending was missing for "libdir" in libcrypto.pc in OpenSSL 3.3 (no such issues with 3.2.1 and earlier). Turns out I was also using full path for --libdir (perhaps it should have been named --libdirname!). But unlike icing, your comment fixed the bug for me! Thanks!

@levitte
Copy link
Member

levitte commented Apr 11, 2024

Turns out I was also using full path for --libdir (perhaps it should have been named --libdirname!)

Perhaps it should be renamed, indeed. That will be a surprising build break for most, though. We've had --libdir for ages, and it's unfortunate that this name competes with autotools' use of this exact same option, which does cause quite a mess at times...

@icing
Copy link

icing commented Apr 25, 2024

Now Homebrew is delivering pkgconfig files with wrong libdirs.

@nhorman
Copy link
Contributor

nhorman commented Jun 5, 2024

is this perhaps a duplicate of #24298 ?

@nhorman
Copy link
Contributor

nhorman commented Jun 24, 2024

No response, marking as a duplicate of #24298

@nhorman nhorman closed this as completed Jun 24, 2024
@levitte
Copy link
Member

levitte commented Jun 24, 2024

is this perhaps a duplicate of #24298 ?

It wasn't, FYI. But considering we couldn't reproduce it, it was difficult to fix it.

icing added a commit to icing/curl that referenced this issue Jul 4, 2024
OpenSSL has a bug that messes the config `--libdir=path` to become
the wrong path in its pkgconfig files. If we just pass `--libdir=lib`
it should avoid this.

See also: openssl/openssl#23569
bagder pushed a commit to curl/curl that referenced this issue Jul 5, 2024
Also mention in HTTP3.md

OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.

Ref: #14099
See also: openssl/openssl#23569

Closes #14102
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triaged: bug The issue/pr is/fixes a bug
Projects
Status: Done
Development

No branches or pull requests

6 participants