[WIP] Attempt to add /usr/local search paths

[anordal]

While this didn't actually work, please see:
https://bugzilla.opensuse.org/show_bug.cgi?id=1175039

[kaniini]

In Alpine, we use autoconf build system still because meson isn't in the bootstrap.

To set the paths we use --with-pkg-config-dir: https://git.alpinelinux.org/aports/tree/main/pkgconf/APKBUILD#n24

The meson build system should be adjusted to have the same configuration option.

[kaniini]

Anyway, opensuse uses autoconf and --with-pkg-config-dir correctly:

%global pkgconf_libdirs %{_libdir}/pkgconfig:%{_datadir}/pkgconfig
[...]
%build
autoreconf -fiv
%configure --disable-static \
           --with-pkg-config-dir=%{pkgconf_libdirs} \
           --with-system-includedir=%{_includedir} \
           --with-system-libdir=%{_libdir}

While it is true that the Meson build system does not support configuring the search path, they aren't using Meson, so it's not a problem.

The redhat customer complaining about pkgconf claims that PKG_CONFIG_PATH is not working for him.

Most likely it is actually a problem with his .pc file, or some RHEL-specific packaging issue, but we cannot be certain because he is using a portal that only RHEL customers may participate on.

Can you elaborate on what problem you are actually trying to solve, since it appears SuSE does not include /usr/local paths in the default search path on purpose?

[kaniini]

I would have said the latter part on opensuse bugzilla but I cannot for the life of me figure out how to create an account there that actually works. Either way, this is not an issue in pkgconf, so closing this PR.

[Conan-Kudo]

The RH/Fedora and openSUSE packaging are functionally identical since I wrote both. The /usr/local paths are not in the automatic search paths by policy to prevent accidental, difficult-to-uncover issues.

As far as I'm aware $PKG_CONFIG_PATH should still work to append to the paths.

[Conan-Kudo]

I've confirmed this does work as intended with a minimally valid test.pc file:

~> podman run --pull=always --rm -it registry.opensuse.org/opensuse/tumbleweed
Trying to pull registry.opensuse.org/opensuse/tumbleweed...
Getting image source signatures
Copying blob 18a29e57145b done
Copying config b500be0ac9 done
Writing manifest to image destination
Storing signatures
b2deda799032:/ # zypper install -C pkg-config
Retrieving repository 'openSUSE-Tumbleweed-Non-Oss' metadata .............[done]
Building repository 'openSUSE-Tumbleweed-Non-Oss' cache ..................[done]
Retrieving repository 'openSUSE-Tumbleweed-Oss' metadata .................[done]
Building repository 'openSUSE-Tumbleweed-Oss' cache ......................[done]
Retrieving repository 'openSUSE-Tumbleweed-Update' metadata ..............[done]
Building repository 'openSUSE-Tumbleweed-Update' cache ...................[done]
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 4 NEW packages are going to be installed:
  libpkgconf3 pkgconf pkgconf-m4 pkgconf-pkg-config

4 new packages to install.
Overall download size: 109.7 KiB. Already cached: 0 B. After the operation,
additional 158.3 KiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Retrieving package libpkgconf3-1.7.3-1.2.x86_64
                                           (1/4),  35.6 KiB ( 68.8 KiB unpacked)
Retrieving: libpkgconf3-1.7.3-1.2.x86_64.rpm .................[done (3.8 KiB/s)]
Retrieving package pkgconf-m4-1.7.3-1.2.noarch
                                           (2/4),  18.7 KiB ( 13.9 KiB unpacked)
Retrieving: pkgconf-m4-1.7.3-1.2.noarch.rpm ..............................[done]
Retrieving package pkgconf-1.7.3-1.2.x86_64
                                           (3/4),  41.3 KiB ( 75.2 KiB unpacked)
Retrieving: pkgconf-1.7.3-1.2.x86_64.rpm .................................[done]
Retrieving package pkgconf-pkg-config-1.7.3-1.2.x86_64
                                           (4/4),  14.2 KiB (  457   B unpacked)
Retrieving: pkgconf-pkg-config-1.7.3-1.2.x86_64.rpm .........[done (10.7 KiB/s)]

Checking for file conflicts: .............................................[done]
(1/4) Installing: libpkgconf3-1.7.3-1.2.x86_64 ...........................[done]
(2/4) Installing: pkgconf-m4-1.7.3-1.2.noarch ............................[done]
(3/4) Installing: pkgconf-1.7.3-1.2.x86_64 ...............................[done]
(4/4) Installing: pkgconf-pkg-config-1.7.3-1.2.x86_64 ....................[done]
b2deda799032:/ # cd ~
b2deda799032:~ # cat >> test.pc <<EOF
> Name: test
> Description: test library
> Version: 0
> Libs: -ltest
> Cflags: -I/usr/include
> EOF

b2deda799032:~ # PKG_CONFIG_PATH=/root pkg-config --libs test
-ltest

[anordal]

The /usr/local paths are not in the automatic search paths by policy to prevent accidental, difficult-to-uncover issues.

That's extremely sad. I don't know what good reason you have, but you're breaking every build system (case in point: Meson) on purpose. Nobody expects to set PKG_CONFIG_PATH. This used to work.

$PKG_CONFIG_PATH should still work to append to the paths.

Indeed, PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig makes it work for me, but I seriously didn't have the fantasy to try that.

[kaniini]

For what its worth, you can install a site config file to change this behavior. Read the pkgconf-personality man page. You will want to name the personality file default.personality.

I suspect Fedora/RHEL/SUSE have excluded /usr/local to ensure that only their libraries are pulled in on their builds.

It may be helpful to streamline the site configuration somehow to make it easier for your specific scenario. I'm open to ideas.

[anordal]

The site configuration might be a solution for the workplace, but as a library author (https://github.com/anordal/narg), I'm still devastated – what's the point of shipping a pkg-config file anymore? Now that Meson supports CMake packages, it makes more sense to ship that.

[Conan-Kudo]

I suspect Fedora/RHEL/SUSE have excluded /usr/local to ensure that only their libraries are pulled in on their builds.

This is pretty much the reason. Given that pkgconf is configurable and allows for overrides via variables and config files, it's not really a big deal that it's not searched by default.

For what it's worth, most distributions have never enabled search of /usr/local paths with legacy pkgconfig, so this behavior already existed across the board.

[anordal]

Isn't that a bit backwards? Distro-packaging is normally the exception. E.g. all software is supposed to install to /usr/local by default, while distro packagers are entitled to set the prefix to /usr. If that's a principle to follow, it would make sense if distro packagers were also the entitled ones to override pkg-config.

I'm surprised to hear that most distros never searched in /usr/local, but in that case, consider this a feature request ;-)
Letting it "just work" for end users (those who are just trying to build the damn thing) is attractive, even to someone as indirectly responsible as a library author.

[kaniini]

Personally I have to agree with @anordal. The way that we handle this in alpine is to just override the search path in the build environment. But either way, this is probably a conversation to have downstream -- the pkgconf project can't change how downstream distributions are shipping the software.

(Oh, and if someone wants to fix the meson --with-pkg-config-path feature, that would be welcome here.)

[orbea]

For what it's worth, most distributions have never enabled search of /usr/local paths with legacy pkgconfig, so this behavior already existed across the board.

I just want to point out this is not true in Slackware where both pkg-config and pkgconf search /usr/local by default.

[kaniini]

I feel like perhaps writing a document outlining best practices for distributions shipping pkg-config implementations might be useful.

Alpine and Debian ship /usr/local paths in default PKG_CONFIG_LIBDIR, then in package build environment explicitly set PKG_CONFIG_LIBDIR to /usr paths only. I suspect this is what most distributions should be doing. That allows for /usr/local to work as expected by end users, while ensuring the build environment is clean of /usr/local influence.

[Conan-Kudo]

On the flip side, the RPM ecosystem tends to lean towards synchronizing build and user environment settings as much as possible, to eliminate "surprise" inconsistencies.

It's also still unfortunately common for people to build RPMs on their regular environments by just running rpmbuild. Not having /usr/local paths by default makes it so those simply don't leak in for those circumstances. And folks in the RPM ecosystem platforms do tend to make RPMs that mix prefixes (largely due to the larger third-party ISV ecosystems around Red Hat and SUSE distributions). In both commercial platforms, there is a very strong encouragement to ship software through the RPM build process, though this has been de-emphasized in recent years.

However, I am tempted to consider the idea of making the pkg-config wrapper behave differently depending on the environment. My concern is reliably switching behavior.

[kaniini]

Well, in Alpine, we can change the environment variables that are exposed when you run abuild by using /etc/abuild.conf. It's similar with Debian's debuild.

If rpmbuild has similar functionality, then it's just a matter of doing export PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig in some config file somewhere.

[orbea]

In alpine or in any of the RPM distros do other tools like ld(1) protect against using /usr/local? For example in Slackware I see this as well.

$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
SEARCH_DIR("/usr/x86_64-slackware-linux/lib64")
SEARCH_DIR("/usr/lib64")
SEARCH_DIR("/usr/local/lib64")
SEARCH_DIR("/lib64")
SEARCH_DIR("/usr/x86_64-slackware-linux/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib"

I'm not sure how useful it would be to configure these protections for just pkgconf and not the rest of the system?

[Conan-Kudo]

We used to have them, but they've been chipped away over the years as people have wanted relocatable RPMs and RPMs built on alternative prefixes to work (e.g. Flatpak runtimes and apps composed from RPM builds...).

[Conan-Kudo]

I've adjusted the behavior for Fedora 33+: https://src.fedoraproject.org/rpms/pkgconf/c/05c8560a2eadac70cdc0e66554b5a4a6ac298f01

As for openSUSE Tumbleweed, I've made an SR to change this too: https://build.opensuse.org/request/show/825209

[kaniini]

Alpine's LD does search /usr/local paths by default. Perhaps we should fix that. Thanks for the observation @orbea.