add build option pkg_config_static to make dependency() default to static

[ffontaine]

Allow the user to always call pkg-config with --static when choosing to build a project as fully static, to prevent missing library errors when only static dependencies are available.

Signed-off-by: Fabrice Fontaine fontaine.fabrice@gmail.com

[xclaesse]

I'm not sure to understand why this is needed, if you do dependency('glib-2.0', static : true) it should work, no?

[ffontaine]

This patch is needed to cross-compile glib2 in buildroot, the build failure is not related to a meson package using glib2 so there is no "dependency('glib-2.0')". The issue is related to the building of glib2 itself.

Without this patch, glib2 would have to be updated to add static: true or false to every dependency calls for example, glib2 retrieves the libmount dependency through:
dependency('mount', version : '>=2.23', required : get_option('libmount'))
as a result, meson only retrieve libmount.a from mount.pc and not libblkid.a and so linking of gio fails with the error that I already put in the commit message.

I also sent this patch to the buildroot's mailing list: https://patchwork.ozlabs.org/patch/1238548 so you can see how I use this new variable depending on the static or dynamic build configuration. We can also wait for their feedback.

[xclaesse]

Hmm, I wondering if we would need something more fine grained than this.You don't necessary want to static link everything, but only a few.

This is somewhat related to #4276

Just to be sure to understand your use-case, you have glib-2.0.pc and libmount.a but not libmount.so ?

[xclaesse]

In any case, I don't think this is limited to pkg-config, the real feature needed here is to turn all (or some) dependency() to have static:true regardless of the underlying mechanism used. In other words, in your patch you should not do if self.static or something but have self.static set to True in the first place.

[ffontaine]

You're right, my use case is that I only have libmount.a (and libblkid.a) and I want to build glib2 (including glib-2.0.pc). You're also right that this patch is somewhat basic as it only manages the buildroot use case. In buildroot, we only give three choices to the user for building his embedded linux system:

• Build libraries statically (i.e we ask all packages that support it to build statically i.e. only .a are built). It should be noted that some packages don't allow static building because they use dlopen or because they don't know how to retrieve correctly their static dependencies;
• Build libraries dynamically (i.e we ask all packages that support it to build dynamically i.e. only .so are built)
• Build static and dynamic libraries (i.e we ask all packages that support it to build statically and dynamically)

[eli-schwartz]

The commit message and pull request titles are confusing and don't message the reader "why do I want this?" -- as a result, reviewers are less likely to accept the PR because they don't know what it's supposed to do.

Please update both to not share information about the file you're modifying (which is rarely if ever important information in a git commit message); instead, use a commit message like e.g.

add build option `pkg_config_static` to make dependency() default to static

Allow the user to always call pkg-config with --static when choosing to build a project as fully static, to prevent missing library errors when only static dependencies are available.

[eli-schwartz]

Dumb question: if meson is already resolving detected libraries and passing them to the compiler command line as /path/to/libfoo.a, so it detects that only the .a is available, why doesn't it automatically re-run the pkg-config lookup command with --static? It's a bit broken to assume that the results of pkg-config --libs foo can be used without issues to link "libfoo.a".

Given meson has so far avoided having a way to do something like PKG_CONFIG="pkg-config --static" ./configure, it would make sense to pick up the load.

[xclaesse]

Dumb question: if meson is already resolving detected libraries and passing them to the compiler command line as /path/to/libfoo.a, so it detects that only the .a is available, why doesn't it automatically re-run the pkg-config lookup command with --static? It's a bit broken to assume that the results of pkg-config --libs foo can be used without issues to link "libfoo.a".

That was my first thought too, but there is a reason explained in this comment: https://github.com/mesonbuild/meson/blob/master/mesonbuild/dependencies/base.py#L806.

[xclaesse]

oh wait, maybe it falls into that case only if neither .so nor .a are found? If we are looking for a shared library and we found a static, I guess we could re-run with --static indeed, otherwise it's unlikely to work any way.

[nirbheek]

oh wait, maybe it falls into that case only if neither .so nor .a are found? If we are looking for a shared library and we found a static, I guess we could re-run with --static indeed, otherwise it's unlikely to work any way.

Hmm, yes, but it has to be done recursively. Consider the following:

• lib-foo (foo.pc) Requires: bar
• lib-bar (bar.pc) Requires: baz Requires.private: faz
• lib-baz (baz.pc) (no depends)
• lib-faz (faz.pc) (no depends)

We call pkg-config --libs foo, which gives us -lfoo -lbar -lbaz. We search, and find libfoo.so, libbar.a, libbaz.a. Now we know that we need to call whatever pkg-config file is associated with libbar.a with --static, not foo.pc.

Since pkg-config's dependency resolution is opaque to us, we have no idea which .pc file provides -lbar.

There are two ways around this:

1. Reimplement pkgconfig inside Meson (ew, but also hmmm)

2. Try to build the 'library': 'pcname' mapping ourselves using available pkg-config arguments. --print-requires and --print-requires-private combined with --libs on each requires/requires-private should allow us to recursively create that graph ourselves and map which pcfiles provide which libraries.

OR, we could just call the parent pcfile with --static and rely on -Wl,--as-needed, -Wl,-dead_strip_dylibs, etc to prevent overlinking. I suspect (2) will be cleaner, if a bit slow if there are lots of deps.

[xclaesse]

@nirbheek Hmm, that quickly become much more complicated. tbh I would assume that if one .so is missing but it has its .a, then they are all missing. Worst case it over-link but still works, no? I mean, you'll re-run with --static, get more -lfoo and for those that have .so files will just be extra useless link flags.

[yann-morin-1998]

(got dragged here fom #6697)

Hmm, yes, but it has to be done recursively. Consider the following:
* lib-foo (foo.pc) Requires: bar
* lib-bar (bar.pc) Requires: baz Requires.private: faz
* lib-baz (baz.pc) (no depends)
* lib-faz (faz.pc) (no depends)

We call pkg-config --libs foo, which gives us -lfoo -lbar -lbaz. We search, and find libfoo.so, libbar.a, libbaz.a. Now we know that we need to call whatever pkg-config file is associated with libbar.a with --static, not foo.pc.

Since pkg-config's dependency resolution is opaque to us, we have no idea which .pc file provides -lbar.

There are two ways around this:
2. Try to build the 'library': 'pcname' mapping ourselves using available pkg-config arguments. --print-requires and --print-requires-private combined with --libs on each requires/requires-private should allow us to recursively create that graph ourselves and map which pcfiles provide which libraries.

There is no need for us to manually recurse in the libraries list provided by pkg-config, as that is already the full list. From the manpage:

    --libs
        This option [...] prints the link flags. [...] and flags for dependencies are included in the output.

Combined with --static, that gives the exact full list of linker flags to use, including the exact list of libraries to use, and in the correct order (as long as the .pc files are corect, but that's another topic) to do a static link.

OR, we could just call the parent pcfile with --static and rely on -Wl,--as-needed, -Wl,-dead_strip_dylibs, etc to prevent overlinking. I suspect (2) will be cleaner, if a bit slow if there are lots of deps.

Well, I think overlinking is a red-hering. The .pc files are (suposed to be) authoritative about the package they describe: if a .pc file says "link with foo" (by way of Requires or Requires.private), there is no reason we should not abide by their wish.

[aduskett]

Yeah, we should be using pkg-config as the authority on this one. pkg-config --static --lib ${package} only returns static libraries needed and will never (afaik) return shared libraries.

[xclaesse]

Yeah, we should be using pkg-config as the authority on this one. pkg-config --static --lib ${package} only returns static libraries needed and will never (afaik) return shared libraries.

Sadly that would be too easy. It returns a list of -lfoo flags and it's up to you (and the compiler) to translate that to one of the [lib]foo.[so, a, dll] it can find.

[yann-morin-1998]

Yeah, we should be using pkg-config as the authority on this one. pkg-config --static --lib ${package} only returns static libraries needed and will never (afaik) return shared libraries.

Sadly that would be too easy. It returns a list of -lfoo flags and it's up to you (and the compiler) to translate that to one of the [lib]foo.[so, a, dll] it can find.

That is exactly the role of the linker (not the buildsystem like meson et al, or the compiler). The flags returned by pkg-config will also contain the necessary -L options to locate the libraries, too. There is no reason to try and resolve the -l flags earlier than that.

And of course, if one calls pkg-config with --static, it is expected that one will also call the linker with --static (or the equivalent when calling the linker via the compiler driver, that is -static or -static-pie et al.).

[xclaesse]

@yann-morin-1998 it's not that easy, you could want to static link some libs but not all.

[yann-morin-1998]

@yann-morin-1998 it's not that easy, you could want to static link some libs but not all.

There are two cases here (forgive me if I use wrong terms, I am far from fluent in meson):

1. the developper of a package writes a meson.build in which they explicitly request static linkign to a set of libraries, and leaves the rest to be auto-determined,
2. the integrator (i.e. an upper-layer buildsystem like Buildroot, Yocto, or a distro) request a complete static build (like the case in compilers/mixins/clike.py: prefer .a shlibext when --default-library=static #6697)

In the first case, when resolving the each library, we know (because that'w requested in meson.build) that it should be resolved statically or not. So when we call pkg-config for those library that are statically requested, we want to grab all its static dependencies and linker flags, and that's exactly what pkg-config -static will return us, as exemplified above. We should be wary to use a non-static dependency of a static library: this opens a can of worms with subtle breakage at runtime. For the other libraries (those that are auto-deteremined), we just need to call pkg-config in-bulk, which should return whatever is needed to link said libraries.

In the second case, we don't care what the user wrote in their meson.build: the buildsystem (or the human) above knows better and explicitly asks for a full static build. In this case, all calls to pkg-config should be made with -static, and that will work as expected.

Sorry if I sounded dense, that's definitely not the goal. I know static linking is in essence a can of worms already, but some systems can only work with static libs (e.g. some noMMU embedded ARM stuff for example, or some trimmed-down, minimalist, single-app container images...).

[xclaesse]

1. the developper of a package writes a meson.build in which they explicitly request static linkign to a set of libraries, and leaves the rest to be auto-determined,

That's the case where meson has to resolve itself static libraries. When user explicitly request a static dependency, pkg-config just says "-lfoo" and the linker will pick the shared library when you have both installed. To fix that, meson replace -lfoo with the path to libfoo.a instead. Note that other dependencies could be shared, so we can't tell the linker to just static link everything.

2. the integrator (i.e. an upper-layer buildsystem like Buildroot, Yocto, or a distro) request a complete static build (like the case in #6697)

That's a new use-case that has never been supported by Meson. This would require a new option, not limited to pkg-config, that would do:

• Make static kwarg of dependency() and find_library() default to true.
• Disable the code that replace -lfoo with path to corresponding libfoo.a
• Pass -static to the linker

[dcbaker]

That's a new use-case that has never been supported by Meson. This would require a new option, not limited to pkg-config, that would do

If we're going to do this we really should have a builtin-option for this instead of a random field in the properties section of the cross/native file.

[H5117]

If we are looking for a shared library and we found a static, I guess we could re-run with --static indeed, otherwise it's unlikely to work any way.

IMO the right solution is to allow pkg-config return the correct set of flags automatically.
Described here: https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/54