-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PkgConfigDependency: Parse library paths in a separate step #4061
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -576,37 +576,67 @@ def _set_cargs(self): | |
self.compile_args = self._convert_mingw_paths(shlex.split(out)) | ||
|
||
def _search_libs(self, out, out_raw): | ||
link_args = [] | ||
raw_link_args = [] | ||
''' | ||
@out: PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 pkg-config --libs | ||
@out_raw: pkg-config --libs | ||
|
||
We always look for the file ourselves instead of depending on the | ||
compiler to find it with -lfoo or foo.lib (if possible) because: | ||
1. We want to be able to select static or shared | ||
2. We need the full path of the library to calculate RPATH values | ||
3. De-dup of libraries is easier when we have absolute paths | ||
|
||
Libraries that are provided by the toolchain or are not found by | ||
find_library() will be added with -L -l pairs. | ||
''' | ||
# Library paths should be safe to de-dup | ||
libpaths = OrderedSet() | ||
raw_libpaths = OrderedSet() | ||
# | ||
# First, figure out what library paths to use. Originally, we were | ||
# doing this as part of the loop, but due to differences in the order | ||
# of -L values between pkg-config and pkgconf, we need to do that as | ||
# a separate step. See: | ||
# https://github.com/mesonbuild/meson/issues/3951 | ||
# https://github.com/mesonbuild/meson/issues/4023 | ||
# | ||
# Separate system and prefix paths, and ensure that prefix paths are | ||
# always searched first. | ||
prefix_libpaths = OrderedSet() | ||
# We also store this raw_link_args on the object later | ||
raw_link_args = self._convert_mingw_paths(shlex.split(out_raw)) | ||
for arg in raw_link_args: | ||
if arg.startswith('-L') and not arg.startswith(('-L-l', '-L-L')): | ||
prefix_libpaths.add(arg[2:]) | ||
system_libpaths = OrderedSet() | ||
full_args = self._convert_mingw_paths(shlex.split(out)) | ||
for arg in full_args: | ||
if arg.startswith(('-L-l', '-L-L')): | ||
# These are D language arguments, not library paths | ||
continue | ||
if arg.startswith('-L') and arg[2:] not in prefix_libpaths: | ||
system_libpaths.add(arg[2:]) | ||
# Use this re-ordered path list for library resolution | ||
libpaths = list(prefix_libpaths) + list(system_libpaths) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this needlessly duplicates all paths in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, it does not duplicate any paths. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. right. I somehow overlooked the |
||
# Track -lfoo libraries to avoid duplicate work | ||
libs_found = OrderedSet() | ||
# Track not-found libraries to know whether to add library paths | ||
libs_notfound = [] | ||
libtype = 'static' if self.static else 'default' | ||
# We always look for the file ourselves instead of depending on the | ||
# compiler to find it with -lfoo or foo.lib (if possible) because: | ||
# 1. We want to be able to select static or shared | ||
# 2. We need the full path of the library to calculate RPATH values | ||
# | ||
# Libraries that are provided by the toolchain or are not found by | ||
# find_library() will be added with -L -l pairs. | ||
for lib in self._convert_mingw_paths(shlex.split(out)): | ||
# Generate link arguments for this library | ||
link_args = [] | ||
for lib in full_args: | ||
if lib.startswith(('-L-l', '-L-L')): | ||
# These are D language arguments, add them as-is | ||
pass | ||
elif lib.startswith('-L'): | ||
libpaths.add(lib[2:]) | ||
# We already handled library paths above | ||
continue | ||
elif lib.startswith('-l'): | ||
# Don't resolve the same -lfoo argument again | ||
if lib in libs_found: | ||
continue | ||
if self.clib_compiler: | ||
args = self.clib_compiler.find_library(lib[2:], self.env, | ||
list(reversed(libpaths)), libtype) | ||
libpaths, libtype) | ||
# If the project only uses a non-clib language such as D, Rust, | ||
# C#, Python, etc, all we can do is limp along by adding the | ||
# arguments as-is and then adding the libpaths at the end. | ||
|
@@ -650,16 +680,11 @@ def _search_libs(self, out, out_raw): | |
if lib in link_args: | ||
continue | ||
link_args.append(lib) | ||
# Also store the raw link arguments, and store raw_libpaths | ||
for lib in self._convert_mingw_paths(shlex.split(out_raw)): | ||
if lib.startswith('-L') and not lib.startswith(('-L-l', '-L-L')): | ||
raw_libpaths.add(lib[2:]) | ||
raw_link_args.append(lib) | ||
# Add all -Lbar args if we have -lfoo args in link_args | ||
if libs_notfound: | ||
# Order of -L flags doesn't matter with ld, but it might with other | ||
# linkers such as MSVC, so prepend them. | ||
link_args = ['-L' + lp for lp in raw_libpaths] + link_args | ||
link_args = ['-L' + lp for lp in prefix_libpaths] + link_args | ||
return link_args, raw_link_args | ||
|
||
def _set_libs(self): | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we rename
out
andout_raw
to something that gives a clue what they contain?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
out means "pkg-config output" and out_raw means "pkg-config output without PKG_CONFIG_ALLOW_SYSTEM_LIBS". I documented it in the docstring what each means.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's documented. But i don't think the names are very good. Maybe something like
out_with_system
andout_without_system
would be more clear and contrasting.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, sure, I can rename them, but I don't see the point because they're just used once (
shlex.split()
) and then never again ;)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, we use the term
self.raw_link_args
on the object itself, so there is precedent for this terminology.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is, but as instance variable i think it's even worse, because it does have even less context. 'raw' just mean, without something or some processing that is unspecified...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I was making the change when I realized that it'll touch a lot of other code unnecessarily, since
out
andout_raw
is used elsewhere too. Let's not churn the CI unnecessarily, since this PR has to go into the stable branch.