Mac - problem using static libs installed by Homebrew

[DDeanBrown]

Trying to use the find_library('..., static: true) option and/or the "-Dprefer_static=true" options.

This line -

iconv_lib_dep = c_compiler.find_library('iconv', dirs: cross_lib_path, static: true)

gives "ERROR: C static library 'libiconv' not found", even though libiconv.a is sitting next to libiconv.dylib in the same folder, and it finds libiconv.dylib just fine without the "static: true". Another thing I tried got me this msg - "ERROR: '/usr/local/opt/libiconv/lib/libiconv.a' is not a static library." Are Homebrew installed .a files not static libraries? They look like it to me.

This line -

iconv_lib_dep = c_compiler.find_library('/usr/local/opt/libiconv/lib/libiconv', static: true)

gives no error, but then building gets Undefined symbols link errors for, e.g. "_iconv_close". Using hex editor, I see neither libiconv.dylib nor libiconv.a contain string "_iconv_close", but both contain "_libiconv_close".

MacOS 11.7.2
meson 0.64.1
Python 2.7.16
ninja 1.11.1

[eli-schwartz]

It might be a better idea to use dependency('iconv') and include the path to the homebrew iconv in CFLAGS and LDFLAGS.

Note that the issues with iconv_close vs libiconv_close sound like you have compiled with a different iconv.h header than the library you are linking to. GNU libiconv uses a header macro to redefine the symbol so that code written for iconv_* links to the namespaced libiconv_*. Meson's builtin iconv dependency checks for this consistency.

[DDeanBrown]

You are correct it would be better to use dependency() instead of find_library(), but I'm not in charge of the meson.build file I'm dealing with, which contains several calls to find_library(). I was hoping I could just use the '-Dprefer_static=true' option when calling 'meson setup...' to use static libraries but that does not seem to be possible. I will have to try to get changes made to that meson.build file. Using iconv in my example was a red herring, including the path to the homebrew iconv in CFLAGS and LDFLAGS takes care of the link error.

Whatever, I think there is a problem with find_library(), it treats static libs and dylibs differently. Using gtk3 as a better example -
find_library('gtk3')
gives no error, but how does it find libgtk-3.dylib? Is it using pkg-config? The reference manual doesn't say.
find_library('gtk3', static: true)
gives a meson error "ERROR: C static library 'gtk3' not found". Is it not using pkg-config?
find_library('gtk3', dirs: '/usr/local/lib/', static: true)
is the only way that works. My opinion is that find_library() should work the same way regardless of whether 'static: true' or '-Dprefer_static=true' is being used. Either they should both require meson to search with a "dirs: ..." or both use pkg-config. If this can't be done, then the reference manual should explain the difference.

[eli-schwartz]

but how does it find libgtk-3.dylib? Is it using pkg-config? The reference manual doesn't say.

The find_library method is a compiler object method, and like all other compiler methods, it uses the compiler itself to handle this. No pkg-config. I'm not sure why it would find a shared library but not a static one, however.

Another thing I tried got me this msg - "ERROR: '/usr/local/opt/libiconv/lib/libiconv.a' is not a static library." Are Homebrew installed .a files not static libraries? They look like it to me.

This error is probably a bit confusing, the error string you encountered is only generated deep inside target generation when you attempt to link_whole: to another target -- something that is only possible to do to an internally built static_library(). The error detection code just checks whether it is one of those, but the error string doesn't take into account that you might try to pass an external dependency instead of an internal build target.

[DDeanBrown]

My example had a typo, should be 'gtk-3'. Another weirdness - dependency('gtk-3') fails, it has to be dependency('gtk+-3.0'). Must be because of pkg-config. Which works correctly and I see '/usr/local/Cellar/gtk+3/3.24.34/lib/libgtk-3.dylib' in the linker args. And '-I/usr/local/Cellar/gtk+3/3.24.34/include/gtk-3.0' in the compiler args.

But c_compiler.find_library('gtk-3'), which succeeds in the meson config, just puts '-lgtk-3' in the linker args, and adds no '-I/...', so the build fails to find the header files, and the link would fail to find 'libgtk-3.dylib'.

c_compiler.find_library('gtk-3', '/usr/local/opt/gtk+3/lib') succeeds in the meson config, and puts '/usr/local/opt/gtk+3/lib/libgtk-3.dylib' in the linker args. And '-Ifreeciv-gtk3.22.p' in the compiler args. But at build time there's nothing in 'freeciv-gtk3.22.p' and the build fails to find the header files.

My theory now is that c_compiler.find_library('gtk-3'), without additional search dirs, is actually not finding anything, and should be reporting an error. It only adds '-lgtk-3' to the linker args, which is not enough for the build to succeed.

[eli-schwartz]

My example had a typo, should be 'gtk-3'. Another weirdness - dependency('gtk-3') fails, it has to be dependency('gtk+-3.0'). Must be because of pkg-config.

Right, because dependency() looks up the name of a namespaced project interface, which may be unrelated to actual -l flags. In Gtk 3's case, it certainly is. It also supports version checks.

But c_compiler.find_library('gtk-3'), which succeeds in the meson config, just puts '-lgtk-3' in the linker args, and adds no '-I/...',

... which makes sense, because find_library and has_header ask it to find a library / header file on the current set of default -L and -I directories, that's all.

My theory now is that c_compiler.find_library('gtk-3'), without additional search dirs, is actually not finding anything, and should be reporting an error.

When you don't specify a preference for static/shared, it's possible for Meson to detect that compiling a test file and linking it with -l{name} succeeds, and Meson may end up using that. When you specify a linkage model preference, Meson has to resolve it to an exact filename (or -Wl,--push-state,-Bdynamic / -Bstatic -lfoo -Wl,--pop-state).

But it should definitely be finding something -- check out the file builddir/meson-logs/meson-log.txt (and maybe post it here for analysis), this file contains information about the inner state of build checks.

And '-Ifreeciv-gtk3.22.p' in the compiler args. But at build time there's nothing in 'freeciv-gtk3.22.p' and the build fails to find the header files.

That include path is irrelevant. It exists for the project's own use, to create generated headers and use them. Not all projects do.

[DDeanBrown]

But c_compiler.find_library('gtk-3'), which succeeds in the meson config, just puts '-lgtk-3' in the linker args, and adds no '-I/...',

... which makes sense, because find_library and has_header ask it to find a library / header file on the current set of default -L and -I directories, that's all.

What I have finally figured out by looking at meson-log.txt is that '/usr/local/lib' is a search path for libraries. Here's what I see -

Detecting Apple linker via: ...
...
Library search paths:
/usr/local/lib
...

This is apparently not documented anywhere, including "man clang". The meson manual says "By default the library is searched for in the system library directory (e.g. /usr/lib). Specifying more directories here, causes Meson to search in those directories as well as the system directories.", which is misleading/incorrect. I suggest improving that part of the manual to say what actually happens on the Mac.

[eli-schwartz]

which is misleading/incorrect. I suggest improving that part of the manual to say what actually happens on the Mac.

What is misleading/incorrect? Whether clang documents it or not, if clang itself will find libraries there, so will Meson.

[eli-schwartz]

Note also the big note at the top of the page:

Note that if you have a single prefix with all your dependencies, you might find it easier to append to the environment variables C_INCLUDE_PATH with GCC/Clang and INCLUDE with MSVC to expand the default include path, and LIBRARY_PATH with GCC/Clang and LIB with MSVC to expand the default library search path.

This is a pretty big indicator that if variables which the compiler itself respects will change the search paths, then in general, whatever the compiler claims to search is also what Meson will search.

[DDeanBrown]

What's incorrect is where it says "e.g. /usr/lib" when it's actually "/usr/local/lib" on the Mac. Where it's misleading is where it says "the system directories" without saying what those are or how to find out what they are.

On my Mac the environment vars are CPPFLAGS and LDFLAGS. When I enter "echo $CPPFLAGS" or "echo $LDFLAGS" on the terminal command line I get nothing.

[eli-schwartz]

What's incorrect is where it says "e.g. /usr/lib" when it's actually "/usr/local/lib" on the Mac. Where it's misleading is where it says "the system directories" without saying what those are or how to find out what they are.

"e.g." -- "for example". If it is "actually something else on the Mac" then, well, examples are by definition neither incorrect (they never promised to be correct) nor correct (they will fail to match on other systems).

"The system directories" is not misleading. It may fail to be as helpful as it could be, but failing to state what it's talking about doesn't trick one into selecting a wrong choice or believing false information, or... whatever.

I would be happy to explain what's going on here in a support ticket, and I would be happy to help improve the documentation for clarity, but I don't really think that it's fair to say the current documentation is telling falsehoods.

On my Mac the environment vars are CPPFLAGS and LDFLAGS. When I enter "echo $CPPFLAGS" or "echo $LDFLAGS" on the terminal command line I get nothing.

Those are optional user-defined variables which don't have to exist, but if they do then Makefiles, autotools, cmake, meson, and most other build systems read them.

(But compilers do not.)

[eli-schwartz]

Anyway, it seems the original problem was being unable to find libraries without specifying dirs: '/usr/local/lib' only now it turns out that that directory works fine by default because it's the default search path even if the documentation doesn't say so?

[DDeanBrown]

Sorry to be so annoying, I do appreciate your helpful answers. The short answer to your last question is - yes, sorta. Have now learned that "man ld" gives the info missing from "man clang" - it says

Search paths
ld maintains a list of directories to search for a library or framework to use. The default
library search path is /usr/lib then /usr/local/lib.

I saw -print-search-dirs in a meson-log.txt, and "man clang" says

-print-search-dirs
Print the paths used for finding libraries and programs.

But when I try that it does not print /usr/lib nor /usr/local/lib - maybe I should file a bug report with Apple about that?

Anyway - my gtk-3 example is also bad because it turns out homebrew does not supply a static lib for gtk-3. So, trying again with this meson.build file -

project('test', ['c'], meson_version: '>= 0.57.0')

c_compiler = meson.get_compiler('c')

gtk322_dep = c_compiler.find_library('sdl2')
gtk322_dep = c_compiler.find_library('sdl2', dirs: '/usr/local/lib', static: true)
gtk322_dep = c_compiler.find_library('sdl2', static: true)

I see this in the meson-log.txt

Detecting Apple linker via: cc -Wl,-v
linker stdout:

linker stderr:
@(#)PROGRAM:ld PROJECT:ld64-650.9
BUILD 00:19:30 Mar 17 2021
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib
Framework search paths:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/

cc -Wl,-v just tells the linker to print version, but it actually prints that list of "Library search paths:", which, again, is not correct - it's missing /usr/lib. Maybe I should file a bug report with Apple about that?

So now I'm thinking meson is doing the best it can given the limitations of Apple's stuff. I now know the linker could find "/usr/local/lib/libSDL2.a" with no -L path help, but how could meson's 'c_compiler.find_library('sdl2', static: true)' know that, unless it runs 'cc -Wl,-v' and stashes the results so it can search those paths. And even then it would miss '/usr/lib'.

I think you're right that the manual is not misleading, just not as helpful as it could be. What would be nice is to add an enhancement to the compiler methods - compiler.print_search_dirs(). But implementing it correctly on the Mac would require knowing what 'man ld' says and not just relying on calls to cc. If that were done, that could also be used internally by meson so 'c_compiler.find_library('sdl2', static: true)' would work.

Another suggestion - add something in the manual compiler.find_library() static section to say

A value of true may cause meson to fail to find a library that exists, in this case you need to add the keyword argument "dirs:" to specify the path to it.