Meson appends incorrect linker flags to compiler invocation

[jefftime]

System Information
$ meson --version: 0.53.1
$ ninja --version: 1.8.2
$ uname -srmo: Linux 4.15.0-88-generic x86_64 GNU/Linux

Issue
When cross compiling, meson appends incorrect linker flags to the linking stage of the build

Reproduction

main.c:

int foo(void) { return 10; }

meson.build:

project(
  'example',
  'c',
  default_options: [
    'b_lundef=false',
    'b_asneeded=false',
    'b_pch=false'
  ]
)

executable('example', 'main.c')

wasm.meson:

[binaries]
c = 'clang-9'
ld = 'wasm-ld'

[properties]
c_args = [ '--target=wasm32' ]
c_link_args = [ '--target=wasm32', '--no-standard-libraries', '-Wl,--no-entry', '-Wl,--export-all', '-Wl,--allow-undefined' ]

[host_machine]
system = 'unknown'
cpu_family = 'wasm32'
cpu = 'wasm32'
endian = 'little'

With the above files, running $ meson --cross-file wasm.meson bin && ninja -C bin returns with

The Meson build system
Version: 0.53.1
Source dir: /home/jefftime/programming/meson_issue
Build dir: /home/jefftime/programming/meson_issue/bin
Build type: cross build
Project name: example
Project version: undefined
C compiler for the build machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
C linker for the build machine: cc GNU ld.bfd 2.30
C compiler for the host machine: clang-9 (clang 9.0.0-2 "clang version 9.0.0-2~ubuntu18.04.2 (tags/RELEASE_900/final)")
C linker for the host machine: clang-9 lld 9.0.0
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: wasm32
Host machine cpu: wasm32
Target machine cpu family: wasm32
Target machine cpu: wasm32
Build targets in project: 1

Found ninja-1.8.2 at /usr/bin/ninja
WARNING: Cross file does not specify strip binary, result will not be stripped.
WARNING: Cross file does not specify strip binary, result will not be stripped.
ninja: Entering directory `bin'
[2/2] Linking target example.
FAILED: example 
clang-9  -o example 'example@exe/main.c.o' -Wl,--allow-shlib-undefined --target=wasm32 --no-standard-libraries -Wl,--no-entry -Wl,--export-all -Wl,--allow-undefined
wasm-ld: error: unknown argument: --allow-shlib-undefined
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Meson says it's configured to use clang-9 lld for the host linker, but on my system, clang-9 actually uses wasm-ld under the hood. As you can see in wasm.meson, setting ld = 'wasm-ld' doesn't seem to help the issue.

Meson proceeds to add the -Wl,--allow-shlib-undefined flag to the linker flags, but wasm-ld doesn't recognize this option. The proper flag should be -Wl,--allow-undefined or -Wl,--allow-undefined-file=<some_file>.

I would like to prohibit meson from adding flags on its own, but I couldn't seem to find a way to do this. I could disable the -Wl,--as-needed flag from being added, but I didn't have any luck on removing -Wl,--allow-shlib-undefined

[dcbaker]

The reason this is happening is that meson doesn't take clang's --target option into account, so it tests clang as if it was for your host system (or whatever clang's default target is). We really should fix this, but internally it's a bit of a chicken and egg problem (we need to know about the compiler to figure out what args it supports, but currently the linker is part of the compiler). As a work around you can create a symlink from clang to clang-<target> and use that as your compiler, and meson may probably do the right thing. Although, I don't know if it will do right thing with lld or not?

[dcbaker]

The more I think about this, this may actually work with meson master already. The only thing I don't know about is whether the --allow-shlib-undefined will work correctly.

[saurik]

I seem to be running into this with lld, which also doesn't support --allow-shlib-undefined :/.

[saurik]

My hack for right now is to not pass -fuse-ld= through meson as it happens to work well enough to complete the build (there's no guarantee that meson's incorrect autodetection would work: it just so happens to for the configurations I'm building) and I don't actually really want meson linking anything anyway... but is there really no way that you can just run the linker command I provided and see what linker you got? I really don't want meson attempting to parse my commands or separate binaries from arguments in its attempt to autodetect things: the detection should be designed to work correctly even if the linker is something like "/usr/bin/env distcc crazy -Wl,-oh-no" where "crazy" is a shell script on a remote machine that happens to run clang with -fuse-ld=lld. Can I at least have a way to override the detection (fwiw, maybe this already exists and I don't know?)... like, this isn't even the first time I've run into a serious issue with this (I have to have the entire 0.52 release series of meson blacklisted for effectively not supporting cross compile linking at all due to #6057). :(

[saurik]

Except now I need to use lld for my next target, so now I'm just stuck. I can't allow the default linker. If I try to override the default linker I get weird arguments being passed to it that aren't OK (like -no_weak_imports apparently... I'm using lld to cross compile from macOS to MIPS... I can't pass -no_weak_imports, which is an Apple ld64 flag of all things).

[saurik]

I'm now running into this same problem "in reverse", in that I've been required to use a build of clang that has been forcibly configured to default to lld on all platforms... even though lld doesn't work on macOS (and I'm compiling for macOS on macOS). I thereby need to pass -fuse-ld=/usr/bin/ld in order to force it to compile correctly with ld64. Of course, meson is now insisting on passing lld-specific command line arguments (such as --start-group), due to the totally-incorrect way it attempts to detect the linker :(. Is this likely to ever get fixed?

[saurik]

I just ran across #6650, wherein it is mentioned you can pass an array as the compiler... this is awesome, and should be documented as the primary way to configure the compiler. This does seem to fix the detection of the linker for flags that can be passed as part of the compiler, so I bet this will actually work correctly for people who merely need to get a default linker selected from --target (such as @jefftime)!! c = ['clang-9', '--target=wasm32']

However, this isn't so easy for me, as while passing -fuse-ld= as part of the compiler does cause meson to correctly detect the linker, if you pass -fuse-ld= to a command that doesn't link it causes an unused-command-line-argument warning, which meson is explicitly turning on with -Werror= :/. HOWEVER, it turns out clang now supports "configuration files" which can take arguments--notably -fuse-ld=--that won't be considered an unused argument!

llvm-mirror/clang@67245e0

I've thereby managed to make this work, though in a way that is frankly kind of obtuse, by dumping an additional file on disk that includes -fuse-ld=/usr/bin/ld and then passing that with --config to the compiler as part of the c/cpp/objc variables. :D

[Cartman0]

@dcbaker @saurik , this issue live? current meson version requires project() in meson.build file. how to write in order to cross-compile above example ?

[saurik]

@Cartman0 I am merely cross-compiling another project--glib--and thereby have no knowledge of or control over meson.build: I can only edit the --cross-file that is being passed to meson, and I'm doing exactly what I described to work around this seemingly-fundamental-at-this-point flaw in meson: I separate out certain flags--such as -fuse-ld=--into a different list which I pass using --config as part of the compiler instead of the linker. FWIW, my strong recommendation is that meson be avoided, as cross-compilation is simply too important for issues like this to just be sitting around un-addressed for years, and I find it extremely frustrating that I'm forced to use it for glib :/... if you are in a position to be editing meson.build, maybe you are in a position to use autoconf, which gets a bad reputation for being "old" (as if that's a bad thing) and yet has worked correctly for decades and continues to work correctly today.

[Cartman0]

@saurik sorry, i missread wasm.meson and meson.build .

i also have similar trouble when crosscompile with emscripten.

[saurik]

@Cartman0 FWIW, in case it is helpful, these files are where I implement these behaviors for my build system.

https://github.com/OrchidTechnologies/orchid/blob/1f517e3b5cd48b050b3fa5342377a5078a58b4b8/env/meson.sh

https://github.com/OrchidTechnologies/orchid/blob/1f517e3b5cd48b050b3fa5342377a5078a58b4b8/env/output.mk

[vonj]

Oh dear, that was quite the workaround. :-/

Here I was convinced there must be some incantation to convince meson to do the right thing, but when even the mighty saurik must resort to such workarounds, it's a bad sign to me.

Like saurik, I didn't even want meson to link in the first place, just compile a bunch of files. (We use a weird intermediate "linking" step to a regular .o file, so we don't actually have neither .so nor .a )

[eli-schwartz]

I'm not sure I understand this ticket overall, but can't you just use CC=clang C_LD=lld meson setup builddir and meson will detect that the linker is an lld and not apple ld64 or whatever?

[vonj]

Not sure. I just tested, and it doesn't seem to do any difference in my setup. (I may have done something wrong.)
Besides, I thought a design goal was to move away from depending on environment variables.

[eli-schwartz]

Sure, the same thing is available in the machine file as c = ['clang'] and c_ld = ['lld'], no variables required.

[vonj]

Ah, thanks!

Two questions if you have the time:

1. C++ linker is cxx_ld? cpp_ld? I can probably find this options name myself if I dig but I couldn't find it in a cursory search.
2. Do you know if one can just build a bunch of files to .o and then not link to a library or or an EXE?

[eli-schwartz]

Due to legacy reasons, Meson calls C++ "cpp" (as in the C preprocessor) instead of "cxx".

This manifests in:

• the project() languages list, which accepts e.g. 'c', 'cpp'
• build target kwargs accept cpp_args: ...
• machine files want you to set binaries for cpp = and linker ids for cpp_ld =

[eli-schwartz]

Do you know if one can just build a bunch of files to .o and then not link to a library or or an EXE?

I think the best you can do is static_library() which just ars them up instead of involving the linker. Way back in the day, no one bothered to add the concept of an "object library which produces a list of .o files" because it was assumed that static libraries was "good enough" (it is very nearly mostly the same thing).

[vonj]

Thank you so much! static_library() it is, then! If I get everything to work, I will report back here for posterity.

[vonj]

It worked just fine.

I set a variable x to what static_library() returned.

To extract the obj files, I used:

x.extract_objects(x_source_list)
This output I used in a custom target

custom_target(
    'dummy_name', # Mandatory in Meson 0.55
    input            : x.extract_objects(x_source),
    output          : 'x.tmp.o',
    command     : customlink_command,
    install           : false,
    build_by_default : true
)

customlink_command is script file which does the custom linking I needed

customlink_command = [
        find_program('customlink.cmd'),
        'C:/Path/to/customlinker.exe',
        '-m', 'elf_i386', '-X', '-r', '--eh-frame-hdr',
        '@INPUT@',
        '-o', '@OUTPUT@'
]

[pedronavf]

I agree with @saurik . It's unfortunate that more and more projects are switching to meson, which is not capable of handling cross-compilation in many cases and makes maintainers life very very difficult. At the very least, the ability to override the detected linker type should exist.

I'm using clang with -fuse=lld but using ld64.lld because I'm cross-compiling for mac, meson insists on passing "-soname" and some other invalid flags because the linker is being detected as llvm and not as apple.

[eli-schwartz]

At the very least, the ability to override the detected linker type should exist.

I have mentioned this already, but you can absolutely override the detected linker. You always could, and you always will.

[eli-schwartz]

On another note:

I'm not sure I understand this ticket overall, but can't you just use CC=clang C_LD=lld meson setup builddir and meson will detect that the linker is an lld and not apple ld64 or whatever?

Can y'all try out #10654 and tell me if that makes things easier for you? With that PR, Meson will recognize -fuse-ld=... in $LDFLAGS, and it will have the same effect as setting c_ld = '...'

I worked on this after a report from a Gentoo user that using -fuse-ld led to crashes, e.g. apparently on systems where the default linker is bfd, but bfd doesn't work with lto because of a missing plugin, you get errors and "use -v to see invocation", and then that triggers Apple ld64 of all things on a Linux system because that bizarre error message is "the" way to detect their liner due to $REASONS.

[pedronavf]

At the very least, the ability to override the detected linker type should exist.

I have mentioned this already, but you can absolutely override the detected linker. You always could, and you always will.

I want to override the detected type. My linker is lld, but because I'm targeting x86_64-apple-darwin21, lld's MachO frontend is being used. In that ld64 emulation mode only ld64 flags are supported. Because meson does a -Wl,--version to detect the linker type and gets back the LLVM string (and not what it expects for what Apple's LD should return) it sets the Linker to be LLVMDynamicLinker and not AppleDynamicLinker

As a result, flags like -soname and --as-needed get added to the link. I have used b_asneeded = false and b_lundef=false in my machine file but not all flags are removed and the linking fails.

[eli-schwartz]

Okay. This is interesting and useful information.

What's the output of lld when it does this? Is there some way to detect this when running cc -Wl,--version $LDFLAGS?

[pedronavf]

$ lld --version
lld is a generic driver.
Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead

$ ld.lld --version
Ubuntu LLD 14.0.0 (compatible with GNU linkers)

$ ld64.lld --version
Ubuntu LLD 14.0.0

The output of the --version linker parameter is coming from LLVM, as LLVM is adding its own command line options to all linker frontends.

If the linker is detected as LLVM, passing -Wl,--help will display the actual driver being used, which could be used to tell which flavor LLVM is emulating:

$ /usr/local/x86_64-apple-darwin21-22.2/clang/bin/clang -target x86_64-apple-darwin21 -arch x86_64 -fuse-ld=lld -Wl,--help

OVERVIEW: LLVM Linker

USAGE: /usr/local/x86_64-apple-darwin21-22.2/clang/bin/ld64.lld [options] file...

In the help screen you'll see somewhere in the string after USAGE:

• ld64.lld
• ld.lld
• wasm-ld
• llvm-link

And you could switch from LLVMDynamicLinker (which is what ld.ldd is) to AppleDynamicLinker or the ones for wasm or windows.

In any case, being able to override the detected type in the machine with something along the lines of c_ld_type = 'Apple' will make it easier to circumvent these kinds of issues.

[pedronavf]

Having the ability to have a custom linker (CustomDynamicLinker?) specified in a machine file, where you can set all properties of the DynamicLinkerClass, like allow_undefined_args, ltd_args, pie_args, asneeded_args, etc would be useful.

[eli-schwartz]

In the help screen you'll see somewhere in the string after USAGE:

Oh hmm, that should work:

$ gcc -Wl,--help -fuse-ld=lld
[...]
USAGE: /usr/bin/ld.lld [options] file...
[...]

$ clang -target x86_64-apple-darwin21 -arch x86_64 -fuse-ld=lld -Wl,--help
[...]
USAGE: /usr/bin/ld64.lld [options] file...
[...]

I'll try to take a look at this sometime next week.

Having the ability to have a custom linker (CustomDynamicLinker?) specified in a machine file, where you can set all properties of the DynamicLinkerClass, like allow_undefined_args, ltd_args, pie_args, asneeded_args, etc would be useful.

The idea has been previously pondered to allow defining custom compilers/linkers via ini files such that they don't have to be coded into mesonbuild/compilers/*.py; what exactly that would entail hasn't been proposed with enough specificity to implement as of yet. It's certainly an intriguing thought.

[vonj]

A custom linker option would be very nice. Otherwise meson must know how all linkers work and how they take their arguments. Maybe the workaround with custom_target I showed above could be inspiration for such a feature.

Further on in our codebase (I am converting a legacy build system) I happened upon a place where we compile with LLVM but link with some crusty old linker which takes special linking commands on stdin. I created a wrapper for that linker and called it with custom_target

I don't know what to suggest exactly because I don't know yet well enough how to use meson, let alone how to change its internals. But I'm wary of programs which try to detect everything without the option for a manually specified fallback mechanism. Some things can't be detected because the world is messy.

[pedronavf]

I'll try to take a look at this sometime next week.

Thanks!

The idea has been previously pondered to allow defining custom compilers/linkers via ini files such that they don't have to be coded into mesonbuild/compilers/*.py; what exactly that would entail hasn't been proposed with enough specificity to implement as of yet. It's certainly an intriguing thought.

While that's been decided, an option to override the detected type (or to skip detection) might be useful. There linkers meson implement work fine, but if detection is not correct there is no easy way to fix that. Being apple to just use that ld is Apple (or any of the other supported types) from the machine file would be very useful.

[vonj] A custom linker option would be very nice. Otherwise meson must know how all linkers work and how they take their arguments.

I agree. Even a "do nothing" or "system" linker type could help. If that's set then do nothing: there's no need to add flags based on options and what's set in LD, LDFLAGS, etc should be used instead.

From what I have experienced, and I have seen others mention here, a source of issues with meson comes from integrating meson-buillt libraries into a foreign build system that, usually, uses autotools or autotools environment variables, and expects the tools to just blindly used the specified commands for compiling or linking.

Meson aims to simplify all that, and its heart is in the right place, but unless there is some fallback when detection fails or a tool is used in a way that the meson devs haven't added support for, the user is completely blocked. When that happens, extremely hacky measures (like writing a shell script called ld that calls the linker unless it detects "-Wl,-version" being passed, in which case it returns the string that Meson expects, or a pre-build step that searches and removes incorrect linker flags from the generated ninja rules files), need to happen which usually take a long time to reseach and are only available for people that can look at meson's source and get to the bottom of what's happening.

[gbaraldi]

Any update/workaround for this? I hit this while crosscompiling some c++
I had

ld = '/opt/bin/x86_64-apple-darwin14-libgfortran5-cxx11/ld64.lld'
cpp_ld = '/opt/bin/x86_64-apple-darwin14-libgfortran5-cxx11/ld64.lld'
c_ld = '/opt/bin/x86_64-apple-darwin14-libgfortran5-cxx11/ld64.lld'

But it detects

C++ linker for the host machine: /opt/bin/x86_64-apple-darwin14-libgfortran5-cxx11/x86_64-apple-darwin14-clang++ ld.lld 16.0.6

And finally passes

-fuse-ld=/opt/bin/x86_64-apple-darwin14-libgfortran5-cxx11/ld64.lld

To clang while linking which causes the issues that other people have already complained about

[SamYuan1990]

I am not sure if I am right... but

meson/mesonbuild/linkers/detect.py

Line 179 in 4a96ad0

elif e.endswith('(use -v to see invocation)\n') or 'macosx_version' in e or 'ld: unknown option' in e:

currently it seems the code logic is based on error catch, and if error message meet some specific value then go to macOS logic. I want to know if we have any comments if we change the logic basing on platform information get from code blocks below?

>>> import platform
>>> platform.platform()

[eli-schwartz]

That would be unreliable when operating on a cross compiler. A cross compiler could be targeting macOS without platform.platform() being macOS, and I believe there are a few alternative compiler toolchains that act like and identify as macOS ld64 and have this precise quirk.

[dcbaker]

I guess in theory we could use the machine info value for the machine we’re targeting.

It feels like there’s a lot of macOS bugs that aren’t getting fixed because no one has a Mac anymore (mine is a really old Intel Mac not getting updates anymore).