Windows, clang: LINK_ARGS escaping broken in 0.58

[PlushBeaver]

Describe the bug
If link arguments contain a backslash, it's not escaped in LINK_ARGS in ninja.build, resulting link failure.

To Reproduce

meson.build

project('example', 'C')
def = custom_target('def',
    command: ['xcopy', '@INPUT@', '@OUTPUT@'],
    input: 'lib.def', output: 'lib.def')
lib = shared_library('lib',
    sources: files('lib.c'),
    link_args: ['-Wl,/def:' + def.full_path()])

lib.c

void func(void) {}

lib.def

EXPORTS
    func

Configure:

PS C:\Users\Administrator\Desktop\backslash> meson build
The Meson build system
Version: 0.58.1
Source dir: C:\Users\Administrator\Desktop\backslash
Build dir: C:\Users\Administrator\Desktop\backslash\build
Build type: native build
Project name: example
Project version: undefined
C compiler for the host machine: clang (clang 10.0.0 "clang version 10.0.0 ")
C linker for the host machine: clang link 14.24.28316.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program xcopy found: YES (C:\Windows\system32\xcopy.EXE)
Build targets in project: 2

Build:

Found ninja-1.10.2 at "C:\Program Files\Meson\ninja.EXE"
PS C:\Users\Administrator\Desktop\backslash> ninja -C build
ninja: Entering directory `build'
[2/2] Linking target lib.dll
FAILED: lib.dll lib.pdb
"clang"  -Wl,/MACHINE:X64 -Wl,/OUT:lib.dll lib.dll.p/lib.c.obj "-Wl,/nologo" "-Wl,/release" "-Wl,/nologo" "-Wl,/DEBUG" "-Wl,/PDB:lib.pdb" "-Wl,/DLL" "-Wl,/IMPLIB:lib.lib" "-Wl,/def:C:\Users\Administrator\Desktop\backslash\build\lib.def" "-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32" "-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32"
LINK : fatal error LNK1104: cannot open file 'C:\Users\Administrator\Desktop\backslash\build\lib.def'
clang: error: linker command failed with exit code 1104 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Root cause:

build.ninja

build lib.dll | lib.pdb: c_LINKER lib.dll.p/lib.c.obj
 LINK_ARGS = "-Wl,/nologo" "-Wl,/release" "-Wl,/nologo" "-Wl,/DEBUG" "-Wl,/PDB:lib.pdb" "-Wl,/DLL" "-Wl,/IMPLIB:lib.lib" "-Wl,/def:C:\Users\Administrator\Desktop\backslash\build\lib.def" "-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32" "-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32"

Notice unescaped \, they are escaped (doubled) with older versions of meson. If you manually change them to double \ or to /, the build will pass.

Expected behavior
DLL builds OK, as with meson 0.57 and older.

System parameters

• Native build
• Windows 10
• LLVM 10.0.0
• Python 3.8.1
• Meson 0.58.1
• Ninja 1.10.2

[jon-turney]

lib = shared_library('lib',
sources: files('lib.c'),
link_args: ['-Wl,/def:' + def.full_path()])

I think this should be written using the vs_module_defs: kwarg

lib = shared_library('lib',
    sources: files('lib.c'),
    vs_module_defs: def)

which should then work for all linkers.

The regression here still needs investigation, though.

[PlushBeaver]

The issue is not limited to .def files, it was just an example or an option with \ inside. Anyway, thanks for the tip.

[jon-turney]

Trying to reproduce this, I'm not sure that the test case is good.

The custom_target() is not build_by_default, and just referencing the path of the output doesn't create a dependency on it, so the error that build/lib.def doesn't exist seems correct.

Adding build_by_default: true shows that xcopy doesn't know how to copy single files or something and I'm not sure how to fix that...

[PlushBeaver]

You're right, example is broken, sorry about that. I'm trying to craft a proper one, but I'm out of luck for now. Original project, DPDK, has quite an elaborate build config. This is an error I'm actually getting:

FAILED: lib/rte_eal-21.dll
"clang" @lib/rte_eal-21.dll.rsp
clang: error: no such file or directory: 'librte_kvargs.lib'

Inside the build.ninja the rule looks as follows:

build lib/rte_eal-21.dll: c_LINKER_RSP ...multiple object files...  | C$:/Users/Administrator/Desktop/dpdk/lib/eal/version.map lib/rte_eal_exports.def lib/rte_eal_mingw.map lib/rte_kvargs-21.dll.p/rte_kvargs-21.dll.symbols
 LINK_ARGS = "-Wl,/nologo" "-Wl,/release" "-Wl,/nologo" "-Wl,/OPT:REF" "-Wl,/DLL" "-Wl,/IMPLIB:lib\rte_eal.lib" "-lws2_32" "-lmincore" "-ladvapi32" "-lsetupapi" "-ldbghelp" "lib\rte_kvargs.lib" "-Wl,/def:C:\Users\Administrator\Desktop\dpdk\build\lib\rte_eal_exports.def" "-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32" "-loleaut32" "-luuid" "-lcomdlg32"

Notice lib\rte_kvargs.lib in ninja file vs librte_kvargs.lib in error message. In meson 0.57 LINK_ARGS looks as follows:

 LINK_ARGS = -Wl,/nologo -Wl,/release -Wl,/nologo -Wl,/OPT:REF -Wl,/DLL '-Wl,/IMPLIB:lib\\rte_eal.lib' -lws2_32 -lmincore -ladvapi32 -lsetupapi -ldbghelp 'lib\\rte_kvargs.lib' '-Wl,/def:C:\\Users\\Administrator\\Desktop\\dpdk\\build\\0.57\\lib\\rte_eal_exports.def' -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32

Backslashes are escaped, build succeeds.

[jon-turney]

Thanks.

bisection lands on 2be074b (#8507) as the first bad commit, so I guess we messed up for this toolchain there.

Cc: @dcbaker

[PlushBeaver]

Sorry, I'm still failing to reduce the real project to a minimal example. I noticed that meson >= 0.58 fails to consume a pkg-config file produced by meson 0.57.2 with a similar error, and also spaces are not escaped (i. e. 0.57.2 produces an unusable .pc file):

libdpdk.pc:

prefix=c:/
libdir=${prefix}/lib
includedir=${prefix}/include

Name: DPDK
Description: The Data Plane Development Kit (DPDK).
Note that CFLAGS might contain an -march flag higher than typical baseline.
This is required for a number of static inline functions in the public headers.
Version: 21.08.0-rc3
Requires: libdpdk-libs
Libs.private: -Wl,--whole-archive -L${libdir} -l:librte_common_iavf.a -l:librte_bus_auxiliary.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_common_mlx5.a -l:librte_mempool_ring.a -l:librte_net_i40e.a -l:librte_net_ice.a -l:librte_net_mlx5.a -l:librte_net_vmxnet3.a -l:librte_pdump.a -l:librte_latencystats.a -l:librte_gso.a -l:librte_gro.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -ldbghelp -lsetupapi -lws2_32 -lmincore C:/Program Files/Mellanox/MLNX_WinOF2_DevX_SDK/lib/mlx5devx.lib C:/Program Files/Mellanox/MLNX_WinOF2_DevX_SDK/lib/mlx5devx.lib
Cflags: -I${includedir}

meson.build of the app:

project('test', 'C')
dpdk = dependency('libdpdk',
        dirs: ['C:/Users/Administrator/Desktop/dpdk/install/clang/lib/pkgconfig'],
        method: 'pkg-config',
        static: true)
app = executable('app',
        sources: files('app.c'),
        include_directories: ['C:/Users/Administrator/Desktop/dpdk/lib/eal/windows/include'], # irrelevant to the bug
        dependencies: dpdk)

Build log clearly shows that path separators are missing (liblib) and the path with spaces got split:

FAILED: app.exe app.pdb
"clang" @app.exe.rsp
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_common_iavf.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_bus_auxiliary.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_bus_pci.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_bus_vdev.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_common_mlx5.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_mempool_ring.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_net_i40e.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_net_ice.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_net_mlx5.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_net_vmxnet3.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_pdump.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_latencystats.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_gso.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_gro.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_cfgfile.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_bitratestats.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_timer.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_hash.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_metrics.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_cmdline.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_pci.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_ethdev.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_meter.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_net.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_mbuf.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_mempool.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_rcu.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_ring.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_eal.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_telemetry.a'
clang: error: no such file or directory: 'C:/Users/Administrator/Desktop/dpdk/install/clang/liblibrte_kvargs.a'
clang: error: no such file or directory: 'C:/Program'
clang: error: no such file or directory: 'Files/Mellanox/MLNX_WinOF2_DevX_SDK/lib/mlx5devx.lib'
clang: error: no such file or directory: 'C:/Program'
ninja: build stopped: subcommand failed.

P.S. This is meson 0.59.999 (27b3195).

[tmonjalo]

Any news about this open bug?