Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NetBSD error linking against C library with link dependency #96328

Closed
tmfink opened this issue Apr 23, 2022 · 5 comments
Closed

NetBSD error linking against C library with link dependency #96328

tmfink opened this issue Apr 23, 2022 · 5 comments
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. O-netbsd Operating system: NetBSD

Comments

@tmfink
Copy link
Contributor

tmfink commented Apr 23, 2022

I tried this code:

Repo: https://github.com/tmfink/rust-link-issue

build.rs:

fn main() {
    println!("cargo:rustc-link-lib=z");

    let mut builder = cc::Build::new();
    builder.file("foo.c");
    builder.compile("foo");
}

src/main.rs:

#![allow(unused_imports, dead_code)]

use std::os::raw::{c_char, c_int};

extern "C" {
    fn foo() -> usize;

    fn zlibVersion() -> *const c_char;
}

fn main() {
    unsafe {
        println!("{:x}", foo());
    }

    // Uncomment to workaround link issue
    //println!("zlibVersion() = {:?}", zlibVersion as *const u8);
}

foo.c:

#include <stddef.h>
#include <zlib.h>

size_t foo() {
    return (size_t) zlibVersion();
}

I expected this to compile, but got a link error:.

$ cargo run
   Compiling simple-link-test v0.1.0 (/tmp/simple-link)
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.19mun35o71dlrdyq.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.1vhj6pe24ldw4n7s.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.2nf1opqsnl2ilj43.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.3ab3dkn977m6fjp9.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.3ocp4b4cpsu29h9o.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.427yxpxcwm3nubkp.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.4d8kozeslihxvrhw.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.51jbqo2gpt5s1yac.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.cml30bca6em8mat.rcgu.o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.15yztrz1tks35lmt.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/simple-link/target/debug/deps" "-L" "/tmp/simple-link/target/debug/build/simple-link-test-7198944c1ce9a580/out" "-L" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib" "-lz" "-Wl,-Bstatic" "-Wl,--whole-archive" "-lfoo" "-Wl,--no-whole-archive" "-Wl,--start-group" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libstd-88860e2f4119f93f.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libpanic_unwind-990d204efb11a33f.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libminiz_oxide-cdfa5bc4905a97ab.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libadler-4c05554bc3c92490.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libobject-3067f51de7d10974.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libmemchr-ed5ee6ee56f88db6.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libaddr2line-2d5849326f3bbcd0.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libgimli-4868503f350e774c.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_demangle-7afca5d45a1b8bad.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libstd_detect-1e6f1275d7fa7721.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libhashbrown-40532dff13a43508.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_std_workspace_alloc-09b9a8d2a247ae60.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libunwind-714b2e0912c21213.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcfg_if-a59f2f2f9c828895.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/liblibc-851bb90d894793e4.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/liballoc-0aefc5e9fefbf214.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_std_workspace_core-0a7f08ac42a2e82d.rlib" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcore-d76f5c5df0fb96f9.rlib" "-Wl,--end-group" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcompiler_builtins-a67bc12173dadc5f.rlib" "-Wl,-Bdynamic" "-lpthread" "-lrt" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lrt" "-lutil" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib" "-o" "/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow"
  = note: ld: /tmp/simple-link/target/debug/build/simple-link-test-7198944c1ce9a580/out/libfoo.a(foo.o): in function `foo':
          /tmp/simple-link/foo.c:5: undefined reference to `zlibVersion'
          
  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `simple-link-test` due to previous error

Meta

rustc --version --verbose:

rustc 1.60.0 (7737e0b5c 2022-04-04)
binary: rustc
commit-hash: 7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c
commit-date: 2022-04-04
host: x86_64-unknown-netbsd
release: 1.60.0
LLVM version: 14.0.0
$ cc -v
Using built-in specs.
COLLECT_GCC=cc
COLLECT_LTO_WRAPPER=/usr/libexec/lto-wrapper
Target: x86_64--netbsd
Configured with: /usr/src/tools/gcc/../../external/gpl3/gcc/dist/configure --target=x86_64--netbsd --enable-long-long --enable-threads --with-bugurl=http://www.NetBSD.org/Misc/send-pr.html --with-pkgversion='NetBSD nb4 20200810' --with-system-zlib --without-isl --enable-__cxa_atexit --enable-libstdcxx-time=rt --enable-libstdcxx-threads --with-diagnostics-color=auto-if-env --with-tune=nocona --with-default-libstdcxx-abi=new --with-mpc-lib=/var/obj/mknative/amd64-x86_64/usr/src/external/lgpl3/mpc/lib/libmpc --with-mpfr-lib=/var/obj/mknative/amd64-x86_64/usr/src/external/lgpl3/mpfr/lib/libmpfr --with-gmp-lib=/var/obj/mknative/amd64-x86_64/usr/src/external/lgpl3/gmp/lib/libgmp --with-mpc-include=/usr/src/external/lgpl3/mpc/dist/src --with-mpfr-include=/usr/src/external/lgpl3/mpfr/dist/src --with-gmp-include=/usr/src/external/lgpl3/gmp/lib/libgmp/arch/x86_64 --enable-tls --disable-multilib --disable-libstdcxx-pch --build=x86_64--netbsd --host=x86_64--netbsd --with-sysroot=/var/obj/mknative/amd64-x86_64/usr/src/destdir.amd64
Thread model: posix
gcc version 7.5.0 (nb4 20200810) 

$ ld -v
GNU ld (NetBSD Binutils nb1) 2.31.1

Root Cause

The issue is caused by the link arg -lz not appearing at the end.

$ cargo run |& grep 'note:.*cc' | sed 's/= note://' | xargs -n1
cc
-m64
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.19mun35o71dlrdyq.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.1vhj6pe24ldw4n7s.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.2nf1opqsnl2ilj43.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.3ab3dkn977m6fjp9.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.3ocp4b4cpsu29h9o.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.427yxpxcwm3nubkp.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.4d8kozeslihxvrhw.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.51jbqo2gpt5s1yac.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.cml30bca6em8mat.rcgu.o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0.15yztrz1tks35lmt.rcgu.o
-Wl,--as-needed
-L
/tmp/simple-link/target/debug/deps
-L
/tmp/simple-link/target/debug/build/simple-link-test-7198944c1ce9a580/out
-L
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib
-lz
-Wl,-Bstatic
-Wl,--whole-archive
-lfoo
-Wl,--no-whole-archive
-Wl,--start-group
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libstd-88860e2f4119f93f.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libpanic_unwind-990d204efb11a33f.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libminiz_oxide-cdfa5bc4905a97ab.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libadler-4c05554bc3c92490.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libobject-3067f51de7d10974.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libmemchr-ed5ee6ee56f88db6.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libaddr2line-2d5849326f3bbcd0.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libgimli-4868503f350e774c.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_demangle-7afca5d45a1b8bad.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libstd_detect-1e6f1275d7fa7721.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libhashbrown-40532dff13a43508.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_std_workspace_alloc-09b9a8d2a247ae60.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libunwind-714b2e0912c21213.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcfg_if-a59f2f2f9c828895.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/liblibc-851bb90d894793e4.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/liballoc-0aefc5e9fefbf214.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_std_workspace_core-0a7f08ac42a2e82d.rlib
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcore-d76f5c5df0fb96f9.rlib
-Wl,--end-group
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib/libcompiler_builtins-a67bc12173dadc5f.rlib
-Wl,-Bdynamic
-lpthread
-lrt
-lgcc_s
-lc
-lm
-lrt
-lpthread
-lutil
-lrt
-lutil
-Wl,--eh-frame-hdr
-Wl,-znoexecstack
-L
/home/ryucl0ud/.rustup/toolchains/stable-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd/lib
-o
/tmp/simple-link/target/debug/deps/simple_link_test-4fc7c922e98fb1d0
-Wl,--gc-sections
-pie
-Wl,-zrelro,-znow

Workarounds

Use a link wrapper script

You can use a wrapper script that copies the -l* arguments to the end:
rust_link_fix.sh:

#!/bin/sh

LINKER="cc"

link_args=
for arg in "$@"; do
    case "${arg}" in
    -l*) link_args="${link_args} ${arg}" ;;
    esac
done

# add link args to end (again)
exec "${LINKER}" "$@" ${link_args}

.cargo/config:

[target.x86_64-unknown-netbsd]
linker = "rust_link_fix.sh"

Reference the symbol in the Rust code

You can "trick" the linker into pulling in the library by referencing a symbol from that library in the Rust code:
https://github.com/tmfink/rust-link-issue/blob/c82d426559c4cf4c9a2537d52d7643b9c59b0639/src/main.rs#L16-L17

@tmfink tmfink added the C-bug Category: This is a bug. label Apr 23, 2022
@petrochenkov
Copy link
Contributor

-lz doesn't appear at the end because the build script doesn't put it there.

fn main() {
    let mut builder = cc::Build::new();
    builder.file("foo.c");
    builder.compile("foo");

    println!("cargo:rustc-link-lib=z");
}

should work.

@tmfink
Copy link
Contributor Author

tmfink commented Apr 25, 2022

-lz doesn't appear at the end because the build script doesn't put it there.

fn main() {
    let mut builder = cc::Build::new();
    builder.file("foo.c");
    builder.compile("foo");

    println!("cargo:rustc-link-lib=z");
}

should work.

@petrochenkov yes, that was exactly the issue!

It seems that the build script prints:

cargo:rustc-link-lib=static=foo
cargo:rustc-link-search=native=/home/ryucl0ud/dev/simple-link/target/debug/build/simple-link-test-b0ea09127de7178f
cargo:rustc-link-lib=z

which are "greedily" expanded into linker arguments.

I (incorrectly) assumed that rustc would track static and dynamic objects separately and put all of the dynamic linking arguments (like -lz) at the end. After thinking more, I realize that it may not make sense for rustc to try to reorder linker arguments to make the linker happier. Different platforms with the same "linker flavor" may even have slightly different behavior.

In this case, the original code compiled/linked fine on Linux, FreeBSD, and OpenBSD but failed on NetBSD. However, rustc being "fancy" by trying to optimize the order of linker arguments may make it difficult/impossible to get the desired argument order on some platforms--especially on platforms with atypical toolchains/linkers.

With that said, it probably would make sense to at least document the "order preserving behavior" of cargo build scripts and rustc in the Build Scripts section of The Cargo Book.

Thoughts? Does anyone think a behavior change makes sense as opposed to a pure documentation change?

@petrochenkov
Copy link
Contributor

Since the order of libraries is significant, rustc tries to preserve it.
It doesn't always happen, for legacy reasons, but we are at least trying to fix that.

Cargo also preserves the order, from my experience, but it's better to ask cargo maintainers whether are any corner cases.

@petrochenkov
Copy link
Contributor

I wouldn't want to specify precise details for rustc, due to those legacy issues and because in rustc we want to deduplicate the linked native libraries more aggressively.
So what we want to preserve is not the specific order, but rather dependencies between libraries.

But in simple cases like

println!("cargo:rustc-link-lib=a");
println!("cargo:rustc-link-lib=b");
println!("cargo:rustc-link-lib=c");

(no duplicates or anything), it's obvious that we'll need to infer dependencies as a <- b, b <- c and a <- c and the specific order will be preserved too.

bors added a commit to rust-lang/cargo that referenced this issue May 25, 2022
doc: discuss build script instruction order

### What does this PR try to resolve?

It is currently not documented that the order of build script `cargo:` instructions may be relevant to linking.

This has caused issues such as: rust-lang/rust#96328

### How should we test and review this PR?

Build/view documentation.

### Additional information

- Cargo maintainers should fact check my wording.
- We may need to discuss if this should also be documented for `rustc`
- Maintainers should ensure that this change does not prevent a change in what is currently unspecified behavior. Perhaps `cargo` will want to rearrange link arguments itself to resolve issues in the future?
@ChrisDenton ChrisDenton added the needs-triage-legacy Old issue that were never triaged. Remove this label once the issue has been sufficiently triaged. label Jul 16, 2023
@Enselic
Copy link
Member

Enselic commented Jul 30, 2024

Triage:

I wouldn't want to specify precise details for rustc, due to those legacy issues and because in rustc we want to deduplicate the linked native libraries more aggressively.

No objections were raised, so let's close this issue now.

@Enselic Enselic closed this as not planned Won't fix, can't repro, duplicate, stale Jul 30, 2024
@saethlin saethlin added A-linkage Area: linking into static, shared libraries and binaries O-netbsd Operating system: NetBSD and removed needs-triage-legacy Old issue that were never triaged. Remove this label once the issue has been sufficiently triaged. labels Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. O-netbsd Operating system: NetBSD
Projects
None yet
Development

No branches or pull requests

5 participants