Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redundant linker flag warning is wrong and breaks linking #47989

Closed
wthrowe opened this issue Feb 3, 2018 · 5 comments
Closed

redundant linker flag warning is wrong and breaks linking #47989

wthrowe opened this issue Feb 3, 2018 · 5 comments
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug.

Comments

@wthrowe
Copy link
Contributor

wthrowe commented Feb 3, 2018

If one passes the same -lfoo option twice to rustc it warns redundant linker flag specified for library `foo` and ignores the second argument. Linkers are sensitive to the order of their arguments and passing the same argument twice is common and sometimes necessary to get a program to link correctly.

In fact, rustc itself passes the linker (cleaned up a bit) -ldl -lrt -lpthread -lpthread -lgcc_s -lc -lm -lrt -lpthread -lutil -lutil, which includes four "redundant" linker flags (at least two of which are actually redundant, to be fair).

@Mark-Simulacrum Mark-Simulacrum self-assigned this Feb 4, 2018
@Mark-Simulacrum Mark-Simulacrum added A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug. labels Feb 4, 2018
@Mark-Simulacrum Mark-Simulacrum removed their assignment Feb 4, 2018
@Mark-Simulacrum
Copy link
Member

This is somewhat not trivial to fix as we don't know whether a given -l flag was passed with the purpose of changing the link kind or name, or actually passing the same library twice for linkage. This branch has the initial workings of a viable idea, but it doesn't pass tests.

There are also to an extent some backwards compatibility concerns here.

---- [run-pass] run-pass/rfc1717/library-override.rs stdout ----

error: compilation failed!
status: exit code: 101
command: "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "/home/mark/Build/rust/src/test/run-pass/rfc1717/library-override.rs" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.stage1-x86_64-unknown-linux-gnu" "-Crpath" "-O" "-Zmiri" "-Zunstable-options" "-Lnative=/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-lstatic=wronglibrary:rust_test_helpers" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.stage1-x86_64-unknown-linux-gnu.aux"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.library_override0-317d481089b8c8fe83113de504472633.rs.rcgu.o" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.library_override1-317d481089b8c8fe83113de504472633.rs.rcgu.o" "-o" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.stage1-x86_64-unknown-linux-gnu" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.crate.allocator.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/test/run-pass/rfc1717/library-override.stage1-x86_64-unknown-linux-gnu.aux" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-l" "rust_test_helpers" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "-l" "rust_test_helpers" "-Wl,--no-whole-archive" "-L" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-l" "std-57fdceb1c52b4db2" "-Wl,-Bstatic" "/home/mark/Build/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-136e26942e0df602.rlib" "-Wl,-Bdynamic" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-l" "util" "-Wl,-rpath,$ORIGIN/../../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-rpath,/home/mark/Build/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--enable-new-dtags"
  = note: /home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers/librust_test_helpers.a(rust_test_helpers.o): In function `rust_dbg_extern_identity_u32':
          rust_test_helpers.c:(.text.rust_dbg_extern_identity_u32+0x0): multiple definition of `rust_dbg_extern_identity_u32'
          /home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers/librust_test_helpers.a(rust_test_helpers.o):rust_test_helpers.c:(.text.rust_dbg_extern_identity_u32+0x0): first defined here
          /home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers/librust_test_helpers.a(rust_test_helpers.o): In function `rust_dbg_extern_identity_u64':
          rust_test_helpers.c:(.text.rust_dbg_extern_identity_u64+0x0): multiple definition of `rust_dbg_extern_identity_u64'
          /home/mark/Build/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers/librust_test_helpers.a(rust_test_helpers.o):rust_test_helpers.c:(.text.rust_dbg_extern_identity_u64+0x0): first defined here
...

@lovesegfault
Copy link
Contributor

Could we perhaps at least add some option to ignore this?

@dcreager
Copy link
Contributor

I was recently bitten by this bug, trying to link multiple internal helper C++ libraries into a Rust executable. I was able to put together a minimal repro: on Linux, one of the subpackages builds successfully, and one gets linker errors, and the only difference between the two is the order in which the helper libraries are built and added to the linker command line.

dcreager added a commit to dcreager/rust that referenced this issue Dec 20, 2018
When a library (L1) is passed to the linker multiple times, this is
sometimes purposeful: there might be several other libraries in the
linker command (L2 and L3) that all depend on L1.  You'd end up with a
(simplified) linker command that looks like:

    -l2 -l1 -l3 -l1

With the previous behavior, when rustc encountered a redundant library,
it would keep the first instance, and remove the later ones, resulting
in:

    -l2 -l1 -l3

This can cause a linker error, because on some platforms (e.g. Linux),
the linker will only include symbols from L1 that are needed *at the
point it's referenced in the command line*.  So if L3 depends on
additional symbols from L1, which aren't needed by L2, the linker won't
know to include them, and you'll end up with "undefined symbols" errors.

A better behavior is to keep the *last* instance of the library:

    -l2 -l3 -l1

This ensures that all "downstream" libraries have been included in the
linker command before the "upstream" library is referenced.

Fixes rust-lang#47989
@dcreager
Copy link
Contributor

I wrote up a potential fix in #57018. We'd still remove redundant -l flags from the linker command, but instead of keeping the first one, we'd now keep the last one. The new behavior passes CI, and I've verified (via cargo +local build) that it also fixes all of the linker errors in my repro repo.

@rohit-136
Copy link

rohit-136 commented Mar 4, 2019

@dcreager It still says "redundant linker flag specified for library stdc++".

bors added a commit that referenced this issue Mar 20, 2019
Keep last redundant linker flag, not first

When a library (L1) is passed to the linker multiple times, this is sometimes purposeful: there might be several other libraries in the linker command (L2 and L3) that all depend on L1.  You'd end up with a (simplified) linker command that looks like:

```
-l2 -l1 -l3 -l1
```

With the previous behavior, when rustc encountered a redundant library, it would keep the first instance, and remove the later ones, resulting in:

```
-l2 -l1 -l3
```

This can cause a linker error, because on some platforms (e.g. Linux), the linker will only include symbols from L1 that are needed *at the point it's referenced in the command line*.  So if L3 depends on additional symbols from L1, which aren't needed by L2, the linker won't know to include them, and you'll end up with "undefined symbols" errors.

A better behavior is to keep the *last* instance of the library:

```
-l2 -l3 -l1
```

This ensures that all "downstream" libraries have been included in the linker command before the "upstream" library is referenced.

Fixes #47989
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries C-bug Category: This is a bug.
Projects
None yet
Development

No branches or pull requests

5 participants