mimalloc is not being linked when building with the Cargo with LTO enabled.

[SpacingBat3]

Description

As I've started to bring the latest version of hunt into the AUR, I've happened to get some linking errors on my machine. What's more weird, it happened only for me while running cargo build within makepkg enviroment, which could explain it went unnoticed for quite some time.

I've linked the issue to the lack of mimalloc in the linker command-line while building it with LTO enabled – in makepkg, this is done simply by appending lto to the options=() array in the makepkg.conf or in package-specific options. I currently have no clue what LTO changes in terms of generating a command for linking the native libraries into the final binary, I didn't dig into this problem that much. So there might be a better proper way to fix it, maybe it's even an issue for an upstream project to resolve in the reality.

Logs

I'll skip the part that mentions exactly which symbols aren't being found by the linker, to keep the logs a bit shorter than they would be normally, since I've already found the right native library that provides them (symbols in logs quite point to mimalloc with the errors telling there's undefined reference to symbols like mi_free or mi_malloc_aligned). I've also splitted the logs by the beginning part and the ending part.

Here's what's at start of the logs (basically the command line used for linting this, with some stuff replaced to not to leak any info about some paths on my PC or my username):

error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="[REDACTED]" VSLANG="1033" "cc" "-m64" "/tmp/rustcIiP3az/symbols.o" "[REPODIR]/hunt-2.1.0/target/release/deps/hunt-9c727d0082a78a8e.hunt.5e3c86399b7808a-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "[REPODIR]/hunt-2.1.0/target/release/deps" "-L" "[REPODIR]/hunt-2.1.0/target/release/build/libmimalloc-sys-d2896777ded63943/out" "-L" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcIiP3az/liblibmimalloc_sys-fdda0fd318ea1a6d.rlib" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-8e138eaf26ebb4a8.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "[REPODIR]/hunt-2.1.0/target/release/deps/hunt-9c727d0082a78a8e" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"

Here's useful part and the last line of the logs I get from cargo, it's basically telling how the one might resolve this in their project (I used that to go with the solution utilising the build.rs):

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `hunt` (bin "hunt") due to previous error

Workarounds

I was able to workaround the error by enforcing to link mimalloc with the following build.rs (placed in the top-level directory of the Rust project, i.e. next to Cargo.toml):

fn main() {
    println!("cargo:rustc-link-lib=mimalloc");
}

I suppose claiming that LTO is unsupported is another way to resolve this issue, so either action could be done. As mentioned before, I'm myself the current maintainer of hunt in AUR, so I currently went for using the build.rs as a workaround, but an option to disable LTO for this package is possible as well.

[LyonSyonII]

Thank you for opening the issue!

I didn't know hunt was in the AUR, that's great!

Regarding the problem, LTO provides a small but measurable boost in performance, and mimalloc a pretty massive one (~15% from my benchmarks), so it would be great if both could be kept.

Does the workaround have any downsides?
If not, I'll add it to the main repository to avoid errors on other packaging systems.

[SpacingBat3]

Does the workaround have any downsides?

Nothing I can think of, I guess maybe build.rs is being compiled under-the-hood and that means slower build times, but I doubt that it would be significant difference to notice by most people, after all what this "program" does is to print a single message to stdout for Cargo to parse.

I should also say I've only reproduced this bug on Linux and I don't know how build.rs or this bug affects other OSes, I've also haven't dig enough yet with your dependencies or even features to know whenever you use mimalloc library outside of Linux and if it is used as a feature that can be disabled with the Cargo or not. You might consider writing a bit more dynamic build.rs if the use of mimalloc is different.

I didn't know hunt was in the AUR, that's great!

I've started maintaining it quite recently, mostly because I felt to be able to adopt some orphaned package and I went for the most popular yet flagged as outdated one back then. I also have some experience with Rust (which will definitely come in handy), it's nothing impresive (I see myself still as a beginner, as I still have to become way more familiar with a language syntax like standard library, macros, multi-threading, practical use of async and C interoperability to consider myself knowledgeable enough to see myself as a Rust intermediate dev).

If you want to, you might share there your AUR username and I might transfer package ownership to you or make you a co-maintainer, so you might as well by yourself maintain hunt. For me, stuff like version bumps etc. are quite relatively easy to do, I've already made a quite clever script to automate it and a GIT repo containing a submodules to maintain it all in more centralised way. I'll try to share that configuration soon with a bug tracker over a GitHub.

[LyonSyonII]

I'll add the build.rs file then, if the deploy workflow fails then we'll look for other solutions.

AUR

Oh, I don't use Arch anymore, I was just suprised that hunt was being mantained in the AUR.
If you could continue mantaining it, I'd appreciate it.

[LyonSyonII]

The deployment worked as expected, closing the issue.
If more problems arise please reopen it.