x86_64-unknown-linux-musl: error: could not find native static library c, perhaps an -L flag is missing?

[vi]

How do I use Xargo for x86_64-unknown-linux-musl?

My attempt:

$ cat Cargo.toml
[package]
name = "xargotest"
version = "0.1.0"
authors = ["Vitaly _Vi Shukela <vi0oss@gmail.com>"]

[profile.release]
panic = "abort"

$ cat Xargo.toml 
[dependencies]
std = {default-features=false, features=["force_alloc_system"]}

$ cat src/main.rs 
#![no_std]
#![feature(alloc_system)]

extern crate std;
extern crate alloc_system;

fn main() {
    let _ = ::std::io::copy(&mut ::std::io::stdin(), &mut ::std::io::stdout());
}

$ xargo build --target=x86_64-unknown-linux-musl --release
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling unwind v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libunwind)
   Compiling core v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore)
   Compiling gcc v0.3.45
   Compiling libc v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/rustc/libc_shim)
   Compiling libc v0.2.22
   Compiling filetime v0.1.10
   Compiling build_helper v0.1.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/build_helper)
   Compiling std v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd)
   Compiling compiler_builtins v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcompiler_builtins)
warning: ../compiler-rt/lib/builtins/divdc3.c:21:1: warning: conflicting types for built-in function ‘__divdc3’
warning:  __divdc3(double __a, double __b, double __c, double __d)
warning:  ^
warning: ../compiler-rt/lib/builtins/divsc3.c:21:1: warning: conflicting types for built-in function ‘__divsc3’
warning:  __divsc3(float __a, float __b, float __c, float __d)
warning:  ^
warning: ../compiler-rt/lib/builtins/divxc3.c:22:1: warning: conflicting types for built-in function ‘__divxc3’
warning:  __divxc3(long double __a, long double __b, long double __c, long double __d)
warning:  ^
warning: ../compiler-rt/lib/builtins/muldc3.c:21:1: warning: conflicting types for built-in function ‘__muldc3’
warning:  __muldc3(double __a, double __b, double __c, double __d)
warning:  ^
warning: ../compiler-rt/lib/builtins/mulsc3.c:21:1: warning: conflicting types for built-in function ‘__mulsc3’
warning:  __mulsc3(float __a, float __b, float __c, float __d)
warning:  ^
warning: ../compiler-rt/lib/builtins/mulxc3.c:23:1: warning: conflicting types for built-in function ‘__mulxc3’
warning:  __mulxc3(long double __a, long double __b, long double __c, long double __d)
warning:  ^
   Compiling std_unicode v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd_unicode)
   Compiling alloc v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc)
   Compiling rand v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/librand)
   Compiling collections v0.0.0 (file:///home/rust/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcollections)
error: could not find native static library `c`, perhaps an -L flag is missing?

error: Could not compile `libc`.
Build failed, waiting for other jobs to finish...
error: build failed
error: `"cargo" "build" "--release" "--manifest-path" "/tmp/xargo.ToNm9ngHiAgZ/Cargo.toml" "--target" "x86_64-unknown-linux-musl" "-p" "std"` failed with exit code: Some(101)
note: run with `RUST_BACKTRACE=1` for a backtrace

Building without --target=x86_64-unknown-linux-musl works.

[vi]

By the way even with xargo rustc --release -- -C lto -C opt-level=z and strip I get a 111K executable. How do I get file size comparable to C version (5 kilobytes) without actual no_std? Shouldn't LTO throw almost everything away?

[japaric]

The Rust compiler assumes that the musl libc.a will be contained in the sysroot for this particular target. That library won't be in Xargo sysroot because Xargo doesn't build libc.a when it builds sysroot. It doesn't do that because the standard crates don't contain instructions for that in their build scripts. To fix this you'll have to compile musl libc.a yourself and then pass a -L to the linker (xargo rustc -- -C link-arg=..)

I get a 111K executable.

Formatting stuff heavily bloats Rust programs. Even an empty program fn main() contains formatting code because the code that runs before main to initialize the Rust runtime may panic and those panic messages need to be formatted.

[vi]

It was 111K because, although being built by xargo, usual non-customized libstd was linked in. A copy of usual libstd_....rlib was delivered to $HOME/.xargo. Custom-built libstd in /tmp was discarded somehow.

After manual compilation of libstd and manually linking it it got to about 51K.

[japaric]

Custom-built libstd in /tmp was discarded somehow.

That ... sounds like a bug?

about 51K.

You can profile the binary size if you run nm -C --size-sort on the file (probably before stripping it). That should tell what symbol (function / static variable) is taking up the most space.

xargo rustc --release -- -C lto -C opt-level=z

btw, this only builds the top crate with opt-level=z. You should set opt-level in Cargo.toml (profile.release) if you want to compile the whole dependency graph with opt-level=z.

I think there's nothing actionable left to be done for this issue?

[vi]

It fails to recognize z, although s works (although it is before rustup update).

nm -C --size-sort

There was a lot of things about backtracing. I also used Codesize which shows it as a tree in GUI, which emphasized that backtracing-related things are about 30k.

That ... sounds like a bug?

Maybe I'll reproduce it again and file a dedicated report.

[japaric]

backtracing-related things are about 30k.

IIRC, std has a backtrace Cargo feature. If you disable it, you should be able to remove (part of) those 30k with the caveat that RUST_BACKTRACE will no longer work.

[vi]

Yes. The problem was that all those feature settings in Xargo.toml for std had no effect at all because of usualy pre-built rustup-downloaded libstd.rlib were used (just from ~/.xargo instead of the usual location).

Anyway it is offtopic to this issue, because of topic of the issue is that xargo fails when I specify target Musl.

[drmikehenry]

All,

I've been experimenting with building std from source, using both Xargo and the new Cargo build-std feature. Here are the steps I used on my Ubuntu workstation to get a Xargo build for the x86_64-unknown-linux-musl target.

Install a known-good nightly toolchain, with Rust library source, and support for the x86_64-unknown-linux-musl target:

rustup install nightly-2020-08-20
rustup default nightly-2020-08-20
rustup component add rust-src
rustup target add x86_64-unknown-linux-musl
cargo install xargo

Create a temporary directory for testing, and a new project hello-xargo, configured with Xargo.toml and Cargo.toml tailored for small size:

mkdir -p ~/tmp/xargo
cd ~/tmp/xargo

cargo new hello-xargo
cd hello-xargo

cat <<EOF > Xargo.toml
[dependencies]
std = {default-features=false, features=["panic_immediate_abort"]}
EOF

cat <<EOF >> Cargo.toml
[profile.dev]
panic = "abort"

[profile.release]
lto = true
codegen-units = 1
opt-level = "z"
panic = "abort"
EOF

Attempt build:

xargo build --target x86_64-unknown-linux-musl --release

Failed:

error occurred: Failed to find tool. Is `musl-gcc` installed?

Install Ubuntu package musl-tools to get the wrapper executable musl-gcc (this package also implies the musl and musl-dev packages):

sudo apt install -y musl-tools

Retry build:

xargo build --target x86_64-unknown-linux-musl --release

Failed:

cargo:warning=cc: error: ../../src/llvm-project/libunwind/src/Unwind-EHABI.cpp: No such file or directory

This file is part of the llvm-project repository which is not distributed for some reason with rustup component add rust-src.

Clone source repositories:

cd ~/tmp/xargo
git clone https://github.com/rust-lang/rust
git clone https://github.com/rust-lang/llvm-project

Determine the rust repository commit-hash for this toolchain:

rustc -vV | grep commit-hash

Output:

commit-hash: 32c654a9795b0d88541e56ba9da4150e34f1d5f9

Determine the llvm-project submodule's commit hash using git ls-tree:

cd rust
git ls-tree 32c654a9795b0d88541e56ba9da4150e34f1d5f9 src/llvm-project

Output:

160000 commit 86b120e6f302d39cd6973b6391fb299d7bc22122  src/llvm-project

Create source tarball for this version of llvm-project, inserting a leading llvm-project/ path for tidiness:

cd ../llvm-project

git archive 86b120e6f302d39cd6973b6391fb299d7bc22122 \
  --prefix llvm-project/ \
  --output ../llvm-project-src.tar.gz

Install llvm-project sources from the tarball into the expected area of ~/.rustup for the current toolchain:

cd ~/.rustup/toolchains/nightly-2020-08-20-x86_64-unknown-linux-gnu
mkdir lib/rustlib/src/rust/src
tar -C lib/rustlib/src/rust/src -xf ~/tmp/xargo/llvm-project-src.tar.gz

Retry build:

cd ~/tmp/xargo/hello-xargo
xargo build --target x86_64-unknown-linux-musl --release

Output:

error: could not find native static library `c`, perhaps an -L flag is missing?

Though installed, Musl's libc.a isn't in the link path by default.

Determine the libc.a directory (it comes from the musl-dev package):

dpkg -L 'musl-dev' | grep 'libc\.a'

Output:

/usr/lib/x86_64-linux-musl/libc.a

Setup an -L option pointing to the above directory for libc.a; also include the "-Cembed-bitcode=yes" work-around for #292, since we'll need that soon anyway):

cat <<EOF >> ~/.cargo/config
[build]
# Work-around for https://github.com/japaric/xargo/issues/292:
rustflags = ["-Cembed-bitcode=yes"]

[target.x86_64-unknown-linux-musl]
# Path to Musl libc.a (plus bitcode work-around)
rustflags = ["-L/usr/lib/x86_64-linux-musl", "-Cembed-bitcode=yes"]
EOF

Retry build:

xargo build --target x86_64-unknown-linux-musl --release

Success; it's a statically linked executable:

file ./target/x86_64-unknown-linux-musl/release/hello-xargo

Output:

./target/x86_64-unknown-linux-musl/release/hello-xargo:
  ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked,
  BuildID[sha1]=323a305a723f0b29f81ee185865ffe85d51de28e, with debug_info,
  not stripped

It runs:

./target/x86_64-unknown-linux-musl/release/hello-xargo

Output::

Hello, world!

Size:

size ./target/x86_64-unknown-linux-musl/release/hello-xargo

   text    data     bss     dec     hex filename
  33196     800    4688   38684    971c ./target/x86_64-unknown-linux-musl/release/hello-xargo

Stripped file size:

strip ./target/x86_64-unknown-linux-musl/release/hello-xargo
ls -lh ./target/x86_64-unknown-linux-musl/release/hello-xargo

-rwxrwxr-x 2 mike mike 37K Aug 26 19:15 ./target/x86_64-unknown-linux-musl/release/hello-xargo

For comparison, here is a build using Cargo's build-std feature (https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#build-std):

cargo clean
cargo build \
  --target x86_64-unknown-linux-musl \
  --release \
  -Z build-std \
  -Z build-std-features=panic_immediate_abort

Failed with many errors like this:

error: duplicate lang item in crate `core`: `Range`.

This is fixed by restricting the crates to be built to std,panic_abort:

cargo clean
cargo build \
  --target x86_64-unknown-linux-musl \
  --release \
  -Z build-std=std,panic_abort \
  -Z build-std-features=panic_immediate_abort

Success:

size ./target/x86_64-unknown-linux-musl/release/hello-xargo

   text    data     bss     dec     hex filename
  33212     800    4688   38700    972c ./target/x86_64-unknown-linux-musl/release/hello-xargo

Hopefully these terse notes are helpful for using the x86_64-unknown-linux-musl toolchain.