Experimental MUSL target support for the compiler #24777

alexcrichton · Apr 24, 2015

These commits build on some great work on reddit for adding MUSL support to the compiler. This goal of this PR is to enable a --target x86_64-unknown-linux-musl argument to the compiler to work A-OK. The outcome here is that there are 0 compile-time dependencies for a MUSL-targeting build except for a linker. Currently this also assumes that MUSL is being used for statically linked binaries so there is no support for dynamically linked binaries with MUSL.

MUSL support largely just entailed munging around with the linker and where libs are located, and the major highlights are:

The entirety of libc.a is included in liblibc.rlib (statically included as an archive).
The entirety of libunwind.a is included in libstd.rlib (like with liblibc).
The target specification for MUSL passes a number of ... flavorful options! Each option is documented in the relevant commit.
The entire test suite currently passes with MUSL as a target, except for:
- Dynamic linking tests are all ignored as it's not supported with MUSL
- Stack overflow detection is not working MUSL yet (I'm not sure why)
There is a language change included in this PR to add a target_env #[cfg] directive. This is used to conditionally build code for only MUSL (or for linux distros not MUSL). I highly suspect that this will also be used by Windows to target MSVC instead of a MinGW-based toolchain.

To build a compiler targeting MUSL you need to follow these steps:

Clone the current MUSL repo from git://git.musl-libc.org/musl. Build this as usual and install it.
Clone and build LLVM's libcxxabi library. Only the libunwind.a artifact is needed. I have tried using upstream libunwind's source repo but I have not gotten unwinding to work with it unfortunately. Move libunwind.a adjacent to MUSL's libc.a
Configure a Rust checkout with --target=x86_64-unknown-linux-musl --musl-root=$MUSL_ROOT where MUSL_ROOT is where you installed MUSL in step 1.

I hope to improve building a copy of libunwind as it's still a little sketchy and difficult to do today, but other than that everything should "just work"! This PR is not intended to include 100% comprehensive support for MUSL, as future modifications will probably be necessary.

rust-highfive · Apr 24, 2015

r? @pcwalton

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · Apr 24, 2015

r? @brson

brson · Apr 24, 2015

Why 'x86_64-unknown-linux-musl'? It seems reasonable to me, but Google doesn't have a lot of hits for it.

nikomatsakis · Apr 24, 2015

Neat!

alexcrichton · Apr 24, 2015

@brson I found it followed clang's target triple format, but I personally got the idea from the reddit post.

brson · Apr 24, 2015

r=me

alexcrichton · Apr 24, 2015

@brson I've pushed some more commits clarifying some documentation and also feature gating target_env by just not defining it unless the gate is turned on. I've avoided rebasing for now but I'll rebase all the commits up after review.

alexcrichton · Apr 24, 2015

@bors: r=brson 01963a6

bors · Apr 25, 2015

⌛️ Testing commit 01963a6 with merge 9e28730...

bors · Apr 25, 2015

💔 Test failed - auto-win-64-nopt-t

alexcrichton · Apr 27, 2015

@bors: r=brson faa1a81

bors · Apr 28, 2015

⌛️ Testing commit faa1a81 with merge a574d45...

bors · Apr 28, 2015

💔 Test failed - auto-win-32-nopt-t

alexcrichton · Apr 28, 2015

@bors: r=brson d6e7c7a

bors · Apr 28, 2015

⌛️ Testing commit d6e7c7a with merge c975c65...

bors · Apr 28, 2015

💔 Test failed - auto-win-64-nopt-t

alexcrichton · Apr 28, 2015

@bors: r=brson 247842b

bors · Apr 28, 2015

⌛️ Testing commit 247842b with merge cadc67e...

bors · Apr 28, 2015

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-32-nopt-t, auto-win-32-opt, auto-win-64-nopt-t, auto-win-64-opt

aidanhs · May 10, 2015

@alexcrichton got any details of the issues you had with libunwind upstream? I only ask because a toy program I tried seemed to work ok, so I guess it's related to the additional complexity in the compiler?

alexcrichton · May 10, 2015

@aidanhs unfortunately I could never get any more detail other than "unwinding didn't work". I was able to build libunwind upstream with musl-gcc (aka build it against MUSL's libc.a), and it was also correctly linked into the standard library and Rust programs. When running, however, it either couldn't find the unwind tables or couldn't find the handler for the unwind, as all attempts to unwind failed with "handler not found".

In your test program, did you trigger an unwind, or was it just running as usual? Also, did you configure libunwind with any special flags or anything?

nagisa · May 10, 2015

nagisa reviewed May 10, 2015

View changes

nagisa · May 10, 2015

src/libstd/sys/unix/thread.rs

+        let lib = match DynamicLibrary::open(None) {
+            Ok(l) => l,
+            Err(..) => return,
+        };


This might be why stack overflow detection is not working.

Hm, could you elaborate a little more? This should in theory just affect the return value of this function, which I'm not sure how connected it is to stack sizes.

Before it failed loud and clear when this didn’t work, but now min_stack_size returns PTHREAD_STACK_MIN instead which might or might not be a correct value to use in the stack size calculations.

/me shrugs.

aidanhs · May 10, 2015

@alexcrichton oops, I didn't get unwinding working (though normal execution worked). I did get a different error to you though:

$ ./target/debug/binary
thread '<main>' panicked at 'mypanic', src/main.rs:28
fatal runtime error: Could not unwind stack, error = 5
Illegal instruction (core dumped)

Sorry for the noise!

alexcrichton · May 10, 2015

I did get a different error to you though:

Ah that's actually the same error that I was seeing, oh well!

rust-highfive assigned pcwalton Apr 24, 2015

rust-highfive assigned brson and unassigned pcwalton Apr 24, 2015

alexcrichton force-pushed the alexcrichton:musl branch from fe31352 to 01963a6 Apr 24, 2015

alexcrichton force-pushed the alexcrichton:musl branch from 01963a6 to faa1a81 Apr 27, 2015

alexcrichton force-pushed the alexcrichton:musl branch from faa1a81 to d6e7c7a Apr 28, 2015

alexcrichton force-pushed the alexcrichton:musl branch from d6e7c7a to 247842b Apr 28, 2015

bors merged commit 247842b into rust-lang:master Apr 28, 2015
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

alexcrichton deleted the alexcrichton:musl branch Apr 30, 2015

bstrie referenced this pull request May 5, 2015
Closed
Link statically against musl libc #24

aidanhs referenced this pull request May 9, 2015
Closed
Add support for alternative C libraries #625

kornelski referenced this pull request Jul 4, 2015
Closed
What if pngquant was written in Rust? #161

asaaki referenced this pull request Aug 4, 2015
Closed
Build fails with musl #27290

SimonSapin referenced this pull request Aug 21, 2015
Closed
Binary releases of Rust targeting musl #27933

ranma42 referenced this pull request Oct 28, 2015
Merged
1.4 release notes #29416

aidanhs referenced this pull request Aug 17, 2017
Merged
Support dynamically-linked and/or native musl targets #40113

jethrogb referenced this pull request Jan 9, 2019
Merged
Stabilize cfg_target_vendor #57465

tiero referenced this pull request Feb 18, 2019
Closed
Release: would be possible to statically compile to a single binary file? #115

rust-lang/rust

Join GitHub today

Experimental MUSL target support for the compiler #24777

Conversation

alexcrichton commented Apr 24, 2015

rust-highfive assigned pcwalton Apr 24, 2015

This comment has been minimized.

rust-highfive commented Apr 24, 2015

This comment has been minimized.

alexcrichton commented Apr 24, 2015

rust-highfive assigned brson and unassigned pcwalton Apr 24, 2015

This comment has been minimized.

brson commented Apr 24, 2015

This comment has been minimized.

nikomatsakis commented Apr 24, 2015

This comment has been minimized.

alexcrichton commented Apr 24, 2015

This comment has been minimized.

brson commented Apr 24, 2015

This comment has been minimized.

alexcrichton commented Apr 24, 2015

alexcrichton force-pushed the alexcrichton:musl branch from fe31352 to 01963a6 Apr 24, 2015

This comment has been minimized.

alexcrichton commented Apr 24, 2015

This comment has been minimized.

bors commented Apr 25, 2015

bors added a commit that referenced this pull request Apr 25, 2015

This comment has been minimized.

bors commented Apr 25, 2015

alexcrichton added some commits Apr 21, 2015

alexcrichton force-pushed the alexcrichton:musl branch from 01963a6 to faa1a81 Apr 27, 2015

This comment has been minimized.

alexcrichton commented Apr 27, 2015

This comment has been minimized.

bors commented Apr 28, 2015

bors added a commit that referenced this pull request Apr 28, 2015

This comment has been minimized.

bors commented Apr 28, 2015

alexcrichton force-pushed the alexcrichton:musl branch from faa1a81 to d6e7c7a Apr 28, 2015

This comment has been minimized.

alexcrichton commented Apr 28, 2015

This comment has been minimized.

bors commented Apr 28, 2015

bors added a commit that referenced this pull request Apr 28, 2015

This comment has been minimized.

bors commented Apr 28, 2015

alexcrichton force-pushed the alexcrichton:musl branch from d6e7c7a to 247842b Apr 28, 2015

This comment has been minimized.

alexcrichton commented Apr 28, 2015

bors added a commit that referenced this pull request Apr 28, 2015

This comment has been minimized.

bors commented Apr 28, 2015

This comment has been minimized.

bors commented Apr 28, 2015

Hide details View details bors merged commit 247842b into rust-lang:master Apr 28, 2015 2 checks passed

2 checks passed

alexcrichton deleted the alexcrichton:musl branch Apr 30, 2015

bstrie referenced this pull request May 5, 2015

aidanhs referenced this pull request May 9, 2015

This comment has been minimized.

aidanhs commented May 10, 2015

This comment has been minimized.

alexcrichton commented May 10, 2015

nagisa reviewed May 10, 2015 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

aidanhs commented May 10, 2015

This comment has been minimized.

alexcrichton commented May 10, 2015

kornelski referenced this pull request Jul 4, 2015

asaaki referenced this pull request Aug 4, 2015

SimonSapin referenced this pull request Aug 21, 2015

ranma42 referenced this pull request Oct 28, 2015

aidanhs referenced this pull request Aug 17, 2017

jethrogb referenced this pull request Jan 9, 2019

Centril added a commit to Centril/rust that referenced this pull request Jan 14, 2019

Centril added a commit to Centril/rust that referenced this pull request Jan 14, 2019

Centril added a commit to Centril/rust that referenced this pull request Jan 14, 2019

alexcrichton force-pushed the alexcrichton:musl branch from `fe31352` to `01963a6` Apr 24, 2015

alexcrichton force-pushed the alexcrichton:musl branch from `01963a6` to `faa1a81` Apr 27, 2015

alexcrichton force-pushed the alexcrichton:musl branch from `faa1a81` to `d6e7c7a` Apr 28, 2015

alexcrichton force-pushed the alexcrichton:musl branch from `d6e7c7a` to `247842b` Apr 28, 2015

bors merged commit `247842b` into rust-lang:master Apr 28, 2015
2 checks passed

nagisa reviewed May 10, 2015

View changes