Use musl libc for published binaries

[alexmiller-apple]

We've fought with glibc versioning issues over time, and it's the reason for our docker dev environment being centos 6, as it's the oldest glibc (2.12) that has a c++17 toolchain backported to it. Finding a way to link against musl to avoid glibc would free us from having to use older distributions, and be able to use modern tools to produce binaries that could run on ancient systems.

(Filing to have an issue for this, as it's been discussed a few times, but without anything to track the idea.)

[alexmiller-apple]

Which I actually filed to have a place to post updates:

Building a musl toolchain for gcc turned out to be the musl-recommended, but incorrect approach to this task. We would then need to build libstdc++ manually, and all other libraries we depend on as well.

Instead, the easy solution is to compile on Alpine Linux, which is a linux distribution that entirely uses musl, so it has already resolved the task of how to build all dependencies with musl. I've gotten FoundationDB compiling against musl, with a number of changes...

• Boost 1.67.0 is incompatible with musl, as there's a static assert that fails when run against musl's headers. Upgrading to Boost 1.71.0 solves this, but also required addressing Fails to build against Boost 1.70.0 #1312
• gnu_dev_major and gnu_dev_minor need to be their respective more portable counterparts major and minor
• musl doesn't support dladdr1, so using RTLD_DL_LINKMAP to get the name of the file that's providing the currently executing code doesn't work.
• execinfo.h and other backtrace related things don't exist in musl. There's a separate libexecinfo that can be included to provide this instead.
• aligned_alloc doesn't require _ISOC11_SOURCE to be defined, which makes it hard to detect when we need to define it and when we don't.
• Thread-targetted signals via sigevent aren't yet implemented, so SIGEV_THREAD_ID and sev._sigev_un._tid result in compilation errors. There's an equivalent ksigevent interface that can be used if one directly syscalls to the kernel.
• musl hasn't implemented ucontext's make/swapcontext functions, and is reluctant to do so in the future as well. This means that FlowCoro needs to use a setjmp-based stack switching implementation. However, forcing USE_SETJMP instead of USE_UCONTEXT in libcoroutine results in simulation crashes.

So for us to actually proceed with doing this, we'd need to:

• Debug the setjmp crashes
• Test that ksigevent is equivalent to sigeven for our profiling usage
• Verify that libexecinfo is a sufficient replacement
• Verify that libfdb_c.so compiled against musl plays fine when linked into an application using glibc

And then we could begin the Sisyphean task of running an incredible amount of testing on the result to build confidence.

[alexmiller-apple]

There's an article on arangodb using musl for similar reasons. They comment that there's an unexpected gain also, that the resulting binary then works under the Windows Subsystem for Linux, which could be an interesting future story for our Windows support...

[alexmiller-apple]

This is something I might throw random weekend time at, but not anything that I'm actively working on.