1.1.1 fails to build with MUSL libc due to missing linux/mman.h

[sfackler]

Building OpenSSL 1.1.1 against MUSL libc fails to find the linux/mman.h header:

$ curl -L https://www.openssl.org/source/openssl-1.1.1.tar.gz | tar --strip-components=1 -xzf -
$ export CC=musl-gcc
$ ./Configure --prefix=$OPENSSL_DIR linux-x86_64 no-shared no-async no-engine
$ make
[...]
musl-gcc  -I. -Icrypto/include -Iinclude -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/openssl/ssl\"" -DENGINESDIR="\"/openssl/lib/engines-1.1\"" -DNDEBUG  -MMD -MF crypto/mem_sec.d.tmp -MT crypto/mem_sec.o -c -o crypto/mem_sec.o crypto/mem_sec.c
crypto/mem_sec.c:37:27: fatal error: linux/mman.h: No such file or directory
 #   include <linux/mman.h>
                           ^
compilation terminated.
CC="musl-gcc" /usr/bin/perl crypto/modes/asm/aesni-gcm-x86_64.pl elf crypto/modes/aesni-gcm-x86_64.s
Makefile:3816: recipe for target 'crypto/mem_sec.o' failed
make[1]: *** [crypto/mem_sec.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/tmp/openssl_build'
Makefile:171: recipe for target 'all' failed
make: *** [all] Error 2

The same build setup does work with 1.1.0. This is using the musl-tools package on Debian Stretch.

[sfackler]

The header is being pulled in for MLOCK_ONFAULT. MUSL does define that value, but it's in sys/mman.h rather than linux/mman.h (fun!).

[kroeckx]

/usr/include/linux/* are not part of libc, they are part of the kernel headers. We're doing a direct linux kernel system call, so we're using kernel headers. You should install the proper kernel headers for the OS your building for. Is seems that glibc now does have a wrapper for mlock2() since 2.27, and then also has things in <sys/mman.h>. I guess musl also has the wrapper based on your comment. We can add code to check the glibc version and use the glibc wrapper. We can add the same time to do same thing if you can say since which version musl supports that, and how to check that. We have 1 other syscall we do directly when there isn't a glibc wrapper, and that's getrandom(). Do you have any idea if musl supports that, and since which version?

[sfackler]

Ah, apparently this is due to some weirdness with Debian's musl-tools package: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764335. It doesn't include /usr/include/linux in the header search path.

MUSL does include wrappers for mlock2 and getrandom in the new 1.1.20 release:
https://git.musl-libc.org/cgit/musl/commit/include/sys/mman.h?id=b64d66d0b04fde0af63c3a292be423736294dca9
https://git.musl-libc.org/cgit/musl/commit/include/sys/random.h?id=e20658209177667e490c661dfd35b976749ef3f7

[mattcaswell]

These comments in that bug report are quite enlightening:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764335#17

Notably:

I do not consider this a bug of mus-gcc since it is simply not designed and
intended to compile more sophisticated programs.

and

If you want to compile more complicated programs you should look at the
musl-cross project: https://bitbucket.org/GregorR/musl-cross . After
installation you will end up with a full gcc system including linux headers
and binutils. It may sound strange at first to do cross compile for same
arch you are sitting on, but that's the way to go.

[mattcaswell]

A workaround for this issue to to change your configure line to this:

$ ./Configure --prefix=$OPENSSL_DIR linux-x86_64 no-shared no-async no-engine -DOPENSSL_NO_SECURE_MEMORY

Using the above the compile succeeds but at the expense of not having the secure memory feature. All the tests pass with the exception of the secmem test. That's probably an OpenSSL bug...i.e. not detecting in the test that we've disabled secure memory. I'm also not sure why OPENSSL_NO_SECURE_MEMORY isn't a proper Configure style "no-secure-memory" option.

Obviously it's not recommended to compile things this way (its clearly better to have the secure memory feature).

[sfackler]

Thanks for the workaround! It's obviously not ideal but it'll unblock builds at least for now.

This is a kind of weird issue since it's arguably not OpenSSL's problem, but it's still not great if the workflow to build a statically linked binary with OpenSSL goes from apt-get musl-tools to "build gcc, binutils, and elfutils from source".

[sanmai-NL]

If your build pipeline for Linux runs under the Arch Linux OS, you can install kernel-headers-musl. This works for me, and is better in that we keep the secure memory feature.

[jagu-sayan]

On alpine I do an apk add linux-headers

[alextremblay]

Does anyone know of a workaround to this issue for ubuntu?

I've tried installing linux-headers-generic, but I'm still getting the linux/mman.h: No such file or directory error, and I'm a bit lost here

Any help would be greatly appreciated

[richsalz]

simplest answer is remove sec_mem from the Makefile.

[alextremblay]

ahhh...

I'm trying to build openssl as a dependency for a rust project in a CI pipeline. Is there really no way to compile it on ubuntu without needing to modify the openssl source?

[richsalz]

Is there really no way to compile it on ubuntu without needing to modify the openssl source?

I didn't say that. I said this is a simple answer. I would quibble with saying "on Ubuntu" since you're using a different C library but shrug. Maybe someone else will have a more useful answer for you.

[alextremblay]

To be clear, I also tried adding the -DOPENSSL_NO_SECURE_MEMORY flag to my ./Configure step, but it had no effect :(

[richsalz]

That makes no sense, since all the #include statements are wrapped in an ifndef.

[alextremblay]

apologies, i spoke too soon. The -DOPENSSL_NO_SECURE_MEMORY did indeed fix the "fatal error: linux/mmap.h: No such file or directory" error on ubuntu.

What I'm seeing now is actually a different issue:

musl-gcc -fPIE -pie  -I. -Iinclude -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/musl/ssl\"" -DENGINESDIR="\"/musl/lib/engines-1.1\"" -DNDEBUG -DOPENSSL_NO_SECURE_MEMORY -MMD -MF engines/e_afalg.d.tmp -MT engines/e_afalg.o -c -o engines/e_afalg.o engines/e_afalg.c
engines/e_afalg.c:24:10: fatal error: linux/version.h: No such file or directory
   24 | #include <linux/version.h>
      |          ^~~~~~~~~~~~~~~~~

[alextremblay]

@richsalz Thank you for your help.

I apologize if my responses seem dismissive. I do appreciate your help, even if your answers are not what i want to hear hahaha

[alextremblay]

I found a workaround on ubuntu, without needing to disable secure memory!

I added -static -idirafter /usr/include/ -idirafter /usr/include/x86_64-linux-gnu/ to my musl-gcc command.

For reference, here's the complete set of steps I used to build openssl against musl in a github actions ubuntu CI runner:

sudo apt-get update
sudo apt-get install -y musl-dev musl-tools linux-headers-$(uname -r)

sudo ln -s /usr/include/x86_64-linux-gnu/asm /usr/include/x86_64-linux-musl/asm && ln -s /usr/include/asm-generic /usr/include/x86_64-linux-musl/asm-generic && ln -s /usr/include/linux /usr/include/x86_64-linux-musl/linux
sudo mkdir /musl
wget https://github.com/openssl/openssl/archive/OpenSSL_1_1_1f.tar.gz
tar zxf OpenSSL_1_1_1f.tar.gz 
cd openssl-OpenSSL_1_1_1f/
CC="musl-gcc -fPIE -pie -static -idirafter /usr/include/ -idirafter /usr/include/x86_64-linux-gnu/" ./Configure no-shared no-async --prefix=/musl --openssldir=/musl/ssl linux-x86_64
make depend
make -j$(nproc)
sudo make install

this compiled the openssl libraries and makes all the files available in the /musl directory.

To make user of it as a dependency on another build (ie the rust openssl-sys create), i added the following env vars to that build:

PKG_CONFIG_ALLOW_CROSS=1
OPENSSL_STATIC=true
OPENSSL_DIR=/musl

Hope someone else finds this useful

[t8m]

Special build targets are a new feature so -> Post 3.0.0. And of course PRs (against master branch) are welcome for this.

[jamuir]

@sfackler : can this issue be closed?