[macOS Sonoma] cannot compile 26.2.*: undefined symbols _ei_MD5Final _ei_MD5Init _ei_MD5Update

[paulswartz]

Describe the bug
Compiling OTP 26.2 on macOS Sonoma (14.2.1 on an M1 Pro) fails with:

ld: Undefined symbols:
  _ei_MD5Final, referenced from:
      _gen_challenge in libei.a[2](ei_connect.o)
      _recv_challenge_reply in libei.a[2](ei_connect.o)
      _gen_digest in libei.a[2](ei_connect.o)
      _recv_challenge_ack in libei.a[2](ei_connect.o)
  _ei_MD5Init, referenced from:
      _gen_challenge in libei.a[2](ei_connect.o)
      _recv_challenge_reply in libei.a[2](ei_connect.o)
      _gen_digest in libei.a[2](ei_connect.o)
      _recv_challenge_ack in libei.a[2](ei_connect.o)
  _ei_MD5Update, referenced from:
      _gen_challenge in libei.a[2](ei_connect.o)
      _recv_challenge_reply in libei.a[2](ei_connect.o)
      _recv_challenge_reply in libei.a[2](ei_connect.o)
      _gen_digest in libei.a[2](ei_connect.o)
      _gen_digest in libei.a[2](ei_connect.o)
      _recv_challenge_ack in libei.a[2](ei_connect.o)
      _recv_challenge_ack in libei.a[2](ei_connect.o)
      ...
clang: error: linker command failed with exit code 1 (use -v to see invocation)

To Reproduce

> brew install make openssl@3 zlib
> export LDFLAGS="-L/opt/homebrew/Cellar/zlib/1.3/lib -L/opt/homebrew/Cellar/openssl@3/3.2.0_1/lib"
> export CPPFLAGS="-I/opt/homebrew/Cellar/zlib/1.3/include -I/opt/homebrew/Cellar/openssl@3/3.2.0_1/include"
> git checkout OTP-26.2
> ./configure --disable-jit
> gmake clean
> gmake erl_interface

Expected behavior
The build to succeed.

Affected versions

• 26.2
• 26.2.1
• 25.3.2.8

Good versions:

• 26.1.2
• 25.3.2.7

Additional context
I ran git bisect OTP-26.2 OTP-26.1.2 and it looks like this was introduced in 609a562.

609a562ff60e5456739dfd62bb395b9083a78cba is the first bad commit
commit 609a562ff60e5456739dfd62bb395b9083a78cba
Author: Rickard Green <rickard@erlang.org>
Date:   Thu Nov 23 18:32:35 2023 +0100

    [erts, erl_interface] Replace MD5 implementation used

    Replaces the old MD5 implementations used in OTP with the one used in
    OpenSSL 3.1.4

[paulswartz]

Full build log:
26.2-build-log-md5.txt

[voluntas]

@paulswartz I had the same problem, but I used my own build of OpenSSL and it built fine. I hope this helps.

OpenSSL 3.2.0

curl -LO https://www.openssl.org/source/openssl-3.2.0.tar.gz
tar xvfz openssl-3.2.0.tar.gz
cd openssl-3.2.0
./Configure --prefix=/opt/openssl/3.2.0 no-shared no-module

OTP 26.2.1

./configure --prefix=/opt/erlang/26.2 \
            --enable-kernel-poll \
            --enable-dirty-schedulers \
            --enable-jit \
            --disable-sctp \
            --disable-dynamic-ssl-lib \
            --disable-sharing-preserving \
            --with-ssl=/opt/openssl/3.2.0 \
            --without-javac \
            --without-odbc

[paulswartz]

Thanks for this pointer! It's helped me narrow it down a bit to the -I<path to OpenSSL includes bit in CPPFLAGS. It's looking that lines like 609a562#diff-906bc8036acbdcf12ce6254267a102a69e0f0d15c15614b3dc111682fadba06aR13

#include "openssl/md5.h"

are confused between the local version in src/erl_interface and the actual version from OpenSSL.

#7989 has a fix which works for me, at least.

[krant]

Same issue when building 26.2.1 for OpenWrt:

/openwrt/staging_dir/host/bin/gcc -fno-common -Werror=return-type -O2 -I/openwrt/staging_dir/host/include -I/openwrt/staging_dir/hostpkg/include -I/openwrt/staging_dir/target-arm_cortex-a9+neon_musl_eabi/host/include -DEI_64BIT -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -DEI_NO_DEPR_WARN -I. -I../include -Iconnect -Iencode -Idecode -Imisc -Iepmd -Iopenssl/include -Ix86_64-pc-linux-gnu   -Iglobal -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -D_POSIX_THREAD_SAFE_FUNCTIONS -L/openwrt/staging_dir/host/lib -L/openwrt/staging_dir/hostpkg/lib -L/openwrt/staging_dir/target-arm_cortex-a9+neon_musl_eabi/host/lib -o /openwrt/build_dir/hostpkg/otp_src_26.2.1/lib/erl_interface/bin/x86_64-pc-linux-gnu/erl_call prog/erl_call.c \
	-L/openwrt/build_dir/hostpkg/otp_src_26.2.1/lib/erl_interface/obj/x86_64-pc-linux-gnu -lei -lpthread
/usr/bin/ld: /openwrt/build_dir/hostpkg/otp_src_26.2.1/lib/erl_interface/obj/x86_64-pc-linux-gnu/libei.a(ei_connect.o): in function `gen_digest':
ei_connect.c:(.text+0x250): undefined reference to `ei_MD5Init'
/usr/bin/ld: ei_connect.c:(.text+0x265): undefined reference to `ei_MD5Update'
/usr/bin/ld: ei_connect.c:(.text+0x27b): undefined reference to `ei_MD5Update'
/usr/bin/ld: ei_connect.c:(.text+0x286): undefined reference to `ei_MD5Final'
/usr/bin/ld: /openwrt/build_dir/hostpkg/otp_src_26.2.1/lib/erl_interface/obj/x86_64-pc-linux-gnu/libei.a(ei_connect.o): in function `gen_challenge':
ei_connect.c:(.text+0x33a): undefined reference to `ei_MD5Init'
/usr/bin/ld: ei_connect.c:(.text+0x34a): undefined reference to `ei_MD5Update'
/usr/bin/ld: ei_connect.c:(.text+0x355): undefined reference to `ei_MD5Final'
collect2: error: ld returned 1 exit status

[garazdawi]

Fix merged to be part of next OTP-26 patch.