ERL-1428: OTP binary releases for macOS

[OTP-Maintainer]

Original reporter: wojtekmach
Affected version: Not Specified
Component: Not Specified
Migrated from: https://bugs.erlang.org/browse/ERL-1428

---

On the last EEF build & package WG meeting we briefly discussed automatically building macOS binary releases (and in the future possibly other ones too like for popular linux distros) and so I'd like to formally propose that as a new feature.

Here are example use cases for such feature:
- https://github.com/actions/setup-elixir/issues/23
- https://github.com/asdf-vm/asdf-erlang/issues/165

And here's a proof-of-concept: https://github.com/wojtekmach/otp_builds. I wrote some notes in the README. In particular please notice that the action https://github.com/wojtekmach/otp_builds/runs/1479456475?check_suite_focus=true automatically uploaded the tarball to the release: https://github.com/wojtekmach/otp_builds/releases/tag/OTP-23.1.4.

The OTP GitHub setup is quite different, but hopefully the prototype proves that adding such feature is feasible at a first glance. I hit an issue with statically linking openssl (https://bugs.erlang.org/browse/ERL-1408?focusedCommentId=20003&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-20003) but hopefully that can be resolved soon.

Please let me know if that's something you'd be interested in and I'll be happy to work on incorporating this into the erlang/otp repo (any feedback how best to do it would be very appreciated too).

[wojtekmach]

A few notes:

• Link to the PR: Automatically build binary releases #2936

• Some more of my further experiments might be of interest. https://github.com/wojtekmach/beamup explored a bunch of angles of building, hosting, and installing OTP. One of them was automatically syncing OTP 23.2.6+ from https://github.com/erlang/otp/releases, building it, and publishing the artefacts for Linux & macOS at https://github.com/wojtekmach/otp_releases/releases.

• There is an open PR (thanks @seivan!) which makes asdf-erlang on macOS first try download binary builds from Homebrew (https://docs.brew.sh/Bottles, https://formulae.brew.sh/formula/erlang) before compiling it from source.

[wojtekmach]

Just some notes around openssl on macOS so that they don't get lost: (some of them are already mentioned in the linked issues and prs)

While macOS ships with openssl (libressl) it is recommended not to depend on it. See: https://rentzsch.tumblr.com/post/33696323211/wherein-i-write-apples-technote-about-openssl-on

Seems dynamically linking openssl (or any other library for that matter) is tricky on macOS as of a few versions ago.

From dyld(1):

NAME
       dyld - the dynamic linker

(...)

DESCRIPTION
       The  dynamic  linker  checks  the  following environment variables during the launch of each
       process.
       Note: If System Integrity Protection is enabled, these  environment  variables  are  ignored
       when executing binaries protected by System Integrity Protection.

this can be easily verified as follows:

% LD_LIBRARY_PATH="foo" erl -noshell -eval 'io:format("~p~n", [os:getenv("LD_LIBRARY_PATH")]), halt()'
false

So it seems statically linking openssl on macOS is the only viable option. Alternatively, and it seems that's what homebrew is doing, we could have some kind of post-install script that would rewrite library paths in the .so file, not sure how reliable this is. That's an approach that was explored in asdf-vm/asdf-erlang#190.

[seivan]

@wojtekmach That's what I am doing for my "language cvs". I can't speak for its reliability: expect "it works for me" but I just want to point out that it's 100% what Homebrew is doing for Erlang themselves.

Looking at their code (Ruby) they do this for almost everything, even including string templating for various paths because different systems (M1, x64, etc). Doing it for Ruby was a pita but still possible.
Erlang was much much easier - thanks to your help!

Not saying what they are doing is bullet proof, but since I've been using Homebrew for this in the past I never had a problem, so adopting their approach was good enough for me.

Honestly, just relying on Homebrew for their binaries is good enough in my opinion, the downside is you're always lagging behind and it lacks Linux options - but the amount of manpower there is quite significant.