Automatically build binary releases

[wojtekmach]

https://bugs.erlang.org/browse/ERL-1428

Notes:

• First of all, this workflow file is isolated (it just uses init-pre-release.sh for now) and there’s more work to be done to re-use existing scripts

• As a proof of concept, the workflow currently only builds on pushes to the “master” branch and the artifacts are attached to the action run, see e.g.: https://github.com/wojtekmach/otp/actions/runs/428356220. Instead of building on every push to master, there could be instead a “nightly” job for master. If these master/nightly/etc artifacts would prove useful, instead of just being attached to the job, should they have a more permanent home e.g. on erlang.org? Btw, looks like run artifacts are only accessible to users who have access to the repo, so I re-published the tarballs that were built in that action here: http://wojtekmach.pl/otp/. In particular, you can easily test it with Docker by running: docker run -it --rm ubuntu:20.04 sh -c 'apt update && apt install -y curl && curl http://wojtekmach.pl/otp/master-x86_64-ubuntu-20.04.tar.gz | tar xz && cd release/ && ./Install -sasl $PWD && ./bin/erl'

• Beyond this proof of concept that just builds master branch, and this is really the point of all this work, there will be a job that works on releases (tbd how it’s triggered: either on tag push, or on release creation, or by polling, etc) that will attach the artifacts to the release, this way the artifacts will be available under predictable and public URLs like: https://github.com/erlang/otp/releases/download/OTP-24.0/OTP-24.0-x86_64-macos-10.15.tar.gz

• When downloading the run artifacts on macOS using Safari, the tarball is put under quarantine which means you won’t be able run executables unless you allow it in security settings. You can work around it by running this before unpacking the tarball: xattr -d com.apple.quarantine path/to/tar. When we have tarballs attached to releases, they could be downloaded via curl (since we can guess the URL) and that way they won’t be quarantined. Thus, when tools like asdf-erlang/kerl/etc will use binary releases, they’ll simply use curl to fetch and it will just work. I’m not sure people would download release from GitHub via browser, after unpacking you still need to run the ./Install script, so maybe the curl-based flow is enough. Otherwise, it would probably make sense to build (and sign) a gui installer.

• I’ve arbitrarily picked the naming scheme to be: <ref>-<arch>-<os>-<osversion>.tar.gz,

  • master-x86_64-alpine-3.12.tar.gz
  • master-x86_64-macos-10.15.tar.gz
  • master-x86_64-ubuntu-20.04.tar.gz

  Does that seem reasonable? Not sure if it should be more specific (e.g. instead darwin-19.6.0 for macOS Catalina)

• The tarball unpacks a release/ directory (release/Install, release/lib, etc), should it unpack to a directory with a more specific name? or simply to “.”?

• macOS ships with libressl but without header files and it's recommended to bring your own SSL library anyway so we statically link it. See: https://rentzsch.tumblr.com/post/33696323211/wherein-i-write-apples-technote-about-openssl-on.

• No consideration for wx yet

[garazdawi]

Hello, we will look at this and see what we want to do.

A question that pops into my mind to get the discussion going is whether this PR should build the packages for the respective OSs? i.e. .deb, .rpm etc etc. I don't know much about package distribution, but isn't that the way you would want to do this?

[wojtekmach]

i.e. .deb, .rpm etc etc.

I think that's a good idea, this would remove the manual step of unpacking and running ./Install script.

[garazdawi]

I don't think you can use those packages to speed up the workflow used by kerl (and in extension asdf and many other tools), which I think would be very good to do.

Another question is testing that the artifacts work. For windows we use the exact same build in our internal tests to make sure that they work. We would need something similar here. Things that are not tested do not work.

I would suggest that the packages are part of the github release build that is done today and attached to releases in the same way. We could also use github packages for the releases, which may make it easier to do nightly build if we want that.

(Just to be clear again, we have not decided whether we want to do this or not yet. Just noting down my thoughts on the subject.)

[wojtekmach]

I'm going to go ahead and close this PR for now as I mistakenly created it off my master branch that I'd like to get some other, unrelated, commits on. The copy of the branch used for this PR is here https://github.com/wojtekmach/otp/tree/wm-master-backup though it is obviously already quite stale.

If you make a decision whether to move forward with it, please let me know here, on #4487, or however is most convenient to you :) I'm happy to put more work even if you ultimately decide not to move forward with it. If I can help in any way in the meantime, I'm happy to do that too. Thanks for your consideration!