Building for macOS and Ubuntu arm64

[ronaldtse]

There are now two target macOS architectures (arm64 vs x86), so we want to:

1. Target building both on GHA
2. Offer the arm64 executable as an uploaded job artifact, so that you can also test the GHA-compiled arm64 one locally

[ronaldtse]

From @maxirmx :

I have 11.00 x86 version working.
I think I will do 10.15 today. It is a matter of selecting XCode version and SDK on GHA runner. It is there but I cannot figure how to switch to it.

As for arm version there is no arm runner on GHA
It means that in order to support it we need to build x86 dwarfs tools, after that cross-build arm version of "everything" and after that use x86 tools to package arm version.

It does not look too complex unless two caveats:

1. I have some doubts about INCBIN capability to support cross-compilation. So it has to be tested
2. The CI procedure that I described above won't provide for an option to test the output or any intermmediate steps. It will be very hard to debug.

I have the package running locally on arm, so it is purely CI issue.

If I have to build a workflow as described (build and package arm version on x86 GHA runner) I think it will take a week.

[ronaldtse]

I think I will do 10.15 today. It is a matter of selecting XCode version and SDK on GHA runner. It is there but I cannot figure how to switch to it.

Available SDK versions on 10.15:

• https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md#installed-sdks

GH action to select version:

• https://github.com/marketplace/actions/setup-xcode-version

2. The CI procedure that I described above won't provide for an option to test the output or any intermmediate steps. It will be very hard to debug.

Testing macOS with arm64 won't work on GHA, so we can download it and test...

[maxirmx]

Let it be arm64 architecture and #54 - x86_64 architecture

[maxirmx]

It looks like this type of cross-compilation is not supprted by homebrew in a sense that there is no way to install arm64 libraries on x86_64 MacOS 11

Practiaclly it means cross-compilation of all libraries (3-4 hours per build on GHA) or some custom brew formula

Maybe someone is able to help:
https://stackoverflow.com/questions/70821136/can-i-install-arm64-libraries-on-x86-64-with-homebrew
Homebrew/discussions#2843

[ronaldtse]

@maxirmx wow, you already got your answer! That was fast.

[maxirmx]

Dwarfs uses thrift which is essentially a compiler. So build script (1) builds thrift, then (2) uses it to create filesytem metadata definitions, then (3) builds a library that processes metadata definition reusing some of the object files created at step (1). In the case of arm64 build on x86_64 steps (1) and (2) need to be x86 while step (3) needs to be arm64

Similar ruby build script (1) builds miniruby (2) uses miniruby to generate some extra files (3) builds full ruby using some of the object files created at step (1)
In the case of arm64 build on x86_64 steps (1) and (2) need to be x86 while step (3) needs to be arm64

It is possible to create build scripts both for dwarfs and for ruby as described above. For dwarfs complexity is low, for Ruby complexity is high. Howeevr we need to accept the fact that our build scripts will diverge from baselines in both cases.

[maxirmx]

If I attempt to package something bigger then trivial example I am getting error

<jemalloc>: Unsupported system page size

Something like: jemalloc/jemalloc#467
The solution is to specify jemalloc low-level parameters during cross-compilation

We have to use jemalloc since otherwise dwarfs runs out of memory when metanorma is packaged
We have to build jemalloc on MacOS since homebrew version does not link statically

[maxirmx]

We are facing similar issue
rubygems/rubygems#4269

[ronaldtse]

We have to use jemalloc since otherwise dwarfs runs out of memory when metanorma is packaged
We have to build jemalloc on MacOS since homebrew version does not link statically

Sound reasonable.

[ronaldtse]

We are facing similar issue
rubygems/rubygems#4269

Is this in packed-mn or in another implementation? The way to resolve it seems to be to add the platforms to Gemfile.lock or having platform specific Gemfile.lock s.

bundle lock --add-platform ruby
bundle lock --add-platform x86_64-linux 
bundle lock --add-platform arm64-darwin-20

[maxirmx]

It looks like this feature is broken in bundler 2.3.2 and by unlucky coincedence this is the version I am testing with.
'bundle lock' with 2.3.2 works correctly with single gem but fails to process complex dependencies like metanorma has
I will test with the latest bundler. I think it shall fix the isse

[maxirmx]

When it comes to native extensions the only (partially ?) working approach is to build several versions of ruby and combine cross-compilation environment as described here:

https://ruby-talk.ruby-lang.narkive.com/6yJeTNdl/rbconfig-rb-and-destdir-confusion-while-cross-compiling-ruby-gems

However, it requires extconf.rb for all Gems to honour cross-compile settings. That did not come true even for the first one: ffi/ffi#944

[ronaldtse]

@maxirmx the issue in ffi gem seems to have been solved:

• Cross-compiling on MacOS x86_64 for MacOS arm64 fails  ffi/ffi#944 (comment) points to Move /opt before /usr to search for M1 native libs first - solve the conflicts when both x64 and arm64 libs installed ffi/ffi#936
• Move /opt before /usr to search for M1 native libs first - solve the conflicts when both x64 and arm64 libs installed ffi/ffi#936 was closed and is superseded by prioritize /opt/homebrew/lib (for macos on aarch64 only) for searching path ffi/ffi#965
• prioritize /opt/homebrew/lib (for macos on aarch64 only) for searching path ffi/ffi#965 was closed and is superseded by Refactor library lookup ffi/ffi#968
• Refactor library lookup ffi/ffi#968 was merged

Can you revisit this, and close ffi/ffi#944 when appropriate? Thanks.

[maxirmx]

@ronaldtse
I think cross-compilation would be too complex anyway.
Cirrus CI say that they have MacOS Ventura arm64 runners. If you do not mind trying it could you please enable Cirrus CI for this repository, libdwarfs and dwarfs
It now references other organization -- metanorma/tebako; metanorma/dwarfs

Thank you

[ronaldtse]

@maxirmx I have installed CIrrus CI now for this org. Thanks.