Generate a Makefile with "Universal Binary" (e.g. arm64, x86_64) directives?

[systemresearch]

Can ExtUtils::MakeMaker be used to generate a Makefile with macOS "Universal Binary" architecture directives (e.g. arm64, x86_64) for building a Perl module *.bundle? If yes, then how? If not, could be this capability be added to the enhance|fix queue?

The general use case is for Perl on Apple hardware which will execute Perl module binaries with either the native arm64|arm64e processor or the Rosetta2 x86_64 binary translator.

In my particular case, I have run into significant blocking conflicts when working with Perl components of open source applications such as MacTeX, GnuCash, and LibreOffice. See: "StackOverflow: Install & update a Perl module as "universal" (x86_64, arm64)?"

Some investigation found inconsistencies in the Mach-O .bundle architecture of Perl modules on the same arm-based computer. For example:

file /Library/Perl/5.30/darwin-thread-multi-2level/auto/Date/Simple/Simple.bundle
# /Library/Perl/5.30/darwin-thread-multi-2level/auto/Date/Simple/Simple.bundle: 
#     Mach-O 64-bit bundle arm64

file /Library/Perl/5.30/darwin-thread-multi-2level/auto/Encode/Encode.bundle
# /Library/Perl/5.30/darwin-thread-multi-2level/auto/Encode/Encode.bundle: 
#     Mach-O 64-bit bundle x86_64

file /System/Library/Perl/5.30/darwin-thread-multi-2level/auto/Encode/Encode.bundle
# /System/Library/Perl/5.30/darwin-thread-multi-2level/auto/Encode/Encode.bundle: 
#     Mach-O universal binary with 2 architectures: 
#         [x86_64:Mach-O 64-bit bundle x86_64] 
#         [arm64e:Mach-O 64-bit bundle arm64e]

Apple's instructions for building a macOS "Universal Binary" show how to "Update the Architecture List of Custom Makefiles".

x86_app: main.c
    $(CC) main.c -o x86_app -target x86_64-apple-macos10.12
arm_app: main.c
    $(CC) main.c -o arm_app -target arm64-apple-macos11
universal_app: x86_app arm_app
    lipo -create -output universal_app x86_app arm_app

Note: lipo can "create or operate on a universal file: convert a universal binary to a single architecture file, or vice versa."

It would be helpful if ExtUtils::MakeMaker could support some KEY=VALUE command line solution like the following:

perl Makefile.PL UNIVERSAL_BINARY="arm64,arm64e,x86_64"

[Leont]

Can ExtUtils::MakeMaker be used to generate a Makefile with macOS "Universal Binary" architecture directives (e.g. arm64, x86_64) for building a Perl module *.bundle?

Apple seems to do it, but I've never seen a description of how they did it.

If not, could be this capability be added to the enhance|fix queue?

The lipo route looks like it would require a significant amount of work because we'd no longer be able to use the generic unix logic for this but instead have to reimplement a bunch of things. There are also some backwards compatibility risks to that.

If you can make -arch x86_64 -arch arm64 work to compile/link both architectures at the same time, that would be a much easier approach.

[systemresearch]

If you can make -arch x86_64 -arch arm64 work to compile/link both architectures at the same time, that would be a much easier approach.

Indeed.

env ARCHFLAGS='-arch arm64 -arch arm64e -arch x86_64' perl Makefile.PL

make

arch -x86_64 make test
arch -arm64  make test
arch -arm64e make test

(sudo) make install

check

file Encode.bundle
# Encode.bundle: Mach-O universal binary with 3 architectures: 
#     [x86_64:Mach-O 64-bit bundle x86_64] 
#     [arm64:Mach-O 64-bit bundle arm64] 
#     [arm64e:Mach-O 64-bit bundle arm64e]

This approach was finally found in the historicman perlmacosx page (that i had forgotten about), and, which still happens to be on the current macOS:

Related to this support is the new environment variable ARCHFLAGS, … With ARCHFLAGS, this can be changed to whatever architectures the user wants to build. For example:

% env ARCHFLAGS='-arch i386 -arch x86_64' perl Makefile.PL
% make
% make install

will build only 2-way universal.

---

The lipo route looks like it would require a significant amount of work…

It now appears that "what's broken|missing" is simply some good paths for finding the information. Perhaps the following would be reasonable to do? Your thoughts?

1. add a brief hint to the ExtUtils::MakeMaker Tutorial "The Mantra"

   # env ARCHFLAGS='-arch arm64 -arch arm64e -arch x86_64' perl Makefile.PL # Universal Binary
   perl Makefile.PL
   make
   make test
   make install

2. add a "How do I install a universal binary on macOS?" section to the ExtUtils::MakeMaker FAQ

[Leont]

What does perl -V:ccflags -V:ldflags -V:lddlflags return?

Ideally, you would put them in each of those variables when building perl, then you don't have to worry about it later.

[systemresearch]

which -a perl
# /usr/bin/perl

perl -V:ccflags -V:ldflags -V:lddlflags
# ccflags=' -g -pipe -DPERL_USE_SAFE_PUTENV';
# ldflags=' ';
# lddlflags=' -bundle -undefined dynamic_lookup';

file /usr/bin/perl
# /usr/bin/perl: Mach-O universal binary with 2 architectures: 
#     [x86_64:Mach-O 64-bit executable x86_64] 
#     [arm64e:Mach-O 64-bit executable arm64e]

Ideally, you would put them in each of those variables when building perl…

In this case, it's the macOS system perl built and shipped by Apple.

[systemresearch]

Another point of reference (although not the Perl that I have been using) is Homebrew Pearl. I installed Homebrew Perl on a macOS 13.3 computer and checked the flags:

/opt/homebrew/Cellar/perl/5.36.1/perl -V:ccflags -V:ldflags -V:lddlflags 
# ccflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=13.3 -fno-strict-aliasing -pipe -fstack-protector-strong -DPERL_USE_SAFE_PUTENV';
# ldflags=' -mmacosx-version-min=13.3 -fstack-protector-strong';
# lddlflags=' -mmacosx-version-min=13.3 -bundle -undefined dynamic_lookup -fstack-protector-strong';

What is common between the Apple Perl and the Homebrew Perl is -bundle in lddlflags. The -bundle flag directs the output to be "a mach-o bundle that has file type MH_BUNDLE".

[Leont]

The -bundle flag directs the output to be "a mach-o bundle that has file type MH_BUNDLE".

That marks the output as a loadable library, it's not related to making universal binaries.

[systemresearch]

The -bundle flag directs the output to be "a mach-o bundle that has file type MH_BUNDLE".

That marks the output as a loadable library, it's not related to making universal binaries.

A relationship would be that a mach-o bundle (*.bundle) is a loadable library of a particular format which can contain more than one ISA binary slices of the same source build.

In some sense, -bundle is a piece of a universal binary creation process even if it is not a full specification of which ISA binaries to build and include.

That said, I have no idea why Apple and Homebrew do not include more build flags in perl.

In the case of Homebrew perl, Homebrew is now available on Linux, Windows Subsystem for Linux in addition to macOS. Homebrew perl analytics indicates over 300K installs on macOS and 70K installs on Linux in the last 365 days.