[FR] macOS & Apple Silicon Support

[jonpryor]

At WWDC 2020, Apple announced that macOS will be migrating to "Apple Silicon" over the next couple of years. While Rosetta 2 will exist as an immediate solution to allow the execution of existing x86_64 code on devices with Apple Silicon CPUs, the mid-to-long term plan should be to distribute all native binaries as Universal 2 binaries: executables which contain both x86_64 and Apple Silicon slices:

https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary

All native binaries in the Android SDK & NDK, such as clang and the linker, presumably need to be built as Universal 2 binaries.

Are there any plans to distribute the NDK as Universal 2 binaries?

Is your feature request related to a problem? Please describe.

Apple will be migrating macOS to a new hardware architecture. It would be nice if it were properly supported before x86 support is dead. (Admittedly not "soon"…)

Describe the solution you'd like

Ship NDK binaries as Universal 2 binaries, containing both x86_64 and Apple Silicon support.

Describe alternatives you've considered

Crying.

Components that need to be made fat

• M1 support in ndk-build script #1410
• M1 LLVM toolchain #1544
• M1 CMake #1545
• M1 Make #1546
• M1 Python #1547
• M1 Ninja #1548
• M1 yasm #1549
• M1 shader-tools #1577
• M1 simpleperf #1584

Won't fix:

• GDB
• GAS
• renderscript

[DanAlbert]

Thanks for filing. We've been planning for this since the rumors started being published, but hadn't filed a tracking bug.

We're working on getting some of the development devices. We also need to get buildbots that have a new enough xcode (which could possibly require a new version of macOS, I don't know). Some test infra work probably needs to be done, since I'm assuming that the new OS will be cross compiled for rather than natively compiled for.

After we have all the infrastructure we can finally start actually working on this. There may be some porting work, but for the most part we're shipping third-party software and will be dependent on those projects being ready.

In other words, there's a pretty long tail of work here, but we have started on it. There's still a non-zero chance that we'll need to rely on Rosetta at first. There is an outstanding question of whether or not Rosetta will support bare executables or just app bundles.

Crying.

Same.

[alexcohn]

The chances are pretty high that x86_64 NDK will run decently in Rosetta-2 (translation) mode even on the ARM hardware.

[jonpryor]

We can confirm that the x86_64 NDK tools that Xamarin.Android uses run on Rosetta 2 on the Apple DTK. I can't say how "well" they run, but they do run, and we're able to use Rosetta 2-translated SDK & NDK utilities to build & install a Xamarin.Android application.

[DanAlbert]

Great, thanks for that info. Just to check, they run outside your app bundle as well?

[jonpryor]

Great, thanks for that info. Just to check, they run outside your app bundle as well?

No, we re-package and redistribute bits of the Android NDK toolchain such that they are inside our .pkg, e.g.:

/Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/ndk/aarch64-linux-android-ld

I don't have the DTK machine, so I can't verify that they run outside of our default installation location. I don't see why they wouldn't…

[DanAlbert]

I would hope they do as well, but so far I haven't seen anything definitive about whether Rosetta will work for plain executables outside an app package. Our recent experience with notarization has me worried that it won't.

We should have our hands on the hardware soon enough and be able to find out though.

[enh-google]

going back to the original question:

Are there any plans to distribute the NDK as Universal 2 binaries?

we're still thinking about whether we'll have universal binaries or two separate NDKs for darwin/x86-64 and darwin/arm64, so if you have a strong argument why you'd rather have universal binaries than a separate NDK, let us know while we/Studio are still trying to decide! we don't have any timeline to announce yet, but we have started work on building everything for darwin/arm64. (and in the meantime, everything works with Rosetta2.)

[alexcohn]

If you ask me, I'd prefer the smaller separate packages. These days, we mostly download NDK via SDK Manager, and it should know well which flavor of NDK is correct for the given machine.

BTW, same applies to Windows and Linux.

[mylylyl]

(and in the meantime, everything works with Rosetta2.)

How do you run ndk using rosetta2?

[DanAlbert]

Nothing special I don't think. The whole point of Rosetta is to allow binaries to run as if nothing were different.

[enh-google]

yeah, that's my understanding too (and how it was for "classic" Rosetta back in the PowerPC transition days). @kongy has actually tried it...

[osrl]

hi, I get this error while trying to build a project with ndk

Error while executing process /Users/osrl/Library/Android/sdk/ndk/21.3.6528147/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Android.mk NDK_APPLICATION_MK=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Application.mk APP_ABI=armeabi-v7a NDK_ALL_ABIS=armeabi-v7a NDK_DEBUG=1 APP_PLATFORM=android-18 NDK_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/obj NDK_LIBS_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n}
ERROR: Unknown host CPU architecture: arm64

I also tried to build with arch -x86_64 but no luck. How does "everything work with Rosetta2"? Can you help @enh-google?

[osrl]

I solved it by editing ndk-build script:

#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"

After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

[DanAlbert]

Presumably you're getting your system's make and not the NDK's when you make that change. If you're using an older make there are going to be some broken parts of ndk-build.

[enh-google]

(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)

[azureplus]

I solved it by editing ndk-build script:

#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"

After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

Thanks

[jgdatsiemens]

(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)

My macOS 11.2.1 machine with Xcode 12.2 still has GNU make 3.81.

[jgdatsiemens]

Do you have Rosetta 2 installed? That will be needed to run the compiler, which is an Intel binary. Rosetta 2 isn't always installed automatically. John

…

On Fri, Apr 9, 2021 at 7:14 AM H3c ***@***.***> wrote: how to build use cmake??? i get some error on M1 C/C++: /Users/xxx/Documents/THAILib/app/CMakeLists.txt debug|armeabi-v7a : CMake Error at /Users/xxx/Library/Android/sdk/cmake/3.18.1/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake:66 (message): The C compiler "/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang" is not able to compile a simple test program. It fails with the following output: Change Dir: /Users/xxx/Documents/THAILib/app/.cxx/Debug/3t4w4qs6/armeabi-v7a/CMakeFiles/CMakeTmp Run Build Command(s):/Users/xxx/Library/Android/sdk/cmake/3.18.1/bin/ninja cmTC_ef1bc && [1/2] Building C object CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o FAILED: CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o /Users/xxx/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --gcc-toolchain=/Users/h3c/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -fPIE -MD -MT CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -c testCCompiler.c ninja: build stopped: subcommand failed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1299 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATCLA5X3RCLVXMCOIX25UKLTH2LNPANCNFSM4OJWV7QA> .

[myflyingfish]

I solved it by editing ndk-build script:

#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"

After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@

Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?

same problem

[DanAlbert]

I think #1410 is the thread you want. The fix is in the next release, and https://android-review.googlesource.com/c/platform/ndk/+/1855814 was the fix (in the other thread, but buried in the middle).

[hannesa2]

The fix is in the next release

@DanAlbert Do you know what the next is ? At least it's not 23.1.7779620

[enh-google]

The fix is in the next release

@DanAlbert Do you know what he next is ? At least it's not 23.1.7779620

you can see on any given bug... if you look at #1410, say, you can see on the right-hand side that it's currently targeted at r23c and r24.

for dates for any given release, see https://github.com/android/ndk/wiki#release-schedule

[DanAlbert]

Most of the bugs are still open because they aren't fixed in every supported release. The latest r24 beta has nearly complete M1 support. r23b shipped M1 support for the toolchain, and r23c will include the same level of M1 support that's currently in r24. I'm hoping to release r23b in January but we're still waiting for toolchain fixes before that ships.

The only things that aren't fixed in r24 are yasm and simpleperf. Yasm I suspect no one actually uses so isn't a high priority, and simpleperf is out of our hands.

The big remaining components are CMake and Ninja, which are not a part of the NDK so are not tied to any release. New releases for those are being tested as I write this, but there are some infrastructural questions we need to answer about the release process before they ship (we have not been the team that released those in the past, and there is always something to learn during a handoff).

[enh-google]

The only things that aren't fixed in r24 are yasm and simpleperf. Yasm I suspect no one actually uses so isn't a high priority, and simpleperf is out of our hands.

(by which he means "out of the NDK team's hands" ... obviously simpleperf is "ours" in the wider sense of "Android's" too, and the relevant people are working on that. but simpleperf comes from different infrastructure again from any of the other things mentioned here, so that's a third team that has work to do because of this.)

[DanAlbert]

(Don't read too much into the move from r24 from r24b. Progress was made in r24, the tracking bug is just always going to be moving forward until this is 100%)

[DanAlbert]

Please take the ndk-build discussion to the appropriate thread: #1410

[rprichard]

There is a new CMake 3.22.1 package in the SDK Manager's canary channel that has M1 support for macOS.

With this release, and with the latest platform-tools and build-tools packages, it's now possible to build and debug an Android app with a CMake project, on an M1 Mac without Rosetta (but with beta/canary packages). These versions of components are needed:

• Android Studio 2021.1.1
• Android Gradle Plugin 7.1.0 (the plugin includes an aapt2 binary)
• Android SDK Build-Tools 32.1-rc1 (SDK's stable channel)
• Android SDK Platform Tools 32.0.0 (SDK's stable channel)
• Android NDK 24.0.7956693 (beta 2, from the SDK's beta channel)
• CMake 3.22.1 (SDK's canary channel)

Android Studio can be briefly configured to use the canary channel to install just CMake, then switched back to the stable channel, to avoid updating everything else.

The NDK and CMake versions need to be configured in build.gradle, e.g.:

android {
    ndkVersion "24.0.7956693-beta2"
    externalNativeBuild {
        cmake {
            path file('src/main/cpp/CMakeLists.txt')
            version '3.22.1'
        }
    }
}

In NDK r24 beta 2, yasm and simpleperf are still x86_64-only.

Using ndk-build on a non-Rosetta M1 Mac needs a newer NDK r24 build, which has a fix to the makefiles to use a universal Python3 binary. The NDK r24 final release should have this fix.

[DanAlbert]

Locking the thread to encourage folks to use the sub-topics for each component. We'll use this one for announcements about M1 support in general. Please use the appropriate sub-topic (see the list of all components in the OP) if you have questions about M1 support for a specific component, or file a new bug if there's a component we've missed.

[DanAlbert]

As before, don't worry about the fact that this issue was removed from r23c. r23c still contains improvements for M1, but not all the cherry-picks are safe to backport so it's being moved because r23c won't get all the fixes. The only remaining NDK components in r23c that will not be universal binaries are:

• yasm
• simpleperf
• GAS (won't fix)
• GDB (won't fix)
• renderscript (won't fix)

As noted in the main post (though I'd previously forgotten to mention renderscript) the "won't fix" components won't be fixed because they're gone in r24+ anyway.

[DanAlbert]

Same as the last update: just moving things forward as releases move forward. Only affected NDK components in r25 are simpleperf and yasm. Will consider backports to r25 for each once we have fixes available, but for now triaging as r26. r25 is already complete for the vast majority of workflows.

[DanAlbert]

This is now done enough that if you're on up to date tools you likely won't notice the one remaining piece (yasm). Yasm won't be landing for a while since the next major release is next year. If we've overlooked something that makes yasm way more important we believe it is, please chime in on #1549.

[DanAlbert]

Closing this since as far as we can tell this is as done as anyone needs it to be. If you're actually using yasm and need that fixed, please chime in on #1549. As far as we know, that tool is completely unused, so it's not a priority to fix that last remaining non-universal binary.

If you find other binaries that aren't universal, please file new bugs.