ARM compiler option for Swift

[kconner]

I'd love to use this tool to compare compiler output between x86_64 and arm64 for Swift. I think it'd be very instructive to show my team the features our target devices have, like conditional branching instructions. At the moment, there is only one Swift compiler option, x86_64 3.1.1.

[AbrilRBS]

Hello!
Some days ago I saw we were indeed laking proper support for Swift (We have 0 examples for it, only 1 compiler, etc...) and I'm on the middle of trying to improve this.
But I can only do so much at the time, and I'm currently focused on some other issues with more priority. So give me a week or so (Or maybe someone else does this before me, and that also would be great!)

So, what versions would you think would be needed?

:)

[kconner]

Thanks very much!

I think my wish list would be:

• For ARM, the current production Swift version that ships with Xcode, for arm64.
• For x86, the latest stable Swift version available to Linux from swift.org.

Both of these are version 4.0. The latest version of Swift 3.1 would also be nice, and possibly also Swift 3.2 which uses the Swift 4 compiler but builds Swift 3's ABI. But I think their utility will eventually expire, so 4 is most important.

ARM support is probably higher priority since it would serve iOS, tvOS, and watchOS developers. x86_64 would be useful primarily to Mac developers, Swift-on-server devs, and Swift OSS language developers. That is a smaller pool, but I bet a few compiler devs would really appreciate it.

[AbrilRBS]

Ey! So sorry for keeping you waiting for so long, we've been busy over here.

I have been able to add x86 Swift 4.0.1 (Which should be now available on the live site, yay!) , but I have some problems with the rest. Hopefuly you can shed some light to it.
The ARM binaries are nowhere to be found in a place where it's easy to grab them (Would love some help with that) and I have not been able to find version 3.2 anywhere either. The oficial Apple page goes form 3.1.1 to 4

Please let me know if you can help with any of those issues.

Thanks again for your patience :)

[mattgodbolt]

Some suggestions received over email:

• Run under QEMU
• use -target like we do for other clang-ish compilers...and then to fix the issues pass -sdk and give it the path to the arm installation
• run on macOS (not really a starter because...well...Apple)

[wadetregaskis]

From researching this, the root problem (in a sense) appears to be that Linux doesn't support fat binaries (executable files containing code for more than one CPU architecture). Apple have long relied on fat binaries for multi-architecture support, going back three decades now. So the Swift toolchain was engineered from the outset around this concept, rather than the more complicated architecture for cross-architecture compilation used by GCC, LLVM, etc.

Where fat binaries exist, the Swift toolchain has just a single installation covering multiple architectures, because all the library files simply contain versions for multiple architectures. But on e.g. Linux where that's not supported, the toolchain is implicitly restricted to just the native architecture of the host machine.

But Swift has also long-supported cross-platform compilation, via the SDK concept. That's essentially where you bundle up all the headers and libraries needed to compile for a given target platform (e.g. an iPhone, or an Apple Watch). That can support other architectures implicitly, based simply on what architecture(s) are available in the SDK's binaries.

The problem right now is that there's no official SDK's built for non-macOS hosts, and the toolchain support for SDKs apparently has some issues on non-Darwin hosts. That's what SE-0387: Swift SDKs for Cross-Compilation (and corresponding pitch thread) intends to address. It's unclear when that will actually be implemented in the toolchain, though, nor when swift.org will start providing official SDKs.

In the interim, there are some folks producing unofficial SDKs, e.g. finagolfin maintains Android SDKs that work on Linux, supporting aarch64, armv7, and x86-64 targets. Perhaps Compiler Explorer could utilise those in the interim? Or take instruction from how those are installed to jerry-rig a vanilla Linux aarch64 SDK?

[finagolfin]

The problem right now is that there's no official SDK's built for non-macOS hosts,

No standalone SDKs, but there have been official toolchains for linux AArch64 for some time now, and a native toolchain always includes that native Swift resource directory (a Swift SDK consists of a Swift resource directory plus the native C headers and C/C++ libraries for that platform).

So you could probably download the official AArch64 toolchain on linux x86_64 and extract the AArch64 resource directory from it at usr/lib/swift/, then use that with the official x86_64 toolchain and whatever C/C++ AArch64 libraries are normally used to cross-compile for AArch64: /home/gb/swift-5.8.1-x86_64/usr/bin/swiftc -target aarch64-linux-gnu -sdk / -resource-dir /home/gb/swift-5.8.1-aarch64/usr/lib/swift.

and the toolchain support for SDKs apparently has some issues on non-Darwin hosts.

The only issue is that the fat binary approach you mentioned means that the Swift runtime libraries are placed in the resource directory in usr/lib/swift/linux, so the Swift compiler cannot look for multiple platform architectures in one resource directory. This is easily worked around by using separate resource directories for each arch, which is what I suggested above.

That's what SE-0387: Swift SDKs for Cross-Compilation (and corresponding pitch thread) intends to address.

I don't think that will fix the fat binary or any other cross-compilation issues: it's more about specifying a standard way to bundle and use cross-compilation SDKs with the Swift package manager (to be clear, this already worked, but now it is standardized).

It's unclear when that will actually be implemented in the toolchain,

I believe it's mostly done and will ship with Swift 5.9 in a couple months, though I can't speak for its primary implementor, @MaxDesiatov, who also works on the Swift WebAssembly port.

If you want to set this up on Compiler Explorer, just let me know of any questions or problems and I'll answer what I can.

[wadetregaskis]

Thanks for elaborating and correcting the bits I misunderstood, @finagolfin.

Realistically I probably won't have time, any time soon, to try to implement this myself for Compiler Explorer - not being at all familiar with its internals - but it seems like the current method is pretty straightforward, as @finagolfin spelled out:

So you could probably download the official AArch64 toolchain on linux x86_64 and extract the AArch64 resource directory from it at usr/lib/swift/, then use that with the official x86_64 toolchain and whatever C/C++ AArch64 libraries are normally used to cross-compile for AArch64: /home/gb/swift-5.8.1-x86_64/usr/bin/swiftc -target aarch64-linux-gnu -sdk / -resource-dir /home/gb/swift-5.8.1-aarch64/usr/lib/swift.

Possibly @RubenRBS or @mattgodbolt could try that out quicker.

It sounds like probably Swift 5.9 (expected to arrive soonish) will have better-defined support for SDKs, and maybe Swift.org will provide some pre-built ones. So that might make things slightly easier and/or more robust, but not dramatically change how this cross-compilation will work.

[verhovsky]

Adding -target aarch64-linux-gnu to the compiler options of Swift 5.9 and 5.10 doesn't work:

<unknown>:0: error: could not find module '_Concurrency' for target 'aarch64-unknown-linux-gnu'; found: x86_64-unknown-linux-gnu, at: /opt/compiler-explorer/swift-5.10/usr/lib/swift/linux/_Concurrency.swiftmodule

-target aarch64-apple-macos14 doesn't either:

<unknown>:0: error: unable to load standard library for target 'aarch64-apple-macos14'

[finagolfin]

Adding -target aarch64-linux-gnu to the compiler options of Swift 5.9 and 5.10 doesn't work

As I noted last year, you would also need to download the Swift libraries compiled for AArch64, which are available in the separate linux AArch64 toolchains at the Swift website, and point the compiler at that AArch64 -resource-dir and the AArch64 C libraries for your distro using the -sdk flag.

-target aarch64-apple-macos14 doesn't either

The OSS toolchains don't support building for Apple platforms, though one guy did get it to work by downloading some closed Xcode files.

[verhovsky]

@RubenRBS can you look at this again please? Swift code is mostly only compiled for ARM64 these days because Apple doesn't make x86 computers anymore.

[Lancelotbronner]

Bumping this thread, I'd love to have this to compare C/C++ output to Swift

[finagolfin]

Some good news, the Swift team just started distributing a Musl static SDK that supports both AArch64 and x86_64 with the trunk 6.1 compiler. I just used those linked instructions to cross-compile some popular Swift packages like swift-crypto and swift-nio from Fedora x86_64 for linux AArch64 without a problem. It could probably be used for this website.

@al45tair, are there any plans to start providing Musl SDKs for 5.10 and 6.0 also? When I just tried to download one for the latest 6.0 snapshot at https://download.swift.org/swift-6.0-branch/static-sdk/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-06-13-a/swift-DEVELOPMENT-SNAPSHOT-2024-06-13-a_static-linux-0.0.1.artifactbundle.tar.gz, it failed.

[al45tair]

There are no plans to support this in 5.10, but the Static SDK for Linux is a 6.0 feature so a release build will ship when 6.0 is released later in the year. The Static SDK for Linux is being built for both main and the release/6.0 branch, though it looks like the last successful build was on the 8th of June so we probably need to investigate that.

The links for the latest builds are on swift.org.