build: LLD based macOS toolchain #21778

fanquake · 2021-04-26T07:40:29Z

This switches us to using a LLD based toolchain for macOS builds.

Benefits

Less complicated macOS toolchain.
Drop FORCE_USE_SYSTEM_CLANG in favour of always using the system Clang.
No longer beholden to Apple releasing it's source for cctools, ld64 & libtapi.
No more reliance on third parties to modify those sources for us. i.e apple-libtapi, cctools-port (cctools + ld64).

DrahtBot · 2021-04-26T08:36:39Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	theuni, hebasto, michaelfolkson

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#29790 ([DO NOT MERGE] cmake: Migrate CI scripts to CMake-based build system -- WIP by hebasto)
#29739 (depends: swap some cctools tools for LLVM tools by fanquake)
#29725 (depends: build libqrencode with CMake by fanquake)
#29723 (depends: build zeromq with CMake by fanquake)
#29708 (depends: build libnatpmp with CMake by fanquake)
#29707 (depends: build miniupnpc with CMake by fanquake)
#25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)
#25391 (guix: Use LTO to build releases by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ryanofsky · 2021-04-26T13:54:00Z

This definitely seems like a good thing to add support for, but it seems sad to have to drop support for the open source compiler at the same time as adding support for the closed source one. I don't know enough about this to have an informed opinion, though.

EDIT: Just a misunderstanding

fanquake · 2021-04-26T13:59:47Z

but it seems sad to have to drop support for the open source compiler at the same time as adding support for the closed source one.

I might be misunderstanding your comment, but the opposite is happening here. We're migrating away from using Apples, irregularly open sourced tools (cctools, ld64), to just vanilla LLVM (lld), which is entirely open source.

ryanofsky · 2021-04-26T14:11:26Z

I might be misunderstanding your comment, but the opposite is happening here. We're migrating away from using Apples, irregularly open sourced tools (cctools, ld64), to just vanilla LLVM (lld), which is entirely open source.

My bad! I thought this was switching from an unsupported apple open-source toolchain to an supported apple closed source toolchain, not to a vanilla toolchain. My blinding hate of apple has kept me from seeing reality! Thanks for the clarification. This all seems very good.

theuni · 2021-04-26T18:58:09Z

Very nice! Strong concept ACK, but obviously we can't use this until all features we need are supported. But it's great to see that linking finally works!

For a little extra context beyond what @fanquake said above, the macOS clang+cctools build has always been a wonky outlier because we have to build our own (very unsupported) toolchain first. LD64 is apple's libre/open-source linker, and there is a community project that maintains a build environment for cross builds.

For a few years, there has been slow development on a darwin functionality for llvm's lld, but it's only really picked up speed and become usable in the last year or two. This PR would bring our Apple builds much more inline with others in that it would finally be using a supported libre/open-source compiler/linker combo.

@fanquake Is cctools still needed at all? LLVM ships bitcode equivalents of most binutils tools (ar, nm, ranlib, etc), I wonder if we can use those instead. Maybe they require lto?

fanquake · 2021-04-28T03:06:19Z

Is cctools still needed at all? LLVM ships bitcode equivalents of most binutils tools (ar, nm, ranlib, etc), I wonder if we can use those instead. Maybe they require lto?

@theuni no cctools is no longer required. I drop the native_cctools as part of this PR and we just use the LLVM tools.

theuni · 2021-04-28T04:32:27Z

@theuni no cctools is no longer required. I drop the native_cctools as part of this PR and we just use the LLVM tools.

Apologies for reading too quickly, I see now that I only looked at the first commit. Turns out the 3rd one is the one I expected :)

fanquake · 2021-05-01T06:10:48Z

Rebased for #21793, and rebased on top of #19817. Made some additional changes, such as renaming the native_cctools package to native_llvm, and removed some now-redundant info from the macdeploy documentation.

Note: I'm pretty sure the use of ZERO_AR_DATE can now be dropped entirely, but have left that out for now.

Use LLD when compiling for macOS. Always use the system Clang when compiling for macOS (remove FORCE_USE_SYSTEM_CLANG).

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement.

LLD enables ZERO_AR_DATE by default, setting it to zero would enable non-determinism, setting it to any other value is ignored. See: https://github.com/llvm/llvm-project/blob/main/lld/docs/MachO/ld64-vs-lld.rst.

ff42d81 guix: use clang-toolchain-15 for macOS compilation (fanquake) 94955b4 depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake) Pull request description: This will end up being a blocker for bitcoin#28210, and is already part of bitcoin#21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang. ACKs for top commit: TheCharlatan: re-ACK ff42d81 Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement. This is related to bitcoin#21778, and the switchover to using vanilla LLVM for macOS.

DrahtBot · 2024-04-05T15:25:12Z

🐙 This pull request conflicts with the target branch and needs rebase.

This is already used in multiple CIs, and will soon become a requirement for most CIs, i.e when we migrate depends packages to use CMake, for example: bitcoin/bitcoin#21778 (comment). Some of the CIs in 21778 are failing because CMake isn't available, so just break this out and make CMake globally available.

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement. This is related to bitcoin#21778, and the switchover to using vanilla LLVM for macOS.

ff42d81 guix: use clang-toolchain-15 for macOS compilation (fanquake) 94955b4 depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake) Pull request description: This will end up being a blocker for bitcoin#28210, and is already part of bitcoin#21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang. ACKs for top commit: TheCharlatan: re-ACK ff42d81 Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement. This is related to bitcoin#21778, and the switchover to using vanilla LLVM for macOS.

ff42d81 guix: use clang-toolchain-15 for macOS compilation (fanquake) 94955b4 depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake) Pull request description: This will end up being a blocker for bitcoin#28210, and is already part of bitcoin#21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang. ACKs for top commit: TheCharlatan: re-ACK ff42d81 Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f

I need to try and patch around Qts buildsystem to keep bitcoin#21778 moving along (the issue being that even when you tell Qt to build using Clang on Linux, it still calls out to GCC, breaking our ability to have a macOS release build env that doesn't have a GCC toolchain installed, and thus no ld binary). Before trying to patch Qt any further, update to the latest LTS release, and update the current patch set. Github-Pull: bitcoin#29732 Rebased-From: 430f319

ff42d81 guix: use clang-toolchain-15 for macOS compilation (fanquake) 94955b4 depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake) Pull request description: This will end up being a blocker for bitcoin#28210, and is already part of bitcoin#21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang. ACKs for top commit: TheCharlatan: re-ACK ff42d81 Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement. This is related to bitcoin#21778, and the switchover to using vanilla LLVM for macOS.

… prepare for building with upstream LLVM 3df6070 contrib: remove macOS lazy_bind check (fanquake) 9bc357e build: explicitly opt-in to new fixup_chains functionality for darwin (Cory Fields) fb61bc0 depends: Bump MacOS minimum runtime requirement to 11.0 (Cory Fields) c2cd472 depends: bump darwin clang to 11.1 (Cory Fields) Pull request description: This (I believe) resolves the last of the blockers for [switching us away from cctools and instead using out-of-the-box llvm and lld](bitcoin#21778) for building Darwin binaries. For now, we continue building with a pre-packaged llvm and cctools, but after this PR the clang+lld combo should just work for anyone trying it. Additionally after this PR, the new runtime `fixup_chains` behavior will be in-use, as ld64 uses it as well. The commits may seem unrelated, so in detail: lld (llvm's linker) has been a work-in-progress for Darwin for years. Recently though, it has gained nearly all of the features we require. The last missing feature from ld64, `-Wl,-bind_at_load`, is not implemented in lld; as far as I can tell [lazy loading has conceptually been replaced by fixup chains](https://www.emergetools.com/blog/posts/iOS15LaunchTime). So that means we don't need ld64's `bind_at_load` as long as lld can handle `-Wl,-fixup_chains` (which it can). I've added it to our configure as a linker option mostly so that we can see it in the logs; it's default-on as long as the minimum version is >11.0. About that: the runtime functionality required for `-Wl,-fixup_chains` [requires macOS >=11.0](https://github.com/llvm/llvm-project/blob/release/16.x/lld/MachO/Driver.cpp#L1021). Hence the commit that bumps the minimum version. Our current min runtime of `10.15` has been unsupported since September 2022, so I don't expect this bump to be controversial. Lastly, with the minimum runtime version bumped to 11.0, our current version of pre-compiled clang we use for macOS is too old to understand `-mmacosx-version-min=11.0` because it expects `=10.x`. So I've made the smallest possible bump (from 10.0.1 to 11.1.0) to a version that understands. This bump is arbitrary and unfortunate, but likely to be short-lived as we may end up replacing it with llvm+lld for v26 anyway. I've held off on bumping the SDK as I think that makes sense to do as part of the lld switch instead. ACKs for top commit: hebasto: ACK 3df6070 gruve-p: ACK bitcoin@3df6070 fanquake: ACK 3df6070 TheCharlatan: ACK 3df6070 Tree-SHA512: 0200ec4a3b88df33877ae82c15b5c04d745852550923f491a354b391cac65f88e4df116a40055c23a8cbcfcdfb9a376c6ae8fdd0e898e7b966bc213dcb5857cf

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement. This is related to bitcoin#21778, and the switchover to using vanilla LLVM for macOS.

fanquake added macOS Build system labels Apr 26, 2021

DrahtBot mentioned this pull request Apr 26, 2021

build: macOS toolchain bump #19817

Merged

DrahtBot mentioned this pull request Apr 28, 2021

build: use -isysroot over --sysroot on macOS #21793

Merged

fanquake force-pushed the llvm_13_ld64_lld branch from b9426c9 to 9a5ee24 Compare May 1, 2021 06:08

DrahtBot mentioned this pull request May 1, 2021

build, qt: No longer need to patch translation.pro #21420

Closed

fanquake force-pushed the llvm_13_ld64_lld branch from 9a5ee24 to b79ed8c Compare May 3, 2021 06:10

DrahtBot mentioned this pull request May 5, 2021

release: support cross-compiling for arm64-apple-darwin #21851

Merged

fanquake force-pushed the llvm_13_ld64_lld branch from b79ed8c to f068682 Compare June 4, 2021 12:48

DrahtBot mentioned this pull request Jun 13, 2021

build: Avoid remaking makefiles. #22237

Closed

This was referenced Jul 13, 2021

build: use aarch64 Clang if cross-compiling for darwin on aarch64 #22436

Merged

build: Avoid rebuilding makefiles #22486

Closed

fanquake force-pushed the llvm_13_ld64_lld branch from f068682 to e6b20d5 Compare July 20, 2021 11:42

DrahtBot mentioned this pull request Jul 22, 2021

build: use newer config.guess & config.sub in depends #22526

Merged

fanquake force-pushed the llvm_13_ld64_lld branch 7 times, most recently from e1b2e6e to 1179221 Compare October 12, 2021 00:18

DrahtBot mentioned this pull request Oct 12, 2021

build: Fix make apk for Android w/ non-default SOURCES_PATH in depends #22555

Closed

fanquake added 6 commits April 4, 2024 12:24

guix: add lld into macOS build environment

5c834d8

build: use system Clang & LLD when compiling for macOS

54bc0af

Use LLD when compiling for macOS. Always use the system Clang when compiling for macOS (remove FORCE_USE_SYSTEM_CLANG).

build: remove cctools and libtapi packages

f3ab1b3

build: swap otool for llvm-objdump

734e105

Similar to libtool, (llvm-)otool only exists with a version suffix on some systems (Ubuntu), which makes it annoying to use/find. Avoid this, by switching to objdump. Which is a drop-in replacement.

guix: use GUIX_LD_WRAPPER_DISABLE_RPATH for all HOSTS

cb8f4a9

guix: remove ZERO_AR_DATE export

c4aa412

LLD enables ZERO_AR_DATE by default, setting it to zero would enable non-determinism, setting it to any other value is ignored. See: https://github.com/llvm/llvm-project/blob/main/lld/docs/MachO/ld64-vs-lld.rst.

DrahtBot added the Needs rebase label Apr 4, 2024

fanquake force-pushed the llvm_13_ld64_lld branch from 4ecdf4a to c4aa412 Compare April 4, 2024 12:04

DrahtBot removed the Needs rebase label Apr 4, 2024

DrahtBot added the Needs rebase label Apr 5, 2024

maflcko mentioned this pull request May 2, 2024

releases: use LLVM 18 for macOS #30022

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: LLD based macOS toolchain #21778

build: LLD based macOS toolchain #21778

fanquake commented Apr 26, 2021 •

edited

DrahtBot commented Apr 26, 2021 •

edited

ryanofsky commented Apr 26, 2021 •

edited

fanquake commented Apr 26, 2021

ryanofsky commented Apr 26, 2021

theuni commented Apr 26, 2021 •

edited

fanquake commented Apr 28, 2021

theuni commented Apr 28, 2021

fanquake commented May 1, 2021

DrahtBot commented Apr 5, 2024

build: LLD based macOS toolchain #21778

Are you sure you want to change the base?

build: LLD based macOS toolchain #21778

Conversation

fanquake commented Apr 26, 2021 • edited

Benefits

DrahtBot commented Apr 26, 2021 • edited

Code Coverage

Reviews

Conflicts

ryanofsky commented Apr 26, 2021 • edited

fanquake commented Apr 26, 2021

ryanofsky commented Apr 26, 2021

theuni commented Apr 26, 2021 • edited

fanquake commented Apr 28, 2021

theuni commented Apr 28, 2021

fanquake commented May 1, 2021

DrahtBot commented Apr 5, 2024

fanquake commented Apr 26, 2021 •

edited

DrahtBot commented Apr 26, 2021 •

edited

ryanofsky commented Apr 26, 2021 •

edited

theuni commented Apr 26, 2021 •

edited