macOS: Bump minimum required runtime version and prepare for building with upstream LLVM #27676

theuni · 2023-05-16T15:52:00Z

This (I believe) resolves the last of the blockers for switching us away from cctools and instead using out-of-the-box llvm and lld for building Darwin binaries.

For now, we continue building with a pre-packaged llvm and cctools, but after this PR the clang+lld combo should just work for anyone trying it. Additionally after this PR, the new runtime fixup_chains behavior will be in-use, as ld64 uses it as well.

The commits may seem unrelated, so in detail:

lld (llvm's linker) has been a work-in-progress for Darwin for years. Recently though, it has gained nearly all of the features we require. The last missing feature from ld64, -Wl,-bind_at_load, is not implemented in lld; as far as I can tell lazy loading has conceptually been replaced by fixup chains.

So that means we don't need ld64's bind_at_load as long as lld can handle -Wl,-fixup_chains (which it can). I've added it to our configure as a linker option mostly so that we can see it in the logs; it's default-on as long as the minimum version is >11.0.

About that: the runtime functionality required for -Wl,-fixup_chains requires macOS >=11.0. Hence the commit that bumps the minimum version. Our current min runtime of 10.15 has been unsupported since September 2022, so I don't expect this bump to be controversial.

Lastly, with the minimum runtime version bumped to 11.0, our current version of pre-compiled clang we use for macOS is too old to understand -mmacosx-version-min=11.0 because it expects =10.x. So I've made the smallest possible bump (from 10.0.1 to 11.1.0) to a version that understands. This bump is arbitrary and unfortunate, but likely to be short-lived as we may end up replacing it with llvm+lld for v26 anyway. I've held off on bumping the SDK as I think that makes sense to do as part of the lld switch instead.

DrahtBot · 2023-05-16T15:52:03Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	fanquake, TheCharlatan, hebasto, gruve-p

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#27897 (guix: use GCC 12.2.0 to build releases by fanquake)
#27099 (build: produce a .zip for macOS distribution by fanquake)
#21778 (build: LLVM 15 & LLD based macOS toolchain by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

theuni · 2023-05-16T15:57:16Z

Ping @fanquake and @hebasto . There's not much happening here technically, but a few big-ish conceptual changes.

Also, we may want to go one step further in configure and do an actual compile check to verify that the minimum version is set to >=11.0 as we now have that as an assumption.

fanquake · 2023-05-16T15:57:51Z

Concept ACK - note that this also needs a change to clang-toolchain-11 in Guix, which should be available without a time-machine change.

hebasto · 2023-05-16T16:33:56Z

Concept ACK.

fanquake · 2023-05-16T16:37:15Z

check to verify that the minimum version is set to >=11.0 as we now have that as an assumption.

Note that you'll also need to update our minimum version check in:

bitcoin/contrib/devtools/symbol-check.py

Line 235 in 904631e

if binary.build_version.minos == [10,15,0]:

theuni · 2023-05-16T19:28:54Z

Note that you'll also need to update our minimum version check in:

bitcoin/contrib/devtools/symbol-check.py

Line 235 in 904631e

if binary.build_version.minos == [10,15,0]:

Thanks. Done. I used #22993 as a template for bumping, we'll see if c-i is happy.

theuni · 2023-05-16T22:54:38Z

Pushed a test for fixup_chains, but I don't think it ever ran. @fanquake What sets RUN_SECURITY_TESTS?

fanquake · 2023-05-17T09:59:07Z

@fanquake What sets RUN_SECURITY_TESTS?

Nothing. symbol/security checks are no-longer run in the CI, because the CI environment is semi-regularly changing, and the binaries produced don't represent release binaries, meaning there are some tests that will never pass. RUN_SECURITY_TESTS is leftover from when we used run (some) tests; I've opened a PR to remove it (#27683). The current way to run the security/symbol checks is to perform a Guix build.

fanquake · 2023-05-17T10:17:35Z

The CI failure is unrelated, and will be fixed post-rebase. Guix currently fails to build. i.e:

time HOSTS="x86_64-apple-darwin arm64-apple-darwin" ./contrib/guix/guix-build
<snip>
Extracting libevent...
/home/ubuntu/sources/libevent-2.1.12-stable.tar.gz: OK
Preprocessing libevent...
Configuring libevent...
checking for a BSD-compatible install... /home/ubuntu/.guix-profile/bin/install -c
checking whether build environment is sane... yes
checking for x86_64-apple-darwin-strip... x86_64-apple-darwin-strip
checking for a thread-safe mkdir -p... /home/ubuntu/.guix-profile/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether make supports the include directive... yes (GNU style)
checking for x86_64-apple-darwin-gcc... env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include accepts -g... yes
checking for env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include option to accept ISO C89... none needed
checking whether env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include understands -c and -o together... yes
checking dependency style of env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include... none
checking how to run the C preprocessor... /lib/cpp
configure: error: in `/bitcoin/depends/work/build/x86_64-apple-darwin/libevent/2.1.12-stable-3b53fa1ad55':
configure: error: C preprocessor "/lib/cpp" fails sanity check
See `config.log' for more details
make: *** [funcs.mk:292: /bitcoin/depends/x86_64-apple-darwin/.libevent_stamp_configured] Error 1
make: Leaving directory '/bitcoin/depends'

config.log https://gist.github.com/fanquake/e0e1cc9f2976eb745dbb92a69c350994:

configure:4403: checking how to run the C preprocessor
configure:4434: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/.guix-profile/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=11.0 -B/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem -Xclang /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include -Xclang -internal-externc-isystem -Xclang /home/ubuntu/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include -E   -I/bitcoin/depends/x86_64-apple-darwin/include -D_FORTIFY_SOURCE=3 conftest.c
In file included from conftest.c:13:
In file included from /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include/limits.h:21:
In file included from /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/limits.h:26:
In file included from /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/bits/libc-header-start.h:33:
/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/features.h:397:4: warning: _FORTIFY_SOURCE requires compiling with optimization (-O) [-W#warnings]
#  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
   ^
In file included from conftest.c:13:
In file included from /gnu/store/2vd3s80pipf4dffzrf4niphl2mdhrjx1-clang-11.0.0/lib/clang/11.0.0/include/limits.h:21:
In file included from /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/limits.h:195:
In file included from /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/bits/posix1_lim.h:161:
/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/include/bits/local_lim.h:38:10: fatal error: 'linux/limits.h' file not found
#include <linux/limits.h>
         ^~~~~~~~~~~~~~~~
1 warning and 1 error generated.

theuni · 2023-05-17T22:10:56Z

Converting to draft while I investigate the clang issue.

DrahtBot · 2023-05-17T23:41:20Z

Guix builds

File	commit `a75c77e` (master)	commit `efb1789` (master and this pull)
SHA256SUMS.part	`434bbf6e6617a5de...`	`099c7640a6db2b63...`
*-aarch64-linux-gnu-debug.tar.gz	`0e712bb15da16909...`	`fddd8aa0cf0ce204...`
*-aarch64-linux-gnu.tar.gz	`ae2f0f4a424693d6...`	`5ed6550917d12e04...`
*-arm-linux-gnueabihf-debug.tar.gz	`cd532071bf713066...`	`3d18851528024c8a...`
*-arm-linux-gnueabihf.tar.gz	`4c1728e9d71867ce...`	`fdc498151e33a2ca...`
*-powerpc64-linux-gnu-debug.tar.gz	`f9b571a4968316f6...`	`0e7e7c3cf45931ab...`
*-powerpc64-linux-gnu.tar.gz	`55fdbbad89963dad...`	`33559c9a7fd0c705...`
*-powerpc64le-linux-gnu-debug.tar.gz	`f38d53401f133237...`	`29260312162919dd...`
*-powerpc64le-linux-gnu.tar.gz	`0b36ad26323ff653...`	`dfe702e77f8f7216...`
*-riscv64-linux-gnu-debug.tar.gz	`3eda34ba2def1068...`	`7c5419a8a2d5fa07...`
*-riscv64-linux-gnu.tar.gz	`0eb983ca2a074b4b...`	`b02230cb5643b7f9...`
*-x86_64-linux-gnu-debug.tar.gz	`db4fe4e006203a8f...`	`e6c3141c9872c8c0...`
*-x86_64-linux-gnu.tar.gz	`53535f16dac817fe...`	`e67513e1b8c852da...`
*.tar.gz	`54187f92f6847350...`	`39a5c64edc814fa1...`
guix_build.log	`33cd49590ace6813...`	`eafbb9580e213697...`
guix_build.log.diff		`a94604804a100954...`

theuni · 2023-05-18T22:31:56Z

It took several days to track down, but... I believe the issue with clang >=11 comes from: llvm/llvm-project@6fa3894 . From my local tests I'm able to revert that and get back to expected results.

So I guess we'll need to patch it out of guix.

depends/packages/native_clang.mk

theuni · 2023-05-22T17:08:51Z

So I guess we'll need to patch it out of guix.

Suggestions for how to go about this are welcome. It's not at all straightforward to me how to patch out.

The llvm toolchain bumps are currently blocked on this :(

theuni · 2023-05-24T20:40:02Z

Added a custom forked guix repo containing the patch we need for llvm's build.

I believe I've fixed authorization and it should just work(tm).

My local guix build works now, but I've reworked things so many times it would be helpful to get a confirmation.

Going to mark as ready for review since guix builds might actually work now.

theuni · 2023-05-24T23:09:19Z

See relevant #25098. If we go forward with this we'll want the guix fork as part of our github org rather than my personal repo.

theuni · 2023-06-23T18:41:01Z

just need to decide out what to do about guix.

What options are currently being considered. Is an upstream patch still possible?

FWIW, though we didn't end up needing it, an upstream guix "fix" is slotted to go in as well, which we can pick up in some future time-machine bump.

That gives us the ability to customize our clang build if we ever need to in the future.

fanquake

ACK 3df6070

hebasto · 2023-06-26T12:42:17Z

the runtime functionality required for -Wl,-fixup_chains requires macOS >=11.0.

However, Xcode 13 Release Notes state:

All programs and dylibs built with a deployment target of macOS 12 or iOS 15 or later now use the chained fixups format. This uses different load commands and LINKEDIT data, and won’t run or load on older OS versions.

What effect will the -Wl,-fixup_chains option have for OSX_MIN_VERSION=11.0?

hebasto · 2023-06-26T13:03:52Z

It would be nice if someone confirms that Guix binaries run on macOS 11 BigSur.

TheCharlatan · 2023-06-26T13:45:24Z

Guix build

find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
109ed45706b8cbdc897b6c51efa0247f8e8f1e6bb35c57e69f30093b62189a51  guix-build-3df60704661c/output/aarch64-linux-gnu/SHA256SUMS.part
4e28463f1c074c7c121abd5fbad89aa2269c5978829ccd4898d33a21eedfeb4f  guix-build-3df60704661c/output/aarch64-linux-gnu/bitcoin-3df60704661c-aarch64-linux-gnu-debug.tar.gz
f20d074f0529202274a40f0530f79f601f5c575f1cd0addc099418719982485e  guix-build-3df60704661c/output/aarch64-linux-gnu/bitcoin-3df60704661c-aarch64-linux-gnu.tar.gz
414d7c4bf531a8d8589c15c9636ec0b092b28cba359b7c5ff46cd5bc0a768ac6  guix-build-3df60704661c/output/arm-linux-gnueabihf/SHA256SUMS.part
97e5cc1caf38e9e6cb4efde9be3a163c7373a1bdf1f7bdfbaa5fa7c9dd22a87c  guix-build-3df60704661c/output/arm-linux-gnueabihf/bitcoin-3df60704661c-arm-linux-gnueabihf-debug.tar.gz
cc174cb93c1b13b0efd5e002741ee6a13fcf1388f766eba5c2e723c11bb43775  guix-build-3df60704661c/output/arm-linux-gnueabihf/bitcoin-3df60704661c-arm-linux-gnueabihf.tar.gz
b836ca05dbc6521c15c07d4defebeb0da42d5e45add7eca22afd828178b97d5e  guix-build-3df60704661c/output/arm64-apple-darwin/SHA256SUMS.part
72f212a251d9ca25c3d0b0619fa405e92688aecf3b7fbf72ee510ca56f9f54f6  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin-unsigned.dmg
ec876babdbfc23c9381950ccbbe311f1e9106ff2efa58cd0821a0b5704e8586e  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin-unsigned.tar.gz
a359e6e1da3d3e9a3335a2c7f09ca4fa5b9ae5fa884dd3827d4dd29422b801bd  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin.tar.gz
5155153ca28a58c4c53cdd101567d3bb72f540a14f998a91d717ac1673b7ec6c  guix-build-3df60704661c/output/dist-archive/bitcoin-3df60704661c.tar.gz
afed3d1dc33acf2a7532eee12b090c379c205354ef21ee40d868187dab2d31ad  guix-build-3df60704661c/output/powerpc64-linux-gnu/SHA256SUMS.part
6914deb97cc52917fe59a80bf670ca171bcfca5a63fcf613b9072691a403a795  guix-build-3df60704661c/output/powerpc64-linux-gnu/bitcoin-3df60704661c-powerpc64-linux-gnu-debug.tar.gz
c8b6518fbed9181bd025a6e6f539fd16e5043e0aa887aec2a394448bacb250b2  guix-build-3df60704661c/output/powerpc64-linux-gnu/bitcoin-3df60704661c-powerpc64-linux-gnu.tar.gz
826feef8848521ecca935dd97edbf71b1616c83251c21fb667ceaac51e339cc2  guix-build-3df60704661c/output/powerpc64le-linux-gnu/SHA256SUMS.part
64c3a87cfe4b50c7b0561e208eb796ad46313d534ad6178e44280d5f2f78b757  guix-build-3df60704661c/output/powerpc64le-linux-gnu/bitcoin-3df60704661c-powerpc64le-linux-gnu-debug.tar.gz
668aa0c20fba5e23b15cf2971bd2235138ca333f7e5676a8ce270a30a9d3c01e  guix-build-3df60704661c/output/powerpc64le-linux-gnu/bitcoin-3df60704661c-powerpc64le-linux-gnu.tar.gz
05911bf9c6d3000bee0e8a57f181dfb9038ab957c93a0f5fa407610bfa012828  guix-build-3df60704661c/output/riscv64-linux-gnu/SHA256SUMS.part
b9be6c766869290fc18b10fe5dc7d5dd9c4e5e3d4ba75fcf3718595d74f69ce1  guix-build-3df60704661c/output/riscv64-linux-gnu/bitcoin-3df60704661c-riscv64-linux-gnu-debug.tar.gz
2f81847c316e7f5766cd6a50f87e9935f6257a4a22dfb15eabdf3c30f5599990  guix-build-3df60704661c/output/riscv64-linux-gnu/bitcoin-3df60704661c-riscv64-linux-gnu.tar.gz
0a4ae5b222ba181430a9ad7f9c5639d90fc40c94da8fe71da83dfff0f2119771  guix-build-3df60704661c/output/x86_64-apple-darwin/SHA256SUMS.part
c34705191796f2bbed9ee38a81866e68be076265a8215439fd29c0f501504c67  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin-unsigned.dmg
fd6c6dbcb6dd218d4463dc383ef107cd4b634ac21fd730e73f2d064981fb1ed3  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin-unsigned.tar.gz
8ff3c9b52bcb4b707dc6e6da7048f9ce9930810d4ca2c6b5e466ac956378d672  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin.tar.gz
b531a3269f27274a0a29a8ca5f4e9450d19d9c0e14e63bc2c078a0160e537488  guix-build-3df60704661c/output/x86_64-linux-gnu/SHA256SUMS.part
2926dc8f562ff0fc0944f5e27e68d86f0bc58c138bd20275b732b11911710714  guix-build-3df60704661c/output/x86_64-linux-gnu/bitcoin-3df60704661c-x86_64-linux-gnu-debug.tar.gz
69fa88d2d46cebbba109514c51ca694ec37a03c4c02e87a64f4b65dc4f7c6378  guix-build-3df60704661c/output/x86_64-linux-gnu/bitcoin-3df60704661c-x86_64-linux-gnu.tar.gz
801e5394b1561bf20461fa4494ea71ba02e61735b964e4393096f9077fc54a97  guix-build-3df60704661c/output/x86_64-w64-mingw32/SHA256SUMS.part
0d9b2b615630ad4d9a164dd05d316f386270261d4dc2ade9622bed999a0abb90  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-debug.zip
a7f08c2d3ba4f93623b7a2634766bed1464523653f4223aa13302e3ce39b226a  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-setup-unsigned.exe
855309534b379c3a27ecc9690d4553376a3aabb7383242bbe6f790ad43230bc5  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-unsigned.tar.gz
d9c21190a3b1e437c9d2e25a2ee72e22848cbd00b6eb329e65f769c138c0ec7f  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64.zip

hebasto · 2023-06-26T13:58:01Z

Guix (self-signed) bitcoin-qt tested on macOS Ventura 13.4.1 (Apple M1).

theuni · 2023-06-26T16:46:21Z

the runtime functionality required for -Wl,-fixup_chains requires macOS >=11.0.

However, Xcode 13 Release Notes state:

All programs and dylibs built with a deployment target of macOS 12 or iOS 15 or later now use the chained fixups format. This uses different load commands and LINKEDIT data, and won’t run or load on older OS versions.

What effect will the -Wl,-fixup_chains option have for OSX_MIN_VERSION=11.0?

It would be nice if someone confirms that Guix binaries run on macOS 11 BigSur.

Yeah, that's a good point. The docs (and a quick glimpse at the code) shows that it should work fine on macOS 11, but as it wasn't default there it's worth confirming.

I'm also unable to test as I'm on 13.x :(

hebasto · 2023-06-26T18:14:05Z

Guix (self-signed) bitcoin-qt tested on macOS Ventura 13.4.1 (Apple M1).

Guix DMG tested on macOS Monterey 12.6.7 (Intel).

TheCharlatan · 2023-06-26T20:03:19Z

I can confirm that my guix dmg runs on macOS 11.1.

TheCharlatan

ACK 3df6070

hebasto · 2023-06-26T20:10:17Z

ACK 3df6070

tobtoht · 2023-06-26T21:20:23Z

Guix builds:

109ed45706b8cbdc897b6c51efa0247f8e8f1e6bb35c57e69f30093b62189a51  guix-build-3df60704661c/output/aarch64-linux-gnu/SHA256SUMS.part
4e28463f1c074c7c121abd5fbad89aa2269c5978829ccd4898d33a21eedfeb4f  guix-build-3df60704661c/output/aarch64-linux-gnu/bitcoin-3df60704661c-aarch64-linux-gnu-debug.tar.gz
f20d074f0529202274a40f0530f79f601f5c575f1cd0addc099418719982485e  guix-build-3df60704661c/output/aarch64-linux-gnu/bitcoin-3df60704661c-aarch64-linux-gnu.tar.gz
414d7c4bf531a8d8589c15c9636ec0b092b28cba359b7c5ff46cd5bc0a768ac6  guix-build-3df60704661c/output/arm-linux-gnueabihf/SHA256SUMS.part
97e5cc1caf38e9e6cb4efde9be3a163c7373a1bdf1f7bdfbaa5fa7c9dd22a87c  guix-build-3df60704661c/output/arm-linux-gnueabihf/bitcoin-3df60704661c-arm-linux-gnueabihf-debug.tar.gz
cc174cb93c1b13b0efd5e002741ee6a13fcf1388f766eba5c2e723c11bb43775  guix-build-3df60704661c/output/arm-linux-gnueabihf/bitcoin-3df60704661c-arm-linux-gnueabihf.tar.gz
b836ca05dbc6521c15c07d4defebeb0da42d5e45add7eca22afd828178b97d5e  guix-build-3df60704661c/output/arm64-apple-darwin/SHA256SUMS.part
72f212a251d9ca25c3d0b0619fa405e92688aecf3b7fbf72ee510ca56f9f54f6  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin-unsigned.dmg
ec876babdbfc23c9381950ccbbe311f1e9106ff2efa58cd0821a0b5704e8586e  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin-unsigned.tar.gz
a359e6e1da3d3e9a3335a2c7f09ca4fa5b9ae5fa884dd3827d4dd29422b801bd  guix-build-3df60704661c/output/arm64-apple-darwin/bitcoin-3df60704661c-arm64-apple-darwin.tar.gz
5155153ca28a58c4c53cdd101567d3bb72f540a14f998a91d717ac1673b7ec6c  guix-build-3df60704661c/output/dist-archive/bitcoin-3df60704661c.tar.gz
afed3d1dc33acf2a7532eee12b090c379c205354ef21ee40d868187dab2d31ad  guix-build-3df60704661c/output/powerpc64-linux-gnu/SHA256SUMS.part
6914deb97cc52917fe59a80bf670ca171bcfca5a63fcf613b9072691a403a795  guix-build-3df60704661c/output/powerpc64-linux-gnu/bitcoin-3df60704661c-powerpc64-linux-gnu-debug.tar.gz
c8b6518fbed9181bd025a6e6f539fd16e5043e0aa887aec2a394448bacb250b2  guix-build-3df60704661c/output/powerpc64-linux-gnu/bitcoin-3df60704661c-powerpc64-linux-gnu.tar.gz
826feef8848521ecca935dd97edbf71b1616c83251c21fb667ceaac51e339cc2  guix-build-3df60704661c/output/powerpc64le-linux-gnu/SHA256SUMS.part
64c3a87cfe4b50c7b0561e208eb796ad46313d534ad6178e44280d5f2f78b757  guix-build-3df60704661c/output/powerpc64le-linux-gnu/bitcoin-3df60704661c-powerpc64le-linux-gnu-debug.tar.gz
668aa0c20fba5e23b15cf2971bd2235138ca333f7e5676a8ce270a30a9d3c01e  guix-build-3df60704661c/output/powerpc64le-linux-gnu/bitcoin-3df60704661c-powerpc64le-linux-gnu.tar.gz
05911bf9c6d3000bee0e8a57f181dfb9038ab957c93a0f5fa407610bfa012828  guix-build-3df60704661c/output/riscv64-linux-gnu/SHA256SUMS.part
b9be6c766869290fc18b10fe5dc7d5dd9c4e5e3d4ba75fcf3718595d74f69ce1  guix-build-3df60704661c/output/riscv64-linux-gnu/bitcoin-3df60704661c-riscv64-linux-gnu-debug.tar.gz
2f81847c316e7f5766cd6a50f87e9935f6257a4a22dfb15eabdf3c30f5599990  guix-build-3df60704661c/output/riscv64-linux-gnu/bitcoin-3df60704661c-riscv64-linux-gnu.tar.gz
0a4ae5b222ba181430a9ad7f9c5639d90fc40c94da8fe71da83dfff0f2119771  guix-build-3df60704661c/output/x86_64-apple-darwin/SHA256SUMS.part
c34705191796f2bbed9ee38a81866e68be076265a8215439fd29c0f501504c67  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin-unsigned.dmg
fd6c6dbcb6dd218d4463dc383ef107cd4b634ac21fd730e73f2d064981fb1ed3  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin-unsigned.tar.gz
8ff3c9b52bcb4b707dc6e6da7048f9ce9930810d4ca2c6b5e466ac956378d672  guix-build-3df60704661c/output/x86_64-apple-darwin/bitcoin-3df60704661c-x86_64-apple-darwin.tar.gz
b531a3269f27274a0a29a8ca5f4e9450d19d9c0e14e63bc2c078a0160e537488  guix-build-3df60704661c/output/x86_64-linux-gnu/SHA256SUMS.part
2926dc8f562ff0fc0944f5e27e68d86f0bc58c138bd20275b732b11911710714  guix-build-3df60704661c/output/x86_64-linux-gnu/bitcoin-3df60704661c-x86_64-linux-gnu-debug.tar.gz
69fa88d2d46cebbba109514c51ca694ec37a03c4c02e87a64f4b65dc4f7c6378  guix-build-3df60704661c/output/x86_64-linux-gnu/bitcoin-3df60704661c-x86_64-linux-gnu.tar.gz
801e5394b1561bf20461fa4494ea71ba02e61735b964e4393096f9077fc54a97  guix-build-3df60704661c/output/x86_64-w64-mingw32/SHA256SUMS.part
0d9b2b615630ad4d9a164dd05d316f386270261d4dc2ade9622bed999a0abb90  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-debug.zip
a7f08c2d3ba4f93623b7a2634766bed1464523653f4223aa13302e3ce39b226a  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-setup-unsigned.exe
855309534b379c3a27ecc9690d4553376a3aabb7383242bbe6f790ad43230bc5  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64-unsigned.tar.gz
d9c21190a3b1e437c9d2e25a2ee72e22848cbd00b6eb329e65f769c138c0ec7f  guix-build-3df60704661c/output/x86_64-w64-mingw32/bitcoin-3df60704661c-win64.zip

gruve-p · 2023-06-27T07:35:14Z

ACK 3df6070

… prepar…

Followup to bitcoin#27676.

7f96638 contrib: add macOS fixup_chains check to security-check (fanquake) 3dca683 build: support -no_fixup_chains in ld64 (fanquake) Pull request description: Followup to #27676, adding the check for chained fixups. Somewhat annoyingly, we have to patch support for `-no_fixup_chains` into ld64. As it doesn't seem to have been added [until a later version](https://github.com/apple-oss-distributions/ld64/blob/59a99ab60399c5e6c49e6945a9e1049c42b71135/src/ld/Options.cpp#L4172). Guix Build: ```bash 0e17d462808f86aa7157e27a957da88fd1adeb491ad6c01138aca93e5ad1d018 guix-build-7f96638723a0/output/arm64-apple-darwin/SHA256SUMS.part ceb208e6374f5d7367b73128e90ca6eaeea15d50c69e49c8cf75b47212525ad7 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.dmg e31663554cfde8a37a9f3438c9c895dde94b90ff87e28f12f78be71ef6421d93 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.tar.gz 68a7bbc42418641eab391a85725b5c2f3c46d38a7acc07e7a8cef98909be07ec guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin.tar.gz 38d966ad93e7384f4f1ce16faded003a675ecce7be1987e6c4eee8e4b82c0432 guix-build-7f96638723a0/output/dist-archive/bitcoin-7f96638723a0.tar.gz 9d314f595d897a715a321a9fba0d552220fbd4bf69aff84eb8c0001cdb48234f guix-build-7f96638723a0/output/x86_64-apple-darwin/SHA256SUMS.part c218ebfd0e96348c4912e6d522492b621bb043ef45b75105ff1fde979d1004d0 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.dmg 1c5ff7fa82f5c76d7d8b9582ad5202f4a82a917102ecafdc3c1fb7b783f6bc3e guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.tar.gz 15fb01e5afcc842db6a3e793b42c70c05ce07bec79e0d2d605e241901ff9f639 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin.tar.gz ``` ACKs for top commit: theuni: utACK 7f96638. hebasto: ACK 7f96638, I have reviewed the code and the patch, and they look OK. TheCharlatan: ACK 7f96638 Tree-SHA512: 7f94710460f54b2afe3c9f5d57107b71436c59b799b15f78e5e3011c3c4f6b23a3acc1008eccea9c22226a200774c82900bad6c6236ab6c5c48a17dec3f2d5a2

7f96638 contrib: add macOS fixup_chains check to security-check (fanquake) 3dca683 build: support -no_fixup_chains in ld64 (fanquake) Pull request description: Followup to bitcoin#27676, adding the check for chained fixups. Somewhat annoyingly, we have to patch support for `-no_fixup_chains` into ld64. As it doesn't seem to have been added [until a later version](https://github.com/apple-oss-distributions/ld64/blob/59a99ab60399c5e6c49e6945a9e1049c42b71135/src/ld/Options.cpp#L4172). Guix Build: ```bash 0e17d462808f86aa7157e27a957da88fd1adeb491ad6c01138aca93e5ad1d018 guix-build-7f96638723a0/output/arm64-apple-darwin/SHA256SUMS.part ceb208e6374f5d7367b73128e90ca6eaeea15d50c69e49c8cf75b47212525ad7 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.dmg e31663554cfde8a37a9f3438c9c895dde94b90ff87e28f12f78be71ef6421d93 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.tar.gz 68a7bbc42418641eab391a85725b5c2f3c46d38a7acc07e7a8cef98909be07ec guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin.tar.gz 38d966ad93e7384f4f1ce16faded003a675ecce7be1987e6c4eee8e4b82c0432 guix-build-7f96638723a0/output/dist-archive/bitcoin-7f96638723a0.tar.gz 9d314f595d897a715a321a9fba0d552220fbd4bf69aff84eb8c0001cdb48234f guix-build-7f96638723a0/output/x86_64-apple-darwin/SHA256SUMS.part c218ebfd0e96348c4912e6d522492b621bb043ef45b75105ff1fde979d1004d0 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.dmg 1c5ff7fa82f5c76d7d8b9582ad5202f4a82a917102ecafdc3c1fb7b783f6bc3e guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.tar.gz 15fb01e5afcc842db6a3e793b42c70c05ce07bec79e0d2d605e241901ff9f639 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin.tar.gz ``` ACKs for top commit: theuni: utACK 7f96638. hebasto: ACK 7f96638, I have reviewed the code and the patch, and they look OK. TheCharlatan: ACK 7f96638 Tree-SHA512: 7f94710460f54b2afe3c9f5d57107b71436c59b799b15f78e5e3011c3c4f6b23a3acc1008eccea9c22226a200774c82900bad6c6236ab6c5c48a17dec3f2d5a2

… prepare for building with upstream LLVM 3df6070 contrib: remove macOS lazy_bind check (fanquake) 9bc357e build: explicitly opt-in to new fixup_chains functionality for darwin (Cory Fields) fb61bc0 depends: Bump MacOS minimum runtime requirement to 11.0 (Cory Fields) c2cd472 depends: bump darwin clang to 11.1 (Cory Fields) Pull request description: This (I believe) resolves the last of the blockers for [switching us away from cctools and instead using out-of-the-box llvm and lld](bitcoin#21778) for building Darwin binaries. For now, we continue building with a pre-packaged llvm and cctools, but after this PR the clang+lld combo should just work for anyone trying it. Additionally after this PR, the new runtime `fixup_chains` behavior will be in-use, as ld64 uses it as well. The commits may seem unrelated, so in detail: lld (llvm's linker) has been a work-in-progress for Darwin for years. Recently though, it has gained nearly all of the features we require. The last missing feature from ld64, `-Wl,-bind_at_load`, is not implemented in lld; as far as I can tell [lazy loading has conceptually been replaced by fixup chains](https://www.emergetools.com/blog/posts/iOS15LaunchTime). So that means we don't need ld64's `bind_at_load` as long as lld can handle `-Wl,-fixup_chains` (which it can). I've added it to our configure as a linker option mostly so that we can see it in the logs; it's default-on as long as the minimum version is >11.0. About that: the runtime functionality required for `-Wl,-fixup_chains` [requires macOS >=11.0](https://github.com/llvm/llvm-project/blob/release/16.x/lld/MachO/Driver.cpp#L1021). Hence the commit that bumps the minimum version. Our current min runtime of `10.15` has been unsupported since September 2022, so I don't expect this bump to be controversial. Lastly, with the minimum runtime version bumped to 11.0, our current version of pre-compiled clang we use for macOS is too old to understand `-mmacosx-version-min=11.0` because it expects `=10.x`. So I've made the smallest possible bump (from 10.0.1 to 11.1.0) to a version that understands. This bump is arbitrary and unfortunate, but likely to be short-lived as we may end up replacing it with llvm+lld for v26 anyway. I've held off on bumping the SDK as I think that makes sense to do as part of the lld switch instead. ACKs for top commit: hebasto: ACK 3df6070 gruve-p: ACK bitcoin@3df6070 fanquake: ACK 3df6070 TheCharlatan: ACK 3df6070 Tree-SHA512: 0200ec4a3b88df33877ae82c15b5c04d745852550923f491a354b391cac65f88e4df116a40055c23a8cbcfcdfb9a376c6ae8fdd0e898e7b966bc213dcb5857cf

theuni force-pushed the macos-min-bump branch from d1d825d to 9623a5b Compare May 16, 2023 19:26

theuni mentioned this pull request May 16, 2023

ci: Use macos-ventura-xcode:14.1 image for "macOS native" task #26388

Merged

DrahtBot added the CI failed label May 16, 2023

This was referenced May 17, 2023

build: produce a .zip for macOS distribution #27099

Closed

build: LLD based macOS toolchain #21778

Merged

maflcko added the DrahtBot Guix build requested label May 17, 2023

theuni marked this pull request as draft May 17, 2023 22:11

DrahtBot removed the DrahtBot Guix build requested label May 17, 2023

fanquake reviewed May 19, 2023

View reviewed changes

depends/packages/native_clang.mk Outdated Show resolved Hide resolved

theuni force-pushed the macos-min-bump branch 3 times, most recently from edb6ac0 to 24ce1d6 Compare May 24, 2023 20:39

theuni marked this pull request as ready for review May 24, 2023 20:40

This was referenced May 25, 2023

[POC] guix: produce a fully -static-pie bitcoind #25573

Draft

guix: Pointer Authentication and Branch Target Identification for aarch64 Linux #24123

Draft

fanquake approved these changes Jun 26, 2023

View reviewed changes

TheCharlatan approved these changes Jun 26, 2023

View reviewed changes

hebasto approved these changes Jun 26, 2023

View reviewed changes

maflcko added the DrahtBot Guix build requested label Jun 27, 2023

fanquake merged commit 35b01e8 into bitcoin:master Jun 27, 2023

maflcko removed the DrahtBot Guix build requested label Jun 27, 2023

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jun 27, 2023

Merge bitcoin#27676: macOS: Bump minimum required runtime version and…

abbcb96

… prepar…

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS chained_fixups check to security-check

56a9d11

Followup to bitcoin#27676.

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS fixup_chains check to security-check

f3fd280

Followup to bitcoin#27676.

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS fixup_chains check to security-check

c40de19

Followup to bitcoin#27676.

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS fixup_chains check to security-check

f33bfb6

Followup to bitcoin#27676.

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS fixup_chains check to security-check

7f96638

Followup to bitcoin#27676.

fanquake mentioned this pull request Jun 29, 2023

contrib: add macOS test for fixup_chains usage #27999

Merged

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 29, 2023

contrib: add macOS fixup_chains check to security-check

67e7f2a

Followup to bitcoin#27676.

hebasto mentioned this pull request Sep 22, 2023

commit ecb617fefe drops qt/gui support on MacOS 10.15.7 bitcoin-core/gui#760

Closed

bitcoin locked and limited conversation to collaborators Jun 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macOS: Bump minimum required runtime version and prepare for building with upstream LLVM #27676

macOS: Bump minimum required runtime version and prepare for building with upstream LLVM #27676

theuni commented May 16, 2023

DrahtBot commented May 16, 2023 •

edited

Loading

theuni commented May 16, 2023

fanquake commented May 16, 2023

hebasto commented May 16, 2023

fanquake commented May 16, 2023

theuni commented May 16, 2023

theuni commented May 16, 2023

fanquake commented May 17, 2023

fanquake commented May 17, 2023

theuni commented May 17, 2023

DrahtBot commented May 17, 2023

theuni commented May 18, 2023

theuni commented May 22, 2023

theuni commented May 24, 2023

theuni commented May 24, 2023

theuni commented Jun 23, 2023

fanquake left a comment

hebasto commented Jun 26, 2023

hebasto commented Jun 26, 2023

TheCharlatan commented Jun 26, 2023

hebasto commented Jun 26, 2023

theuni commented Jun 26, 2023

hebasto commented Jun 26, 2023

TheCharlatan commented Jun 26, 2023

TheCharlatan left a comment •

edited

Loading

hebasto commented Jun 26, 2023

tobtoht commented Jun 26, 2023

gruve-p commented Jun 27, 2023

macOS: Bump minimum required runtime version and prepare for building with upstream LLVM #27676

macOS: Bump minimum required runtime version and prepare for building with upstream LLVM #27676

Conversation

theuni commented May 16, 2023

DrahtBot commented May 16, 2023 • edited Loading

Reviews

Conflicts

theuni commented May 16, 2023

fanquake commented May 16, 2023

hebasto commented May 16, 2023

fanquake commented May 16, 2023

theuni commented May 16, 2023

theuni commented May 16, 2023

fanquake commented May 17, 2023

fanquake commented May 17, 2023

theuni commented May 17, 2023

DrahtBot commented May 17, 2023

Guix builds

theuni commented May 18, 2023

theuni commented May 22, 2023

theuni commented May 24, 2023

theuni commented May 24, 2023

theuni commented Jun 23, 2023

fanquake left a comment

Choose a reason for hiding this comment

hebasto commented Jun 26, 2023

hebasto commented Jun 26, 2023

TheCharlatan commented Jun 26, 2023

hebasto commented Jun 26, 2023

theuni commented Jun 26, 2023

hebasto commented Jun 26, 2023

TheCharlatan commented Jun 26, 2023

TheCharlatan left a comment • edited Loading

Choose a reason for hiding this comment

hebasto commented Jun 26, 2023

tobtoht commented Jun 26, 2023

gruve-p commented Jun 27, 2023

DrahtBot commented May 16, 2023 •

edited

Loading

TheCharlatan left a comment •

edited

Loading