Migrating to the MacOSX11.sdk symlink #10769

Bo98 · 2021-03-03T04:37:51Z

Have you followed the guidelines in our Contributing document?
Have you checked to ensure there aren't other open Pull Requests for the same change?
Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your changes? Here's an example.
Have you successfully run brew style with your changes locally?
Have you successfully run brew typecheck with your changes locally?
Have you successfully run brew tests with your changes locally?

Not to be merged until CLT 12.5 stable has sufficiently rolled out - see below.

With thanks to Apple, CLT 12.5 beta 3 released a few hours ago now includes a MacOSX11.sdk symlink.

Big Sur SDKs have been a moving target, with SDKs renaming from MacOSX11.0.sdk to MacOSX11.1.sdk etc. This is a problem because some SDK paths are baked into some formulae. To combat issues like Homebrew/homebrew-core#67075, we moved to the unversioned MacOSX.sdk as it was the only option for path stability.

However MacOSX.sdk itself will cause problems come macOS 12/CLT 13. Based on how all major CLT releases have gone in the past few years, MacOSX.sdk in CLT 13 will contain the macOS 12 SDK, even when installed on Big Sur. Using the macOS 12 SDK on Big Sur could cause problems when building against system libraries. The exact nature of the problems depends highly on what library changes are made in macOS 12, but you can read #7134 for an overview of how using the Catalina SDK caused significant problems on Mojave, such as brewed git crashing.

The MacOSX11.sdk symlink will solve all the above problems. It has path stability and is guaranteed to contain the Big Sur SDK (we have brew doctor checks for SDK tampering).

This change however will require users to have updated to CLT 12.5, when it is released. Once merged, our bottles will start to contain paths pointing to MacOSX11.sdk and this will not exist on earlier CLT versions. gcc, for example, will not function properly with an outdated CLT once we build new bottles with this SDK change. As such, this should not be merged until we reach a point where we are happy that most users, and CIs in particular, have updated to CLT 12.5. It should ideally however be merged no later than WWDC (likely CLT 13 beta release date), should we decide to be waiting that long. This will be an easier question to answer once we know exactly when CLT 12.5 stable is getting released.

Expected effects to the return value of the MacOS.sdk method (assuming latest versions of CLT & Xcode are installed):

Mojave:
- CLT: no change (returns MacOSX10.14.sdk)
- Xcode: no change (returns MacOSX10.15.sdk)
Catalina:
- CLT: no change (returns MacOSX10.15.sdk)
- Xcode: no change (returns MacOSX.sdk)
Big Sur:
- CLT: returns MacOSX11.sdk (previously MacOSX.sdk)
- Xcode: no change (returns MacOSX.sdk)

(Xcode only ever ships with one SDK, with a versioned symlink to that)

I've not fully verified the above results yet (I've not downloaded Xcode beta) but I will do before this is merged.

BrewTestBot · 2021-03-03T04:38:06Z

Review period will end on 2021-03-04 at 04:37:51 UTC.

Bo98 · 2021-03-03T04:38:47Z

Library/Homebrew/os/mac/xcode.rb

      def detect_version_from_clang_version
-        detect_clang_version&.sub(/^(\d+)00\./, "\\1.")
+        detect_clang_version&.sub(/^(\d+)0(\d)\./, "\\1.\\2.")


I will likely split this into a separate PR as this affects CLT 12.5 support in general: the Clang version in the beta is currently 1205.0.22.6 which does not match the original regex.

Thankfully the Clang version sort of matches the CLT version here (this often isn't the case with many CLT updates), so we still only need one minimum_version value after this substitution tweak.

MikeMcQuaid · 2021-03-03T13:37:21Z

Nice work @Bo98!

As such, this should not be merged until we reach a point where we are happy that most users, and CIs in particular, have updated to CLT 12.5. It should ideally however be merged no later than WWDC (likely CLT 13 beta release date), should we decide to be waiting that long. This will be an easier question to answer once we know exactly when CLT 12.5 stable is getting released.

We can enforce a minimum version of the CLT. We should do this when we merge, IMO.

Big Sur:

CLT: returns MacOSX11.sdk

Just to clarify: what's the before/after here?

Library/Homebrew/os/mac/xcode.rb

Bo98 · 2021-03-03T13:39:18Z

We can enforce a minimum version of the CLT. We should do this when we merge, IMO.

Yes, that's a part of this PR. It's just a question of when exactly to merge.

Just to clarify: what's the before/after here?

Edited the above to clarify this.

Bo98 · 2021-03-03T13:41:20Z

(For example, it would be ideal to let the GitHub Actions macos-latest image update to CLT 12.5 before we enforce the new minimum.)

MikeMcQuaid · 2021-03-03T14:33:47Z

CLT: returns MacOSX11.sdk (previously MacOSX.sdk)

👍🏻

BrewTestBot · 2021-03-04T06:10:39Z

Review period ended.

BrewTestBot · 2021-03-26T00:08:42Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

Bo98 · 2021-05-05T12:29:16Z

Update here: the GitHub Actions macos-11.0 image now has CLT 12.5.

I'll update our self-hosted runners by the end of the week, if I can find a quiet time where there's not a million massive PRs running at the same time.

Bo98 · 2021-05-10T12:24:44Z

CI is now updated.

The brew doctor soft recommendation to update to 12.5 will be included in the 3.1.6 tag today (with the Perl stuff).

I suggest waiting a little time for that to settle and then merge this. I'm looking at brew config in bug reports and a lot are still on 12.4.

carlocab · 2021-05-22T00:37:29Z

Do we have analytics data on which CLT/Xcode version users are using? If we don't, would it be useful to have it?

MikeMcQuaid · 2021-05-31T11:19:28Z

Do we have analytics data on which CLT/Xcode version users are using?

No, don't think so.

If we don't, would it be useful to have it?

Maybe? I think we can be more aggressive about forcing upgrades when it suits us, though.

Bo98 · 2021-08-11T12:29:24Z

We're probably ok to merge this now (after I rebase)? There's surely enough people on CLT 12.5 now and we definitely need to get this merged before Monterey.

MikeMcQuaid

Makes sense to me, nice work @Bo98!

MikeMcQuaid · 2021-08-11T14:21:03Z

Thanks @Bo98!

Bo98 added the do not merge label Mar 3, 2021

BrewTestBot added the waiting for feedback Merging is blocked until sufficient time has passed for review label Mar 3, 2021

Bo98 commented Mar 3, 2021

View reviewed changes

MikeMcQuaid reviewed Mar 3, 2021

View reviewed changes

Library/Homebrew/os/mac/xcode.rb Show resolved Hide resolved

MikeMcQuaid approved these changes Mar 3, 2021

View reviewed changes

BrewTestBot removed the waiting for feedback Merging is blocked until sufficient time has passed for review label Mar 4, 2021

BrewTestBot approved these changes Mar 4, 2021

View reviewed changes

BrewTestBot added the stale No recent activity label Mar 26, 2021

Bo98 added in progress Maintainers are working on this and removed stale No recent activity labels Mar 26, 2021

Bo98 mentioned this pull request Mar 26, 2021

os/mac/xcode: update Clang version parsing for CLT 12.5 #10941

Merged

7 tasks

Bo98 force-pushed the big-sur-sdk branch from 77591de to 73acbc1 Compare March 30, 2021 03:17

MikeMcQuaid approved these changes Mar 30, 2021

View reviewed changes

Bo98 mentioned this pull request May 5, 2021

TSC Discussion: Type Checking in Homebrew #8991

Closed

carlocab mentioned this pull request Jun 9, 2021

Updates for Monterey Xcode & CLT #11508

Merged

MikeMcQuaid removed the in progress Maintainers are working on this label Aug 11, 2021

Bo98 force-pushed the big-sur-sdk branch from 7c680ee to ebef0f1 Compare August 11, 2021 12:41

MikeMcQuaid removed the do not merge label Aug 11, 2021

MikeMcQuaid added the critical Critical change which should be shipped as soon as possible. label Aug 11, 2021

MikeMcQuaid approved these changes Aug 11, 2021

View reviewed changes

carlocab mentioned this pull request Aug 11, 2021

llvm: various improvements Homebrew/homebrew-core#83017

Closed

6 tasks

carlocab approved these changes Aug 11, 2021

View reviewed changes

Bo98 added 3 commits August 11, 2021 14:19

os/mac/sdk: use MacOSX11.sdk symlink

dae1d89

os/mac/xcode: bump minimum CLT version to 12.5

49f789d

os/mac/diagnostic: prevent some double diagostic failures

9686ebe

Bo98 force-pushed the big-sur-sdk branch from ebef0f1 to 9686ebe Compare August 11, 2021 13:19

MikeMcQuaid merged commit bee3e67 into Homebrew:master Aug 11, 2021

Bo98 deleted the big-sur-sdk branch August 11, 2021 14:21

Bo98 mentioned this pull request Sep 3, 2021

dependency_collector: add depends_on :swift #11964

Closed

7 tasks

github-actions bot added the outdated PR was locked due to age label Sep 11, 2021

github-actions bot locked as resolved and limited conversation to collaborators Sep 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrating to the MacOSX11.sdk symlink #10769

Migrating to the MacOSX11.sdk symlink #10769

Bo98 commented Mar 3, 2021 •

edited

BrewTestBot commented Mar 3, 2021

Bo98 Mar 3, 2021 •

edited

MikeMcQuaid commented Mar 3, 2021

Bo98 commented Mar 3, 2021 •

edited

Bo98 commented Mar 3, 2021 •

edited

MikeMcQuaid commented Mar 3, 2021

BrewTestBot commented Mar 4, 2021

BrewTestBot commented Mar 26, 2021

Bo98 commented May 5, 2021

Bo98 commented May 10, 2021

carlocab commented May 22, 2021

MikeMcQuaid commented May 31, 2021

Bo98 commented Aug 11, 2021

MikeMcQuaid left a comment

MikeMcQuaid commented Aug 11, 2021

Migrating to the MacOSX11.sdk symlink #10769

Migrating to the MacOSX11.sdk symlink #10769

Conversation

Bo98 commented Mar 3, 2021 • edited

BrewTestBot commented Mar 3, 2021

Bo98 Mar 3, 2021 • edited

Choose a reason for hiding this comment

MikeMcQuaid commented Mar 3, 2021

Bo98 commented Mar 3, 2021 • edited

Bo98 commented Mar 3, 2021 • edited

MikeMcQuaid commented Mar 3, 2021

BrewTestBot commented Mar 4, 2021

BrewTestBot commented Mar 26, 2021

Bo98 commented May 5, 2021

Bo98 commented May 10, 2021

carlocab commented May 22, 2021

MikeMcQuaid commented May 31, 2021

Bo98 commented Aug 11, 2021

MikeMcQuaid left a comment

Choose a reason for hiding this comment

MikeMcQuaid commented Aug 11, 2021

Bo98 commented Mar 3, 2021 •

edited

Bo98 Mar 3, 2021 •

edited

Bo98 commented Mar 3, 2021 •

edited

Bo98 commented Mar 3, 2021 •

edited