Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add layering_check support for macOS #22475

Closed

Conversation

keith
Copy link
Member

@keith keith commented May 21, 2024

There were 2 things with the previous implementation that needed to be improved here:

  1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
  2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes #22259.

This reverts commit 1f1b4fd.

@keith keith requested a review from meteorcloudy May 21, 2024 18:31
@keith
Copy link
Member Author

keith commented May 21, 2024

second commit here should fix the reason it was reverted. Can we verify before landing?

@github-actions github-actions bot added team-Rules-CPP Issues for C++ rules awaiting-review PR is awaiting review from an assigned reviewer labels May 21, 2024
@keith keith force-pushed the ks/add-layering_check-support-for-macos branch from c8f1902 to 1c99414 Compare May 21, 2024 18:33
@keith keith force-pushed the ks/add-layering_check-support-for-macos branch 2 times, most recently from 7e0129a to e94020c Compare May 21, 2024 22:27
@meteorcloudy
Copy link
Member

meteorcloudy commented May 22, 2024

We disabled the bootstrap test for Intel mac to save presubmit time, probably we should re-enable it in https://github.com/bazelbuild/bazel/blob/master/.bazelci/presubmit.yml#L217

@keith
Copy link
Member Author

keith commented May 22, 2024

You think that should catch it? enabling those 2 here to see

@meteorcloudy
Copy link
Member

One of them should do, those two takes a long time to run, the only difference is the archive format.

@keith keith force-pushed the ks/add-layering_check-support-for-macos branch from ae7c951 to 8bc99be Compare May 23, 2024 15:31
@keith
Copy link
Member Author

keith commented May 23, 2024

added one back!

Copy link
Member

@meteorcloudy meteorcloudy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@meteorcloudy meteorcloudy added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels May 23, 2024
@iancha1992
Copy link
Member

@meteorcloudy @keith do we need to squash the commits because it has a third party change?

cc: @bazelbuild/triage

@meteorcloudy
Copy link
Member

Yeah, unfortunately the third_party/grpc folder is not checked into google3, so we have to merge it to git-on-borg separately. @keith Can you please squash all commits together, it'll allow us to merge the third_party change easier.

@keith keith force-pushed the ks/add-layering_check-support-for-macos branch 2 times, most recently from 10c86cf to de1d2d2 Compare May 24, 2024 17:10
@keith
Copy link
Member Author

keith commented May 24, 2024

done!

@keith
Copy link
Member Author

keith commented May 24, 2024

looks like https://bazel-review.googlesource.com/c/bazel/+/251183 might be stuck

copybara-service bot pushed a commit that referenced this pull request May 24, 2024
There were 2 things with the previous implementation that needed to be improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes #22259.

This reverts commit 1f1b4fd.

Partial commit for third_party/*, see #22475.

Change-Id: I801121e36de1504c17adfa4736c49c88d470fec0
Signed-off-by: Hee Cha <heec@google.com>
There were 2 things with the previous implementation that needed to be improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes bazelbuild#22259.

This reverts commit 1f1b4fd.
@keith keith force-pushed the ks/add-layering_check-support-for-macos branch from de1d2d2 to 13a6f7e Compare May 25, 2024 01:28
@keith
Copy link
Member Author

keith commented May 25, 2024

rebased now that the 3rd party change is in

@keith
Copy link
Member Author

keith commented May 28, 2024

@meteorcloudy should I rebase this one again?

@iancha1992
Copy link
Member

I'm actually working on the import now for this. If the conflict is only with the lockfiles, I'll take care of it. Thanks @keith

@keith
Copy link
Member Author

keith commented May 28, 2024

ok, thanks!

@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label May 28, 2024
@keith
Copy link
Member Author

keith commented May 28, 2024

@bazel-io flag

@keith keith deleted the ks/add-layering_check-support-for-macos branch May 28, 2024 18:40
@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 28, 2024
@keertk
Copy link
Member

keertk commented May 28, 2024

@bazel-io fork 7.2.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 28, 2024
bazel-io pushed a commit to bazel-io/bazel that referenced this pull request May 28, 2024
There were 2 things with the previous implementation that needed to be improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes bazelbuild#22259.

This reverts commit 1f1b4fd.

Closes bazelbuild#22475.

PiperOrigin-RevId: 637969838
Change-Id: I7d4940a820e3741836239493222ba8d06c4d70e4
bazel-io pushed a commit to bazel-io/bazel that referenced this pull request May 28, 2024
There were 2 things with the previous implementation that needed to be improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes bazelbuild#22259.

This reverts commit 1f1b4fd.

Partial commit for third_party/*, see bazelbuild#22475.

Change-Id: I801121e36de1504c17adfa4736c49c88d470fec0
Signed-off-by: Hee Cha <heec@google.com>
github-merge-queue bot pushed a commit that referenced this pull request May 28, 2024
There were 2 things with the previous implementation that needed to be
improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to
the underlying -cc1 invocation, so we have to manually pass them
directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The
macOS SDK has tons of files that aren't headers, and tons of symlinks
pointing to other files within the SDK. This adds a fork in the script
to run a version that works with Apple SDKs. The time difference on my
machine is 41s->6s. 6s is still pretty long so if desired we can put
this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain,
but similar to the Linux setup here the modulemap file includes absolute
paths.

Closes #22259.

This reverts commit 1f1b4fd.

Partial commit for third_party/*, see #22475.

Change-Id: I801121e36de1504c17adfa4736c49c88d470fec0
Signed-off-by: Hee Cha <heec@google.com>

Commit
de0a37c

Signed-off-by: Hee Cha <heec@google.com>
Co-authored-by: Keith Smiley <keithbsmiley@gmail.com>
iancha1992 pushed a commit to bazel-io/bazel that referenced this pull request May 28, 2024
There were 2 things with the previous implementation that needed to be improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to the underlying -cc1 invocation, so we have to manually pass them directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The macOS SDK has tons of files that aren't headers, and tons of symlinks pointing to other files within the SDK. This adds a fork in the script to run a version that works with Apple SDKs. The time difference on my machine is 41s->6s. 6s is still pretty long so if desired we can put this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain, but similar to the Linux setup here the modulemap file includes absolute paths.

Closes bazelbuild#22259.

This reverts commit 1f1b4fd.

Closes bazelbuild#22475.

PiperOrigin-RevId: 637969838
Change-Id: I7d4940a820e3741836239493222ba8d06c4d70e4
github-merge-queue bot pushed a commit that referenced this pull request May 28, 2024
There were 2 things with the previous implementation that needed to be
improved here:

1. Apple Clang has a bug where it doesn't pass module compiler flags to
the underlying -cc1 invocation, so we have to manually pass them
directly to that invocation with -Xclang
2. The previous search script was too aggressive and slow for macOS. The
macOS SDK has tons of files that aren't headers, and tons of symlinks
pointing to other files within the SDK. This adds a fork in the script
to run a version that works with Apple SDKs. The time difference on my
machine is 41s->6s. 6s is still pretty long so if desired we can put
this behavior behind an env var for users to opt in with.

I've added a hermetic version of this to the apple_support toolchain,
but similar to the Linux setup here the modulemap file includes absolute
paths.

Closes #22259.

This reverts commit 1f1b4fd.

Closes #22475.

PiperOrigin-RevId: 637969838
Change-Id: I7d4940a820e3741836239493222ba8d06c4d70e4

Commit
6abdd2a

Co-authored-by: Keith Smiley <keithbsmiley@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Rules-CPP Issues for C++ rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants