Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Bzlmod for Bazel #19087

Closed

Conversation

meteorcloudy
Copy link
Member

@meteorcloudy meteorcloudy commented Jul 26, 2023

This PR enables Bzlmod as the default external dependencies system for Bazel.

Changes included:

  • Enable bzlmod by default in .bazelrc.
  • Renamed io_bazel to _main at necessary locations.
  • Added lockfile MODULE.bazel.lock. We should enable --lockfile_mode=error on CI and --lockfile_mode=update for local builds after fixing some issues in the lock file feature, probably with 6.4.0.
  • Replaced distdir with repository cache (http artifact cache) in the offline bootstrap build. This will eventually be replaced by vendoring and packaging dependencies.
  • Replaced --override_repository hack with repository cache (http artifact cache) for running integration tests without network access. This will eventually be replaced by a true shared repository cache.
  • Removed --experimental_repository_cache_hardlinks inside integration test to avoid invalid cross-device link. Because the file system of the sandbox dir is different from the repo cache dir.
  • Moved common CI flags to .bazelrc for better maintenance.
  • Added all Maven coords in MODULE.bazel, patched rules_jvm_external to fix some Bzlmod issues.

Next step:

  • Remove WORKSPACE and clean up distdir_deps, distdir.bzl.

Fixes #18957

@meteorcloudy meteorcloudy marked this pull request as draft July 26, 2023 14:52
@github-actions github-actions bot added the awaiting-review PR is awaiting review from an assigned reviewer label Jul 26, 2023
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 6 times, most recently from 019e7ba to 3f84a7e Compare August 2, 2023 16:40
@meteorcloudy meteorcloudy reopened this Aug 2, 2023
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Aug 2, 2023
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 6 times, most recently from c0fad89 to 8456291 Compare August 10, 2023 06:31
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 10 times, most recently from f389962 to f2378ce Compare September 4, 2023 10:00
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 2 times, most recently from 26cae5c to 4e46ee3 Compare September 5, 2023 10:08
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 2 times, most recently from 1627e1d to be0dcde Compare September 6, 2023 10:14
Avoid hard code rules_cc canonical repo name

Update lockfile

Fix workspace_resolved_test and test_repo cache

Fix naming

Fixes

Fetch test repos concurrently

Reuse repository cache

Fix windows

Fix rules_cc path

Don't use hard link for repo cache inside tests

The flag is causing "Invalid cross-device link" error for some reason

Revert shard for macos_arm64

Update rules_python to match Bzlmod

Update bazel_skylib to match Bzlmod

Add bazel_skylib in workspace suffix (required by rules_python since 0.22.0)

Fix bazel_determinism_test

Update protobuf url to match Bzlmod

Update rules_pkg to match

bazel_determinism_test: use default repo cache

Allow rules_proto tests to access internet

declare local_config_cc

Disable windows_arm64 temporarily

Refactor CI flags

Don't use /Users/pcloudy in bazelrc

Revert "Disable windows_arm64 temporarily"

This reverts commit cba92fd.

Fix bazel_sandboxing_networking_test

Polish MODULE.bazel

pkg_tar: use strip_prefix properly

Update lockfile

Use --experimental_downloader_config to improve stability

Add the original URL as well

Polish

Polish more
@meteorcloudy meteorcloudy force-pushed the bzlmod_by_default branch 3 times, most recently from e47d129 to 72f4a3a Compare September 6, 2023 14:05
@meteorcloudy meteorcloudy changed the title WIP: enable Bzlmod for Bazel Enable Bzlmod for Bazel Sep 6, 2023
@meteorcloudy meteorcloudy marked this pull request as ready for review September 6, 2023 14:33
@github-actions github-actions bot added awaiting-review PR is awaiting review from an assigned reviewer team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Sep 6, 2023
android_ndk_repository/android_ndk_repository/' WORKSPACE
- rm -f WORKSPACE.bak
android_ndk_repository/android_ndk_repository/' WORKSPACE.bzlmod
- rm -f WORKSPACE.bzlmod.bak
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we creating this .bak in the first place if we delete it right away?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just for compatibility with macOS, where sed doesn't support in-place replacing without a .bak file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is nitpicking (so feel free to leave as is), but why not do this weird pattern only on Mac?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know the original reason, but I guess it's just easier to keep them the same so you can search and edit all places just one time when you need some changes. We can clean this up in another PR if necessary.

.bazelci/postsubmit.yml Show resolved Hide resolved
@meteorcloudy
Copy link
Member Author

I'm importing this PR as a CL to review, just in case it needs some tweak internally.

@copybara-service copybara-service bot closed this in 136dae1 Sep 8, 2023
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate Bazel to be built and tested with Bzlmod enabled
2 participants