Starlarkify all repository rules #18285

Wyverald · 2023-05-02T12:50:54Z

We originally wanted to allow native repo rule usage in module extensions (#15412), but that turned out to be surprisingly hard to implement.

Instead, we could just go down the other route: Starlarkify all the repo rules!

local_repository (trivial) (added in 6f254ce)
local_config_platform (trivial; but also kind of unnecessary, since nobody should use it other than @local_config_platform itself) (done in Implement local_config_platform in @platforms platforms#86)
Add some sort of repository_ctx API to acquire a DirectoryListingValue (added in 7.1.0: path.readdir) This would unblock the following:
- new_local_repository (added in 6f254ce)
- android_sdk_repository (now in https://github.com/bazelbuild/rules_android)
- android_ndk_repository (partially done: https://github.com/bazelbuild/rules_android_ndk)

The text was updated successfully, but these errors were encountered:

Wyverald · 2023-05-02T12:52:38Z

cc @ted-xie @ahumesky @kotlaja

fmeum · 2023-05-02T12:53:16Z

@Wyverald Starlarkifying local_config_platform would get us halfway to #8766 (comment), so I think it's worth doing.

ahumesky · 2023-05-02T21:14:44Z

A starkark version of android_ndk_repository is mostly complete here: https://github.com/bazelbuild/rules_android_ndk

Wyverald · 2023-05-04T18:17:02Z

See https://docs.google.com/document/d/17RZKMuMjAIgNfFdrsLhqNsMTQDSS-BBA-S-fCSBPV7M/edit# for the design to address the missing API.

iancha1992 · 2023-11-29T17:19:58Z

@bazel-io fork 7.1.0

See relnotes! Work towards #18285. RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions.

Added `local_repository` and `new_local_repository` as Starlark repo rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They're drop-in replacements for their native counterparts. Also made `local_path_override` be backed by this new Starlark rule, which means that `bazel_tools` will be even more special now -- it's technically not even fetched as a `local_repository` as that would be a cyclic dependency. This allows us to eventually remove all native repo rules altogether. Work towards #18285. RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions.

This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod. Work towards #18285.

aaronmondal · 2024-04-16T15:26:35Z

@Wyverald I tried to use the local repo rules in an extension with 8.0.0-pre.20240401.3 which contains 6f254ce, but it doesn't seem to work.

In extension.bzl I have something like this (same issue for new_local_repository):

load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository")

def _extension_impl(ctx):
    local_repository(name = "foo", path = "../../")
    
myext = module_extension(
    implementation = _extension_impl,
)

In MODULE.bazel something like this:

ext = use_extension("//:extension.bzl", "myext")

use_repo(ext, "foo")

It raises this error telling me that the local.bzl file can't be found. Is this intended behavior because it's a prerelease or could it be a bug?

MODULE.bazel:xx:yy:
  at .../extensions.bzl:6:6: cannot load '@@bazel_tools//tools/build_defs/repo:local.bzl': no such file:
  at .../extensions.bzl:6:6: cannot load '@@bazel_tools//tools/build_defs/repo:local.bzl': no such file

fmeum · 2024-04-16T15:45:04Z

@aaronmondal I can't reproduce this with USE_BAZEL_VERSION=8.0.0-pre.20240401.3. Both in WORKSPACE and MODULE.bazel load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository") works just fine.

aaronmondal · 2024-04-16T16:16:45Z

It looks like i messed something up. I don't know what it was but your right it works. Thanks for checking, I was about to defer this and now it looks like we can get LLVM working with bzlmod after all ❤️

This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod. Work towards #18285.

This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod. Work towards #18285. Closes #21913. PiperOrigin-RevId: 626410163 Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12

Added `local_repository` and `new_local_repository` as Starlark repo rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They're drop-in replacements for their native counterparts. Work towards #18285. RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions. Closes #21681. PiperOrigin-RevId: 615926923 Change-Id: I0cc7355b011751da23f2a3aa189f120ef177e0fe

These were added in 6f254ce but never added to the table of contents. Work towards #18285. PiperOrigin-RevId: 617904579 Change-Id: I062f7d2fb4bcd441db566e4d00ed9d5d9af43886

* Upgrade to `platforms` 0.0.9 * `--host_platform` now defaults to `@bazel_tools//tools:host_platform`, which is an alias of `@platforms//host` * `local_config_platform` (the repo rule) now just outputs a thin wrapper; `@local_config_platform//:host` is an alias of `@platforms//host`, and `@local_config_platform//:constraints.bzl` re-exports `@platforms//host:constraints.bzl` * Removed all test mocks of `local_config_platform`. A follow-up will guard prod usage behind a flag. Work towards #8766 and #18285. PiperOrigin-RevId: 620316973 Change-Id: I6593f62569f31faee69e88a520a0f7f42009e05d

… platform repo rule #21877 happened because, even though we define `@platforms` to be the bundled version shipped with Bazel (which is 0.0.9 on HEAD), people tend to define their own versions of `@platforms` in WORKSPACE. If this ends up being a lower version (very likely), the subsequent `load` to use the `host_platform_repo` repo rule (which is a Starlark version of `local_config_platform`) will fail because that repo rule was only introduced in `platforms` version 0.0.9. To resolve this conundrum, we define a new repo `@internal_platforms_do_not_use` that _also_ points to the bundled version of `@platforms`. Nobody is likely to define this repo, so we can fairly reliably get the Starlark repo rule, and also still allow people to override the actual `@platforms` targets they use. This only affects WORKSPACE users; if `--enable_bzlmod` is set, the `@host_platform` seen by `@bazel_tools` will come from the module extension instead. Work towards #18285. Fixes #21877. Closes #21902. PiperOrigin-RevId: 622238362 Change-Id: I6335ab1bc26ba9ef54473d8a21fc5a8510a3ace5