Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give access to a unique module identifier from module extensions and macros #17652

Closed
aherrmann opened this issue Mar 2, 2023 · 1 comment
Closed
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@aherrmann
Copy link
Contributor

aherrmann commented Mar 2, 2023

Description of the feature request:

Please add the capability to obtain a unique identifier for a module in the following contexts:

  • For a module extension to obtain the unique module identifier of a module that uses the module extension. For example, this could take the form of an attribute on the bazel_module objects exposed to a module extension through module_ctx.modules.
  • For a macro to obtain the same unique module identifier of the module out of which the macro is invoked. For example, this could take the form of a function in the native module similar to package_relative_label.
    This should take into account external workspaces generated by a module. For example, if module A defines a module extension that generates an external workspace, then macros invoked out of that external workspace should resolve the calling module identifier to that of module A.

The unique module identifier should be distinct between different versions of the same module introduced by a multi version override.

What underlying problem are you trying to solve with this feature?

I am trying to implement registry in a "hub and spoke model" or "hub repo" for named tags of a module extension, such that a module that requested a tag with a given name, can later gain access to the corresponding external workspace under the provided tag-name.

For example

# @mymod//:MODULE.bazel
module(name = "mymod", version = "1.2.3")

nix_pkg = use_extension("@rules_nixpkgs_core//extensions:packages.bzl", "nix_pkg")
nix_pkg.attr(name = "mytool", ...)
use_repo(nix_pkg, "nixpkgs_packages")

# @mymod//:BUILD.bazel
load("@nixpkgs_packages//:defs.bzl", "get_pkg")

genrule(
   ...
   tools = [get_pkg("mytool")],
)

# @nixpkgs_packages//:defs.bzl

def get_pkg(tag_name):
    module_id = native.calling_module_id()
    return _registry[module_id][tag_name]
    # omitted error handling for clarity

Which operating system are you running Bazel on?

Ubuntu 22.04

What is the output of bazel info release?

release 6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

The aforementioned issues #17493 #17048 describing hub-repos.
Not found, but started this related thread on Bazel Slack.

Any other information, logs, or outputs that you want to share?

As suggested by @fmeum on the related thread on Bazel Slack:

Ideally the ID would be human writable and concise in the sense that it only includes the module version if needed for uniqueness (e.g. my_module in the single version and my_module@1.2.3 in the multi-version case). That way it would look good in error messages and could also be specified by users when they need to refer to a particular module (e.g. to specify an override-like tag for it).

@sgowroji sgowroji added type: feature request team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. untriaged labels Mar 2, 2023
@Wyverald Wyverald added P1 I'll work on this now. (Assignee required) area-Bzlmod Bzlmod-specific PRs, issues, and feature requests and removed untriaged labels Mar 9, 2023
@Wyverald
Copy link
Member

@bazel-io fork 6.2.0

ShreeM01 pushed a commit to ShreeM01/bazel that referenced this issue Mar 23, 2023
Extension authors often want to write some macro and change its behavior depending on which module is using it. For example, for rules_go, they want to look at the `go_deps` tags in a certain module and only allow access to repos generated from those tags.

We do this by introducing two new methods on `native`, callable only during the loading phase. They return the name and version of the module associated with the current repo. If the repo is from WORKSPACE, they return `None`. If the repo is generated by an extension, they return info about the module hosting the extension.

The implementation works by storing the "associated module" information in `RepositoryMappingValue`. I had attempted to store them in `BzlmodRepoRuleValue` or even `RepositoryDirectoryValue`, but those are not the right places since they might happen before we even evaluate the MODULE.bazel file (i.e. for non-registry overrides).

Fixes bazelbuild#17652.

RELNOTES: Added `native.module_name()` and `native.module_version()` to allow BUILD macro authors to acquire information about which Bazel module the current repo is associated with.
PiperOrigin-RevId: 518849334
Change-Id: I06b4bc95b5a57de2412ee02544240b054c708165
ShreeM01 pushed a commit to ShreeM01/bazel that referenced this issue Mar 24, 2023
Extension authors often want to write some macro and change its behavior depending on which module is using it. For example, for rules_go, they want to look at the `go_deps` tags in a certain module and only allow access to repos generated from those tags.

We do this by introducing two new methods on `native`, callable only during the loading phase. They return the name and version of the module associated with the current repo. If the repo is from WORKSPACE, they return `None`. If the repo is generated by an extension, they return info about the module hosting the extension.

The implementation works by storing the "associated module" information in `RepositoryMappingValue`. I had attempted to store them in `BzlmodRepoRuleValue` or even `RepositoryDirectoryValue`, but those are not the right places since they might happen before we even evaluate the MODULE.bazel file (i.e. for non-registry overrides).

Fixes bazelbuild#17652.

RELNOTES: Added `native.module_name()` and `native.module_version()` to allow BUILD macro authors to acquire information about which Bazel module the current repo is associated with.
PiperOrigin-RevId: 518849334
Change-Id: I06b4bc95b5a57de2412ee02544240b054c708165
Wyverald added a commit that referenced this issue Mar 27, 2023
Extension authors often want to write some macro and change its behavior depending on which module is using it. For example, for rules_go, they want to look at the `go_deps` tags in a certain module and only allow access to repos generated from those tags.

We do this by introducing two new methods on `native`, callable only during the loading phase. They return the name and version of the module associated with the current repo. If the repo is from WORKSPACE, they return `None`. If the repo is generated by an extension, they return info about the module hosting the extension.

The implementation works by storing the "associated module" information in `RepositoryMappingValue`. I had attempted to store them in `BzlmodRepoRuleValue` or even `RepositoryDirectoryValue`, but those are not the right places since they might happen before we even evaluate the MODULE.bazel file (i.e. for non-registry overrides).

Fixes #17652.

RELNOTES: Added `native.module_name()` and `native.module_version()` to allow BUILD macro authors to acquire information about which Bazel module the current repo is associated with.
PiperOrigin-RevId: 518849334
Change-Id: I06b4bc95b5a57de2412ee02544240b054c708165
ShreeM01 added a commit that referenced this issue Mar 27, 2023
* Remove BzlmodRepoRuleHelper

The helper logic is only used in the BzlmodRepoRuleFunction so no need to have it in a separate place.
- Remove BzlmodRepoRuleHelper Interface & its implementations
- Refactor BzlmodRepoRuleFunction to use the helper logic
- Update tests and Build files accordingly

PiperOrigin-RevId: 486712547
Change-Id: I9a274a6a0afcc77be56bbe20e0c4d17c41c31c58

* Separate selection from bazel dependency graph

* Created new K/F/V for Bazel dependency graph named "BazelDepGraph"
* Refactored "BazelModuleResolution" to only run resolution (discovery, selection and checks) to create pruned and unpruned graph and not create any dependency value
* BazelDepGraphResolution calls BazelModuleResolution and extracts the dependency graph from it
* Updated tests

PiperOrigin-RevId: 499026445
Change-Id: Id1237f9d09015ffe8987d933431fccfcfa0c0963

* Add native.module_{name,version}

Extension authors often want to write some macro and change its behavior depending on which module is using it. For example, for rules_go, they want to look at the `go_deps` tags in a certain module and only allow access to repos generated from those tags.

We do this by introducing two new methods on `native`, callable only during the loading phase. They return the name and version of the module associated with the current repo. If the repo is from WORKSPACE, they return `None`. If the repo is generated by an extension, they return info about the module hosting the extension.

The implementation works by storing the "associated module" information in `RepositoryMappingValue`. I had attempted to store them in `BzlmodRepoRuleValue` or even `RepositoryDirectoryValue`, but those are not the right places since they might happen before we even evaluate the MODULE.bazel file (i.e. for non-registry overrides).

Fixes #17652.

RELNOTES: Added `native.module_name()` and `native.module_version()` to allow BUILD macro authors to acquire information about which Bazel module the current repo is associated with.
PiperOrigin-RevId: 518849334
Change-Id: I06b4bc95b5a57de2412ee02544240b054c708165

* fix BUILD

* remove test case that was accidentally cherry-picked

---------

Co-authored-by: salma-samy <salmasamy@google.com>
Co-authored-by: kshyanashree <109167932+kshyanashree@users.noreply.github.com>
fweikert pushed a commit to fweikert/bazel that referenced this issue May 25, 2023
Extension authors often want to write some macro and change its behavior depending on which module is using it. For example, for rules_go, they want to look at the `go_deps` tags in a certain module and only allow access to repos generated from those tags.

We do this by introducing two new methods on `native`, callable only during the loading phase. They return the name and version of the module associated with the current repo. If the repo is from WORKSPACE, they return `None`. If the repo is generated by an extension, they return info about the module hosting the extension.

The implementation works by storing the "associated module" information in `RepositoryMappingValue`. I had attempted to store them in `BzlmodRepoRuleValue` or even `RepositoryDirectoryValue`, but those are not the right places since they might happen before we even evaluate the MODULE.bazel file (i.e. for non-registry overrides).

Fixes bazelbuild#17652.

RELNOTES: Added `native.module_name()` and `native.module_version()` to allow BUILD macro authors to acquire information about which Bazel module the current repo is associated with.
PiperOrigin-RevId: 518849334
Change-Id: I06b4bc95b5a57de2412ee02544240b054c708165
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

3 participants