Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support querying repo mappings #20631

Closed
cameron-martin opened this issue Dec 20, 2023 · 4 comments
Closed

Support querying repo mappings #20631

cameron-martin opened this issue Dec 20, 2023 · 4 comments
Labels
P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@cameron-martin
Copy link
Contributor

Description of the feature request:

Add a way of querying for repo mappings, for usage in external tools (such as language servers). The repo mapping is necessary for these tools to resolve labels to files on disk, since the files live under the canonical repo name.

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

Facebook have a starlark parser & lsp which works pretty great with Bazel. However, it doesn't support bzlmod because it assumes the repo name of a label is the canonical name, i.e. it looks up @foo in $(bazel info output_base)/external/foo.

According to @fmeum, this is needed for intellij too.

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Slack discussion: https://bazelbuild.slack.com/archives/C014RARENH0/p1703107424515999

@fmeum
Copy link
Collaborator

fmeum commented Dec 21, 2023

I started working on this in the past in #18990, but noticed a problems with my implementation: All repositories generated by a module extension share the same repository mapping containing all these repos. If stored naively, this results in memory usage that is quadratic in the number of extension repos, which can easily be in the 1,000s. Thus, I think it's necessary to "intern" the repo mappings by having one array containing the deduplicated mappings and a map that references them by ID, e.g.:

{
  "mappings": [{"foo": "foo~1.2.3, ...}, {"bar": "bar~4.5.6", ...}, ...],
  "repoToMappingIndex": {"foo~1.2.3": 0, ...}
}

Happy to support anyone who wants to pick this up!

@fmeum
Copy link
Collaborator

fmeum commented Dec 21, 2023

CC @mai93 as this format will also be needed to make IntelliJ's "Go to Definition" features work for labels with Bzlmod.

@iancha1992 iancha1992 added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Dec 21, 2023
@Wyverald Wyverald added P1 I'll work on this now. (Assignee required) and removed untriaged labels Jan 10, 2024
copybara-service bot pushed a commit that referenced this issue Jan 25, 2024
`bazel mod dump_repo_mapping` with no arguments is explicitly made an
error so that a new mode that dumps all repository mappings with a
single Bazel invocation can be added later if needed, e.g. to support
IntelliJ's "sync" workflow.

RELNOTES: `bazel mod dump_repo_mapping <canonical repo name>...` returns
the repository mappings of the given repositories in NDJSON. This
information can be used by IDEs and Starlark language servers to resolve
labels with `--enable_bzlmod`.

Work towards #20631

Closes #20686.

PiperOrigin-RevId: 601332180
Change-Id: I828d7c88637bea175e11eccc52c6202f6da4c32c
bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 25, 2024
`bazel mod dump_repo_mapping` with no arguments is explicitly made an
error so that a new mode that dumps all repository mappings with a
single Bazel invocation can be added later if needed, e.g. to support
IntelliJ's "sync" workflow.

RELNOTES: `bazel mod dump_repo_mapping <canonical repo name>...` returns
the repository mappings of the given repositories in NDJSON. This
information can be used by IDEs and Starlark language servers to resolve
labels with `--enable_bzlmod`.

Work towards bazelbuild#20631

Closes bazelbuild#20686.

PiperOrigin-RevId: 601332180
Change-Id: I828d7c88637bea175e11eccc52c6202f6da4c32c
iancha1992 pushed a commit to bazel-io/bazel that referenced this issue Jan 26, 2024
`bazel mod dump_repo_mapping` with no arguments is explicitly made an
error so that a new mode that dumps all repository mappings with a
single Bazel invocation can be added later if needed, e.g. to support
IntelliJ's "sync" workflow.

RELNOTES: `bazel mod dump_repo_mapping <canonical repo name>...` returns
the repository mappings of the given repositories in NDJSON. This
information can be used by IDEs and Starlark language servers to resolve
labels with `--enable_bzlmod`.

Work towards bazelbuild#20631

Closes bazelbuild#20686.

PiperOrigin-RevId: 601332180
Change-Id: I828d7c88637bea175e11eccc52c6202f6da4c32c
github-merge-queue bot pushed a commit that referenced this issue Jan 26, 2024
`bazel mod dump_repo_mapping` with no arguments is explicitly made an
error so that a new mode that dumps all repository mappings with a
single Bazel invocation can be added later if needed, e.g. to support
IntelliJ's "sync" workflow.

RELNOTES: `bazel mod dump_repo_mapping <canonical repo name>...` returns
the repository mappings of the given repositories in NDJSON. This
information can be used by IDEs and Starlark language servers to resolve
labels with `--enable_bzlmod`.

Work towards #20631

Closes #20686.

Commit
59ac9ce

PiperOrigin-RevId: 601332180
Change-Id: I828d7c88637bea175e11eccc52c6202f6da4c32c

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
@meteorcloudy
Copy link
Member

Looks like this one is done! \o/

@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 7.1.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

7 participants