Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http_archive patching via bzlmod doesn't use credentials stored in .netrc files #21885

Closed
guw opened this issue Apr 3, 2024 · 4 comments
Closed
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests help wanted Someone outside the Bazel team could own this P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug

Comments

@guw
Copy link
Contributor

guw commented Apr 3, 2024

Description of the bug:

This looks similar to #15286 but seems to be a specific problem when consuming things via proxied/internal bazel registry requiring authentication.

MODULES.bazel:

bazel_dep(name = "rules_oci", version = "1.7.4")
bazel build //....  --registry=https://internal-nexus.server/bazel-central-registy/

WARNING: Download from https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 401 Unauthorized
INFO: Repository rules_oci~ instantiated at:
  <builtin>: in <toplevel>
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/http.bzl:369:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_oci~':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/http.bzl", line 144, column 10, in _http_archive_impl
		patch(ctx, auth = auth)
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 162, column 36, in patch
		patchfile = _download_patch(ctx, patch_url, integrity, auth)
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 85, column 17, in _download_patch
		ctx.download(
Error in download: java.io.IOException: Error downloading [https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch] to /private/var/tmp/_bazel_user/12345/external/rules_oci~/.tmp_remote_patches/module_dot_bazel_version.patch: GET returned 401 Unauthorized
ERROR: <builtin>: fetching http_archive rule //:rules_oci~: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/http.bzl", line 144, column 10, in _http_archive_impl
		patch(ctx, auth = auth)
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 162, column 36, in patch
		patchfile = _download_patch(ctx, patch_url, integrity, auth)
	File "/private/var/tmp/_bazel_user/12345/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 85, column 17, in _download_patch
		ctx.download(
Error in download: java.io.IOException: Error downloading [https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch] to /private/var/tmp/_bazel_user/12345/external/rules_oci~/.tmp_remote_patches/module_dot_bazel_version.patch: GET returned 401 Unauthorized
ERROR: Skipping '//:buildfarm-server': no such package '@@rules_oci~//oci': java.io.IOException: Error downloading [https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch] to /private/var/tmp/_bazel_user/12345/external/rules_oci~/.tmp_remote_patches/module_dot_bazel_version.patch: GET returned 401 Unauthorized
ERROR: no such package '@@rules_oci~//oci': java.io.IOException: Error downloading [https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch] to /private/var/tmp/_bazel_user/12345/external/rules_oci~/.tmp_remote_patches/module_dot_bazel_version.patch: GET returned 401 Unauthorized

Note, curl --netrc works. Thus, somewhere within the bzlmod fetch it seems to loose the authentication info from ~.netrc file.

❯ curl --netrc https://internal-nexus.server/bazel-central-registy/modules/rules_oci/1.7.4/patches/module_dot_bazel_version.patch
===================================================================
--- a/MODULE.bazel
+++ b/MODULE.bazel
@@ -1,9 +1,9 @@
 "bazel-contrib/rules_oci"
 
 module(
     name = "rules_oci",
-    version = "0.0.0",
+    version = "1.7.4",
     compatibility_level = 1,
 )
 
 # Minimum 1.36.0 to include https://github.com/aspect-build/bazel-lib/pull/594

Which category does this issue belong to?

Core, External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

7.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@github-actions github-actions bot added team-Core Skyframe, bazel query, BEP, options parsing, bazelrc team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Apr 3, 2024
@guw guw changed the title http_archive via bzlmod doesn't use credentials stored in .netrc files http_archive patching via bzlmod doesn't use credentials stored in .netrc files Apr 3, 2024
@iancha1992 iancha1992 removed the team-Core Skyframe, bazel query, BEP, options parsing, bazelrc label Apr 3, 2024
@meteorcloudy
Copy link
Member

This is definitely a bug, but I did some investigation, it looks like the IndexRegistry uses the same downloadManager set here. So it should respect the netrc file.

@meteorcloudy meteorcloudy added P2 We'll consider working on this in future. (Assignee optional) area-Bzlmod Bzlmod-specific PRs, issues, and feature requests help wanted Someone outside the Bazel team could own this and removed untriaged labels Apr 16, 2024
@meteorcloudy
Copy link
Member

Oh, I see, the problem is with the remote_patches URLs. We missed getting proper authentication for them at https://cs.opensource.google/bazel/bazel/+/master:tools/build_defs/repo/http.bzl;l=137

@meteorcloudy
Copy link
Member

@bazel-io fork 7.2.0

bazel-io pushed a commit to bazel-io/bazel that referenced this issue Apr 17, 2024
…atch files

Fixes bazelbuild#21885

PiperOrigin-RevId: 625651940
Change-Id: Id8a29c429e3b283dd0e2eecd1e13c023f0feb357
github-merge-queue bot pushed a commit that referenced this issue Apr 17, 2024
…remote patch files (#22035)

Fixes #21885

PiperOrigin-RevId: 625651940
Change-Id: Id8a29c429e3b283dd0e2eecd1e13c023f0feb357

Commit
445dfab

Co-authored-by: Googler <pcloudy@google.com>
Kila2 pushed a commit to Kila2/bazel that referenced this issue May 13, 2024
…atch files

Fixes bazelbuild#21885

PiperOrigin-RevId: 625651940
Change-Id: Id8a29c429e3b283dd0e2eecd1e13c023f0feb357
@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 7.2.0 RC1. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.2.0rc1. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests help wanted Someone outside the Bazel team could own this P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug
Projects
None yet
Development

No branches or pull requests

5 participants