-
Notifications
You must be signed in to change notification settings - Fork 532
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for keyring auth with bzlmod pip parse #1578
Comments
I'm running into the same issue. Can you elaborate on:
Perhaps you have a demo/example repo that I can look at?
Is that link still valid? I noticed it's not a permalink so as of right now it points to this
During my investigation I found that I have access to
Without Dropping into a debugger shows that
|
I don't have a public example to reproduce unfortunately, but the general gist for I finally worked around this by applying a patch to the rules_python dependency in my
Sorry about that! Fixed to a permalink to the correct attribute |
Awesome, thanks for the elaboration and the link fix! I'm hoping that I can get this addressed. |
@brett-patterson-ent Now that #1827 was merged (well, commit 4a615be was made but it looks like the PR is still open for some reason), you might not need the To use the bazel downloader: pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
+ experimental_extra_index_urls = ["https://example.com/foo/bar/simple"],
+ experimental_index_url = "https://pypi.org/simple",
hub_name = "pypi",
python_version = "3.11",
requirements_lock = "//:requirements_lock.txt", For the credential helper, check out the docs in #1834 |
Is there anything left to do in this issue given the above two PRs are merged? My thinking is that an extra hub repo for keyring can be created and used as extra_deps in the in compile_pip_requirements rule, thus eliminating the need for the patch to rules_python and then bazel downloader can be used as documented here. I'll close the issue for now, but feel free to reopen if there is work to be done. |
I haven't had the time to test it out yet, but I think this should work. Using the bazel downloader sounds like a great solution, we already have a credential helper set up for other use cases. I'll report back once I try it out. Thank you @dougthor42 and @aignas! |
🚀 feature request
Relevant Rules
In
MODULE.bazel
:Description
All of our PyPI access is through a private Artifact Registry repo that requires authentication. We typically do this authentication through the use of
keyring
, which requires the keyring packages to be installed for the Python interpreter that is doing the pip installation.By default, a hermetic installation of Python won't have these packages installed. While it's a bit of a chicken/egg problem, I've made some progress by taking inspiration from rules_python's internal dependencies and fetching the necessary wheels using
http_archive
. I was able to get this working withcompile_pip_requirements
by using theextra_deps
argument so that thedependency_resolver.py
script runs with the keyring libraries available.However, I've been unable to get the same approach working with
pip.parse
so thatwheel_installer.py
runs with these libraries available. Is there a standard way to do this? Or a way to take a similar approach of bundling additional dependencies into the tool's PYTHONPATH? I found that there was a private attribute specifying this.Describe the solution you'd like
A way to pipe down additional dependencies to include in the
_python_path_entries
attribute would work, I think. However, given that the actualpip
installation happens in a subprocess I'm not 100% sure of that solution. I'm open to any different approaches that would be better.Describe alternatives you've considered
It looks like
pip.parse
does let you pass in anenvironment
argument that (I think) ultimately gets set for the pip subprocess inside ofwheel_installer.py
. I experimented with settingPYTHONPATH
there but was unsuccessful.I also did get a setup working with
WORKSPACE
rather than bzlmod where thepip_parse
call uses the system interpreter, but I'd like to use bzlmod and not involve the system Python if at all possible.The text was updated successfully, but these errors were encountered: