Skip to content

bzlmod: Pip dependencies with hyphen cannot be added as dependency #952

@acr92

Description

@acr92

🐞 bug report

Affected Rule

Is this a regression?

Not sure

Description

There's an issue with pip packages that contains a hyphen.

🔬 Minimal Reproduction

  1. Setup Bazel, bzlmod and rules_python
  2. In the requirements.txt or requirements.in add a dependency on python-dateutil==2.8.2
  3. Create a py_binary that has a dependency on requirement("python-dateutil")
  4. Try to build it

(It may also be required to setup a lockfile, I've done that)

🔥 Exception or Error

Here's the error I get when trying to build:

bazel:106:6: no such package '@[unknown repo 'pip_typing-extensions' requested from @rules_python~0.16.1~pip~pip]//': The repository '@[unknown repo 'pip_python-dateutil' requested from @rules_python~0.16.1~pip~pip]' could not be resolved: No repository visible as '@pip_python-dateutil' from repository '@rules_python~0.16.1~pip~pip' and referenced by '@rules_python~0.16.1~pip~pip//:python_dateutil_pkg'

Here's some part of the output from bazel query @pip//... --output xml in case that helps:

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<query version="2">
    <rule class="alias" location="/home/blystad/.cache/bazel/_bazel_blystad/9cf11555f91a3e41f07118dfd0af8453/external/rules_python~0.16.1~pip~pip/BUILD.bazel:16:6" name="@rules_python~0.16.1~pip~pip//:python_dateutil_data">
        <string name="name" value="python_dateutil_data"/>
        <label name="actual" value="@[unknown repo 'pip_python-dateutil' requested from @rules_python~0.16.1~pip~pip]//:data"/>
        <rule-input name="@[unknown repo 'pip_python-dateutil' requested from @rules_python~0.16.1~pip~pip]//:data"/>
    </rule>
    <rule class="alias" location="/home/blystad/.cache/bazel/_bazel_blystad/9cf11555f91a3e41f07118dfd0af8453/external/rules_python~0.16.1~pip~pip/BUILD.bazel:31:6" name="@rules_python~0.16.1~pip~pip//:six_whl">
        <string name="name" value="six_whl"/>
        <label name="actual" value="@rules_python~0.16.1~pip~pip_six//:whl"/>
        <rule-input name="@rules_python~0.16.1~pip~pip_six//:whl"/>
    </rule>
    <!-- [....] --->
</query>

and from bazel query @@rules_python~0.16.1~pip~pip//...

@pip//:python_dateutil_data
@pip//:python_dateutil_dist_info
@pip//:python_dateutil_pkg
@pip//:python_dateutil_whl
@pip//:six_data
@pip//:six_dist_info
@pip//:six_pkg
@pip//:six_whl

🌍 Your Environment

Operating System:

  
Ubuntu 22.04
  

Output of bazel version:

  
Build label: 6.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Dec 19 15:52:35 2022 (1671465155)
Build timestamp: 1671465155
Build timestamp as int: 1671465155
  

Rules_python version:

  
bazel_dep(name = "rules_python", version = "0.16.1")
  

Anything else relevant?

My workaround for now has been to just vendor these dependencies into my third_party/ folder. It's okay with two dependencies, but if there's a lot more then it'll quickly become annoying.

I see in multiple places that the code is converting hyphen to underscore, e.g. here: https://github.com/bazelbuild/rules_python/blob/72ac64a9966d3e2017393b9280a4d4575c538235/python/pip_install/extract_wheels/bazel.py#L25

I see in the generated BUILD.bazel:

alias(
    name = "python_dateutil_data",
    actual = "@pip_python-dateutil//:data",
)

alias(
    name = "python_dateutil_dist_info",
    actual = "@pip_python-dateutil//:dist_info",
)

alias(
    name = "six_pkg",
    actual = "@pip_six//:pkg",
)

(So the package name doesn't look perfect)

Hopefully it's just missing that in one place (maybe here: https://github.com/bazelbuild/rules_python/blob/b3bf1243926b674d7041fd49da4865c658b4bddb/python/pip_install/pip_repository.bzl#L254-L292 ), and then it's a quick fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions