Depending on submodules in same package produces cycle #721
Description
🐞 bug report
Affected Rule
pip_install
Is this a regression?
Not to my knowledge
Description
etils is a collection of several submodules, some of which depend on other submodules in the same project—for example, etils[epath] depends on etils[epy]. When I try to pip_install this package, I get a "cycle in dependency graph" error:
ERROR: /private/var/tmp/_bazel_mike/a96fae0aa1175f22d453871c399bb754/external/my_deps/pypi__etils/BUILD.bazel:22:11: in py_library rule @my_deps//pypi__etils:pypi__etils: cycle in dependency graph:
//:main (bff2ef1b0c2d90d0090e38f4b9f6af6827fa64f9a1afb111defa525de8d438b8)
.-> @my_deps//pypi__etils:pypi__etils (bff2ef1b0c2d90d0090e38f4b9f6af6827fa64f9a1afb111defa525de8d438b8) [self-edge]
`--
Taking a look at the /private/.../pypi__etils/BUILD.bazel file, we see that the specific submodules / "extras" (i.e. [epath], [epy]) have been dropped and the py_library recursively depends on itself (other details from the file elided):
py_library(
name = "pypi__etils",
# ...
deps = ["//pypi__etils","//pypi__importlib_resources","//pypi__zipp"],
)
Note: I have no idea how common or unique this submodule dependency pattern is (I've never seen it before, personally) and so I'm not sure if rules_python "should" support this.
🔬 Minimal Reproduction
# WORKSPACE
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_python",
sha256 = "cdf6b84084aad8f10bf20b46b77cb48d83c319ebe6458a18e9d2cebf57807cdd",
strip_prefix = "rules_python-0.8.1",
url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.8.1.tar.gz",
)
load("@rules_python//python:pip.bzl", "pip_install")
pip_install(
name = "my_deps",
requirements = "//:requirements.txt",
)# BUILD
load("@rules_python//python:defs.bzl", "py_binary")
load("@my_deps//:requirements.bzl", "requirement")
py_binary(
name = "main",
srcs = ["main.py"],
deps = [
requirement("etils"),
]
)# requirements.txt
etils[epath]==0.6.0
# main.py
from etils import epath
path = epath.Path("/path/to/file.txt")
print(path.exists())
print("hello world")🔥 Exception or Error
❯ bazel build :main
ERROR: /private/var/tmp/_bazel_mike/a96fae0aa1175f22d453871c399bb754/external/my_deps/pypi__etils/BUILD.bazel:22:11: in py_library rule @my_deps//pypi__etils:pypi__etils: cycle in dependency graph:
//:main (bff2ef1b0c2d90d0090e38f4b9f6af6827fa64f9a1afb111defa525de8d438b8)
.-> @my_deps//pypi__etils:pypi__etils (bff2ef1b0c2d90d0090e38f4b9f6af6827fa64f9a1afb111defa525de8d438b8) [self-edge]
`--
ERROR: Analysis of target '//:main' failed; build aborted
INFO: Elapsed time: 0.853s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (4 packages loaded, 124 targets configured)
🌍 Your Environment
Operating System:
macOS Monterey 12.4
Output of bazel version:
❯ bazel version
Bazelisk version: development
Build label: 5.1.1
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Apr 8 15:58:49 2022 (1649433529)
Build timestamp: 1649433529
Build timestamp as int: 1649433529
Rules_python version:
0.8.1
sha256=cdf6b84084aad8f10bf20b46b77cb48d83c319ebe6458a18e9d2cebf57807cdd
Anything else relevant?
Closest related issues I found were these two about extras in airflow 2: #411 and #439 However, those were cases of some pretty odd airflow-specific issues that appear to have been legitimate dependency loops. In this case, we have just a single package that is broken up into a hierarchy of submodules that (AFAICT) do not have any loops. Presumably if these were all separate repos/packages there would be no problem here.
Reproducing with pip_install was simplest, but in my actual codebase for work we're using pip_parse and experience the same issue.