rules_python build_file_generation integration tests on Ubuntu using minimum supported Bazel version is failing with "requirements_test on Bazel@HEAD

[sgowroji]

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3423#018b7e52-e06d-4e38-8577-97686d10d343

Platform : rules_python (build_file_generation integration tests on Ubuntu)

Logs:

(02:05:20) FAIL: //:requirements_test (see /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/6a1011ee018bdbf38289288c626fc42e/execroot/_main/bazel-out/k8-fastbuild/testlogs/requirements_test/test.log)
 
FAILED: //:requirements_test (Summary)

Steps:

git clone -v https://github.com/bazelbuild/rules_python.git
git reset 5b082bb02b048a4b630843b6b582bd6f40d38b9e --hard
export USE_BAZEL_VERSION=8cab6aa21fa765985a962e59264e12251e4d479f
bazel test -- ...

Autosheriff culprit : bazelbuild/bazel@32cc97e

CC Greenteam @comius

[rickeylev]

qq: did the system python version change on the build hosts? i.e. from python 3.8 to python 3.9? That looks to be a common diff showing up. (I'm not asking to undo that, it would just explain why this failure shows up).

PR 1522 it out to partially address this -- many of those tests are trying to run bazel at head for the configs that should only be running Bazel 5.4. That will, at the least, skip those configs where it doesn't need to run.

I'm still looking into the root cause.

[rickeylev]

Ah, I think this is because of switching bzlmod on by default. Setting --noenable_bzlmod when running some of the failing tests fixes it.