Skip to content

Conversation

@arrdem
Copy link
Collaborator

@arrdem arrdem commented Oct 7, 2025

Fixes #649.

thread 'main' panicked at py/tools/venv_shim/src/main.rs:199:40:
called `Result::unwrap()` on an `Err` value: RunfilesDirIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

There are two issues which combine to produce this problem.

Defect 1 -- venv_shim resolves not enough links

If a user builds a static venv and then creates a link to it, on some sandbox implementations the `realpath` of the `./.venv/bin/python3` interpreter will resolve no links (current behavior) or (previously) resolve all symlinks and produce a file name somewhere in the Bazel execroot which has escaped from the `.runfiles/` tree which bundles together both the venv and the interpreter which it requires to operate.

This is a problem because the runfiles implementations all variously rely on inspecting argv[0] (the current executable name) in order to configure the runfiles tree. When entering the python3 interpreter within the venv within the runfiles tree, argv[0] must contain a *.runfiles/ parent directory in order to configure properly.

Since resolving no links is wrong and resolving all links is also wrong we have to carefully resolve just enough links in the path of the interpreter to identify the .runfiles/ dir.

If there's no intermediate symlink state which contains the .runfiles path segment, then the link as originally created was incorrect and cannot be resolved correctly. If there is such a state, this logic will find it.

Defect 2 -- link.py resolves too many links

By calling `realpath` not `normpath`, the `link.py` script makes the same mistake as above and will create a venv symlink which likely points into the Bazel execroot beyond the limits of the `.runfiles` tree which should serve as a logical sandbox. Switch to `normpath` which canonicalizes but does not resolve links. This preserves the observed `.runfiles/` child location of the virtualenv as the desired destination of the user's link, which allows runfiles-local interpreter identification to operate.

Changes are visible to end-users: yes/no

  • Searched for relevant documentation and updated as needed: yes/no
  • Breaking change (forces users to change their own code or config): yes/no
  • Suggested release notes appear below: yes/no

Test plan

  • Manual testing; please provide instructions so we can reproduce:

On MacOS;

❯ bazel run //examples/py_binary:py_binary.venv
❯ source .examples+py_binary+py_binary.venv/bin/activate
❯ python3
Python 3.9.18 (main, Oct  2 2023, 20:16:14)
[Clang 16.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Could not open PYTHONSTARTUP
FileNotFoundError: [Errno 2] No such file or directory: '/Users/arrdem/.pythonrc'
>>> import site
>>> site.PREFIXES
['/Users/arrdem/Documents/work/aspect/rules_py/.examples+py_binary+py_binary.venv']
>>> import sys
>>> sys.executable
'/Users/arrdem/Documents/work/aspect/rules_py/.examples+py_binary+py_binary.venv/bin/python3'

Also hand-tested on Linux to the same effect, which is the source of the manual current_executable() behavior.

@aspect-workflows
Copy link

aspect-workflows bot commented Oct 7, 2025

Test

All tests were cache hits

46 tests (100.0%) were fully cached saving 1m 34s.

@github-actions
Copy link

github-actions bot commented Oct 7, 2025

e2e/use_release folder: LCOV of commit 3b2a533 during CI #2053

Summary coverage rate:
  lines......: 100.0% (2 of 2 lines)
  functions..: 100.0% (1 of 1 function)
  branches...: no data found

Files changed coverage rate: n/a

@arrdem arrdem marked this pull request as ready for review October 7, 2025 19:21
@arrdem arrdem requested a review from myrrlyn October 7, 2025 22:20
@arrdem arrdem enabled auto-merge (squash) October 9, 2025 03:18
@arrdem arrdem force-pushed the arrdem/resolve-runfiles-through-symlinks branch from 8d080e5 to 6d314ef Compare October 9, 2025 04:34
@arrdem arrdem force-pushed the arrdem/resolve-runfiles-through-symlinks branch from 6d314ef to 3b2a533 Compare October 9, 2025 04:41
@arrdem arrdem merged commit fb83c4b into main Oct 9, 2025
16 checks passed
@arrdem arrdem deleted the arrdem/resolve-runfiles-through-symlinks branch October 9, 2025 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: interpreter from venv fails with RunfilesDirIoError

3 participants