Skip to content

✨ feat(discovery): add iter_interpreters for enumeration#71

Merged
gaborbernat merged 7 commits intomainfrom
feat/iter-interpreters
May 5, 2026
Merged

✨ feat(discovery): add iter_interpreters for enumeration#71
gaborbernat merged 7 commits intomainfrom
feat/iter-interpreters

Conversation

@gaborbernat
Copy link
Copy Markdown
Member

@gaborbernat gaborbernat commented May 5, 2026

Closes #65.

Callers wanting every interpreter on the system, rather than the first match, had to abuse get_interpreter's predicate as a side-channel: return False for every candidate so the search never stops, accumulate into a dict keyed by os.path.realpath. 🪤 The workaround missed any interpreter whose binary on PATH is named pypy or graalpy, and surfaced /bin/python and /usr/bin/python as separate entries even though they resolve to the same file.

flowchart TD
    Call(["iter_interpreters(key)"]) --> Mode{key given?}
    Mode -->|"key=None"| Wide["wide mode<br>cpython, pypy, graalpy<br>filenames on PATH and uv"]
    Mode -->|"key='3.12'"| Narrow["narrow mode<br>only python* on PATH<br>(matches get_interpreter)"]

    Wide --> Walk[walk candidate sources]
    Narrow --> Walk

    Walk --> S1["try_first_with"]
    Walk --> S2["running interpreter"]
    Walk --> S3["Windows registry"]
    Walk --> S4["PATH (left to right)"]
    Walk --> S5["uv installs<br>per-OS layout"]

    S1 --> Pipe[per-candidate filters]
    S2 --> Pipe
    S3 --> Pipe
    S4 --> Pipe
    S5 --> Pipe

    Pipe --> D1[realpath dedup]
    D1 --> D2[satisfies spec]
    D2 --> D3[predicate]
    D3 --> Out([yield PythonInfo])

    style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff
    style Mode fill:#d9904a,stroke:#8f5f2a,color:#fff
    style Wide fill:#9f4ad9,stroke:#5f2a8f,color:#fff
    style Narrow fill:#4a90d9,stroke:#2a5f8f,color:#fff
    style Walk fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style S1 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style S2 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style S3 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style S4 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style S5 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style Pipe fill:#c2873a,stroke:#7a4c1f,color:#fff
    style D1 fill:#c2873a,stroke:#7a4c1f,color:#fff
    style D2 fill:#c2873a,stroke:#7a4c1f,color:#fff
    style D3 fill:#c2873a,stroke:#7a4c1f,color:#fff
    style Out fill:#4a9f4a,stroke:#2a6f2a,color:#fff
Loading

iter_interpreters yields every match in discovery order (try_first_with first, then the running interpreter, then the Windows registry, then PATH left-to-right, then UV-managed installs). ✨ Results are deduplicated by the resolved real path of system_executable so symlinked aliases collapse to a single entry, which is the "one yield per distinct install" semantic callers building choosers or version-range pickers usually want. With no spec it broadens the PATH and uv-install regex to every name in the new KNOWN_IMPLEMENTATIONS constant (python, cpython, pypy, graalpy), because an "all interpreters" call that quietly drops PyPy and GraalPy would just shift the gotcha somewhere else.

get_interpreter's narrow regex stays as it was: when no implementation is in the spec, only python* filenames match on PATH. 🛡️ Tools like virtualenv and tox have always read "no implementation in the spec" as "give me CPython" and changing that would be a behaviour shift across every existing caller. The wide-mode regex is opt-in through iter_interpreters only.

While auditing the uv-install glob to extend it for non-CPython, I noticed the existing */bin/python pattern silently finds nothing on Windows, where uv puts python.exe directly under the install root and PyPy at pypy<M>.<N>.exe. 🐛 GraalPy is the lone exception that keeps bin/graalpy.exe on every OS. The new pattern list reflects uv's actual cross-platform layout (sourced from crates/uv-python/src/managed.rs) and is included here as a bugfix changelog entry.

flowchart LR
    Caller(["iter_interpreters(key)"]) --> Mode{"key is None?"}
    Mode -->|"narrow"| N1["*/bin/python"]
    Mode -->|"narrow"| N2["*/python.exe"]
    Mode -->|"wide"| W1["*/bin/pypy*"]
    Mode -->|"wide"| W2["*/bin/graalpy"]
    Mode -->|"wide"| W3["*/pypy*.exe"]
    Mode -->|"wide"| W4["*/bin/graalpy.exe"]

    N1 --> Dedup[/"realpath dedup"/]
    N2 --> Dedup
    W1 --> Dedup
    W2 --> Dedup
    W3 --> Dedup
    W4 --> Dedup

    Dedup --> Interrogate(["subprocess interrogation"])

    style Caller fill:#4a90d9,stroke:#2a5f8f,color:#fff
    style Mode fill:#d9904a,stroke:#8f5f2a,color:#fff
    style N1 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style N2 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
    style W1 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
    style W2 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
    style W3 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
    style W4 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
    style Dedup fill:#c2873a,stroke:#7a4c1f,color:#fff
    style Interrogate fill:#4a9f4a,stroke:#2a6f2a,color:#fff
Loading

The Diátaxis documentation gains coverage in all four quadrants — tutorial, how-to, reference, and explanation — describing iteration order, dedup model, when wide-mode kicks in, and the per-OS uv install layout.

Callers wanting every interpreter on the system, rather than the first match,
had to abuse get_interpreter's predicate as a side-channel: return False for
every candidate so the search never stops, accumulate into a dict keyed by
realpath. The workaround missed any interpreter whose binary on PATH is named
pypy or graalpy, and surfaced /bin/python and /usr/bin/python as separate
entries even though they resolve to the same file.

iter_interpreters yields every match in discovery order, deduplicated by the
resolved real path of system_executable so symlinked aliases and venvs that
symlink to a base interpreter collapse to a single entry. With no spec it
broadens the PATH and uv-install regex to every name in KNOWN_IMPLEMENTATIONS,
because an "all interpreters" call that quietly drops PyPy and GraalPy would
just shift the gotcha. get_interpreter's narrow regex stays as it was so tools
that have always read "no implementation in the spec" as "give me CPython"
keep that behaviour.

Closes #65.
@gaborbernat gaborbernat added the enhancement New feature or request label May 5, 2026
pre-commit-ci Bot and others added 6 commits May 5, 2026 13:41
The original `iter_interpreters` body crossed the C901 complexity threshold
once all the candidate-skip branches were inlined. Splitting the per-spec
loop into `_iter_for_spec` keeps each function under the limit and reads
better as a pipeline.

Also picks up the codespell `behaviour → behavior` rewrite from the en-GB
to en-US dictionary.
Globbing `*/bin/python` against uv's install root only finds anything on
Unix. On Windows uv places the CPython binary at `<root>/python.exe` directly
and PyPy at `<root>/pypy<M>.<N>.exe`, neither under a `bin/` subdirectory.
GraalPy is the lone exception: uv keeps it at `bin/graalpy.exe` on every OS.

Pattern list now matches uv's actual layout per implementation. Globs that
do not match anything are essentially free, so the same list runs on every
host and there is no platform branching to keep in sync.

A summary of the layout has been added to the explanation docs so future
contributors do not have to rediscover this from uv's source.
Three near-identical uv-layout tests (Unix PyPy, Unix GraalPy, Windows
multi-impl) collapse into one parametrized test with explicit IDs.
Predicate and "filters to empty" cases get the same treatment.

A new `uv_dir` fixture isolates the uv environment by pointing
`UV_PYTHON_INSTALL_DIR` at a fresh empty tmp directory, so tests no
longer scan whatever the host has under `~/.local/share/uv`.

PLR0917 joins PLR0913 in the test-file lint ignores: pytest fixtures
naturally bring many parameters and that is not a code-smell.

Adds the bugfix changelog fragment for the Windows uv discovery fix
landed in the previous commit.
Replaces `# pragma: no branch` on the uv interrogation yield with a real
test that exercises the success path. The test stubs `from_exe` to return
a `MagicMock(spec=PythonInfo)` with a unique `system_executable` so the
result survives realpath dedup and shows up in the yielded list.

The pyvenv.cfg copy in the symlink-dedup test is now wrapped in
`contextlib.suppress(FileNotFoundError)` instead of an existence check,
which avoids the conditional that needed pragma'ing while preserving the
behavior on machines that do or do not run from inside a venv.

PEP 440 references in user-facing docs and docstrings now link to
`packaging.python.org/en/latest/specifications/version-specifiers/`,
which is the canonical, maintained source of the spec - PEPs are
historical and the PyPA spec page is what readers should land on.
PEP 514 stays as a PEP link since the Windows registry spec is not
mirrored on packaging.python.org.
Pairs the per-OS layout table with a mermaid flowchart that shows which
glob pattern fires under narrow vs wide mode, where realpath dedup sits
in the pipeline, and where subprocess interrogation happens. Colors
match the existing diagrams in the file: blue for inputs, orange for
decisions and dedup, purple for the wide-mode-only branches, green for
the final action.
@gaborbernat gaborbernat merged commit 00052bd into main May 5, 2026
16 checks passed
@gaborbernat gaborbernat deleted the feat/iter-interpreters branch May 5, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an API for getting *all* interpreters

1 participant