Require Python 3.13 and build the virtualenv from the checked interpreter

[antiguru]

bin/pyactivate validates the interpreter running it and then hands virtualenv creation to uv venv without naming an interpreter. uv resolves one by its own preference order, which favors uv-managed installs over the system Python, so the virtualenv can end up on a different and older Python than the one the check just accepted. Passing sys.executable removes that second, independent choice and makes the check authoritative. It also aligns the two creation paths, since the venv.create fallback right below already builds from the running interpreter.

That fallback passes clear=True, and uv venv is now given --clear to match. Control reaches this branch only when the virtualenv is missing or its Python will not execute, which is exactly when it should be replaced. Without the flag uv refuses to touch an existing directory and aborts, so a virtualenv left behind without its dep_stamp wedges every later invocation instead of being rebuilt.

The floor moves from 3.10 to 3.13, in bin/pyactivate and in ci/test/lint-main/checks/check-python-version.sh. 3.13 is what the CI builder already runs, since that is the system Python in Debian 13.

How this was found, and why the floor moved with it

On a machine with system Python 3.14.6 and a uv-managed 3.10 present, uv python find returns the 3.10, so that is what the virtualenv was built on. check-pipeline-topics.sh and check-python-files.sh then failed with ImportError: cannot import name 'LiteralString' from 'typing'.

That error is the interesting part. LiteralString needs 3.11, so the tree had already drifted past its own declared floor, in test/pg-cdc-standby/mzcompose.py and test/cluster-spec-sheet/mzcompose.py. check-python-version.sh is meant to catch exactly this, but it only byte-compiles the tree, and compilation does not resolve imports, so a newer-than-floor import passes the check and fails at run time. CI never noticed either, because the builder image has no uv-managed interpreters and runs everything on 3.13.

Fixing only the interpreter selection would have left a 3.10 floor that nothing enforces and that the tree does not honor. Python 3.10 has been security-only for years and reaches end of life in October 2026, so raising it is the honest resolution rather than restoring 3.10 compatibility in those two files.

Raising target-version lets ruff replace datetime.timezone.utc with the datetime.UTC alias and pull LiteralString back from typing, which accounts for all of the mechanical churn in the other files. Ruff trails the floor at py312 because the pinned version predates 3.13 and rejects py313; there is a comment in pyproject.toml saying so.

Verification

• Both previously-broken files fail to import under 3.10 and load cleanly under 3.13, checked by executing the module rather than byte-compiling it.
• Every file this touches import-loads on a clean 3.13 environment built from ci/builder/requirements.txt.
• A fresh bin/pyactivate run builds the virtualenv on the system interpreter rather than a uv-managed one, and deleting dep_stamp from an existing virtualenv now rebuilds it instead of aborting.
• bin/lint passes in full, including check-python-version.sh now exercising 3.13.

Not addressed here

check-python-version.sh still only byte-compiles, so the blind spot that let this drift remains open at the new floor. Resolving imports instead would close it, but importing every mzcompose file has side effects and costs, so it seemed worth deciding separately. The pinned ruff is 0.0.292; upgrading it is what would allow py313 and is likewise its own change.

Release notes

No user-visible changes.

🤖 Posted by Claude Code

[def-]

QA LLM Review

1. MEDIUM -- Floor bump is not enforced against the interpreter that actually runs the tree

bin/pyactivate:31

The version gate only inspects the bootstrap interpreter; an existing misc/python/venv below the new floor is reused silently, so this diff's new from datetime import UTC / from typing import LiteralString imports fail inside it. That makes this the first commit to actually trip the known gap, and every developer whose old interpreter still exists (uv-managed, pyenv, Homebrew multi-version) hits it on the next bin/fmt or bin/mzcompose with no automatic rebuild.

Details

Reproduced with system python3 3.13.5 and a pre-existing venv on uv-managed 3.10.20, dep_stamp present:

• bin/pyactivate -c 'import sys; print(sys.version)' → 3.10.20. The gate passes on 3.13.5, activate_venv only checks that dep_stamp exists and that venv/bin/python -c '' runs (bin/pyactivate:74), and requirements.txt is untouched by this PR so acquire_deps reinstalls nothing.
• from datetime import UTC → ImportError: cannot import name 'UTC' from 'datetime'. Reachable via ci/cleanup/aws.py, ci/mkpipeline.py, test/gcp/mzcompose.py; misc/python/materialize/cli/mz_workload_capture.py previously imported LiteralString from typing_extensions specifically to keep working here.
• Nested invocations do resolve python3 to the venv (line 153 prefixes PATH, so bin/fmt's bin/pyactivate -m black and the lint check scripts re-enter it): fatal: python v3.13.0+ required / hint: you have v3.10.20, which points at the wrong interpreter and does not name the fix. On a stale 3.11/3.12 venv this fatal is the only symptom. bin/lint happens to print a usable hint, but only because can-lint.sh misattributes the failure to broken dependencies.

--clear makes recreation work, so gating the reuse path is enough:

+MIN_HEXVERSION = 0x030D0000
...
     try:
         os.stat(stamp_path)
-        subprocess.check_call([python, "-c", ""])
+        subprocess.check_call(
+            [python, "-c", f"import sys; sys.exit(sys.hexversion < {MIN_HEXVERSION})"]
+        )

Already tracked in MaterializeInc/database-issues#2428.

2. LOW -- Documented Python support policy still says 3.12

doc/developer/guide.md:143

The guide states the minimum supported version as the default Python of the most recent Ubuntu LTS and names 3.12 (24.04), while bin/pyactivate now rejects anything below 3.13. A developer following the guide's Linux advice, system package manager on 24.04, gets 3.12 and is hard-blocked with no hint that the documented floor moved.

Details

The floor now lives in three places with two different values: bin/pyactivate:31 (0x030d0000), check-python-version.sh:21 (3.13, with a keep-in-sync comment pointing only at pyactivate), and this paragraph. If the policy sentence is meant to keep deriving the floor from Ubuntu LTS, it needs the new value and the LTS release it comes from; the PR body justifies 3.13 from the CI builder's Debian 13 instead, which is a different rule than the one documented here.

3. LOW -- --clear still aborts on a venv directory that is not a valid virtualenv

bin/pyactivate:93

uv venv --clear refuses a target that lacks a valid pyvenv.cfg: uv will not clear a directory that is not a virtual environment, with a hint to pass --force. That is a nonzero exit rather than FileNotFoundError, so the venv.create(..., clear=True) fallback does not run and pyactivate dies with a CalledProcessError traceback.

Details

Confirmed against uv 0.12.3 on both an empty-but-junk directory and a venv with pyvenv.cfg removed; --clear --force handles both. This is the interrupted-creation case that the comment at bin/pyactivate:69 calls out ("virtualenv creation is not atomic"), and the new comment claims the flag covers a virtualenv that is "absent or unusable", which holds only for a structurally intact one. Pre-existing, but --force is what actually matches clear=True.

[antiguru]

All three addressed in 4925340. Finding 1 in particular is a good catch, and it is mine: I hit exactly this while developing the branch, deleted the virtualenv by hand to get past it, and then fixed only the creation path.

1. Floor not enforced against the virtualenv. Confirmed. With a 3.10 virtualenv and dep_stamp present, bin/pyactivate returned 3.10.20 under the 3.13 gate, and from datetime import UTC raised ImportError inside it. Taking your suggestion, the liveness probe now reports the version too:

subprocess.check_call(
    [python, "-c", f"import sys; sys.exit(sys.hexversion < {MIN_HEXVERSION})"]
)

A failing probe falls into the existing recreation path, which --clear already made work. Verified in both directions: a 3.10 virtualenv with dep_stamp is now rebuilt onto 3.14.6, and a healthy virtualenv is still reused with no rebuild, which matters because a probe that recreated on every invocation would be worse than the bug.

2. Guide says 3.12. Confirmed, and worse than stale. The rule is "the default Python in the most recent Ubuntu LTS", but that release is now 26.04 "Resolute Raccoon", which ships 3.14. So the sentence named 3.12 while its own rule produced 3.14, and taken literally it would demand a version newer than the 3.13 CI actually runs. The guide now states the minimum directly and explains that it tracks the CI builder image, noting a current LTS satisfies it. You were right that the PR body justified the number from Debian 13 while the guide claimed a different rule; the guide now says what we actually do.

3. --clear on a non-virtualenv directory. Added --force. One correction on the severity: on uv 0.11.23 this is a warning, not an error. uv creates the virtualenv and prints "The --clear option will remove the existing directory ... even though it is not a virtual environment. This will become an error in a future release. Use --force". So it is not currently fatal on that version, though it is on the 0.12.3 you tested and uv says it becomes an error everywhere. The flag is right regardless, and --force is documented as "Allow --clear to remove a non-virtual environment directory", which is what clear=True does. Verified against a directory with junk and no pyvenv.cfg.

The minimum now lives in MIN_HEXVERSION and MIN_VERSION in bin/pyactivate, referenced from check-python-version.sh and the guide, so the gate, the probe, and the messages cannot drift.

On database-issues#2428: the reuse path is now gated, so the specific gap this branch would have tripped is closed. The broader one is not. check-python-version.sh still only byte-compiles, so an import newer than the floor still passes the check and fails at run time, which is how the LiteralString drift reached main in the first place. That remains out of scope here.

🤖 Posted by Claude Code

[def-]

QA LLM Review

1. HIGH -- uv venv --force requires uv 0.11.17+, and the resulting failure escapes the venv.create fallback

bin/pyactivate:116

--force was only added to uv venv in uv 0.11.17 (released 2026-05-28); every earlier uv exits 2 with unexpected argument '--force' found. That surfaces as CalledProcessError, not the FileNotFoundError the venv.create fallback catches, so bin/pyactivate dies with a traceback and every Python entry point in the tree (bin/lint, bin/fmt, bin/mzcompose, the lint checks) is unusable until uv is upgraded or uninstalled.

Details

Verified directly: VenvArgs in crates/uv-cli/src/lib.rs has no force field at uv 0.11.16 and does at 0.11.17. Running the head-commit bin/pyactivate against uv 0.9.10 — the version this repo's own CI builder pins at ci/builder/Dockerfile:72 — fails as described; the same scenario at 60fe117, with --clear alone, succeeds and builds the virtualenv.

Two things make this bite rather than lurk. The new virtualenv version probe puts every developer with an existing sub-3.13 virtualenv onto the recreate branch on their first invocation after this merges, so the flag is exercised on machines that would otherwise never have reached it. And there is no workaround short of touching uv: rm -rf misc/python/venv lands on the same subprocess.check_call, which I confirmed fails identically with no virtualenv present at all.

It also reaches CI. bin/pyactivate runs directly on the mac-aarch64 agents, outside the builder container and therefore without MZ_DEV_CI_BUILDER, at ci/deploy_mz/pipeline.template.yml:45 and the corresponding steps in ci/deploy_mz-debug/ and ci/deploy_mz-deploy/.

Removing the directory ourselves is version-independent and is what the two flags are reaching for anyway. It also makes the --clear-refuses-a-non-virtualenv case moot rather than papering over it with a flag whose availability has to be tracked:

+import shutil
 import subprocess
...
+            shutil.rmtree(venv_dir, ignore_errors=True)
             subprocess.check_call(
-                ["uv", "venv", "--python", sys.executable, "--clear", "--force", venv_dir]
+                ["uv", "venv", "--python", sys.executable, venv_dir]
             )

rmtree inside the try is fine for the uv-is-absent path too: venv.create(..., clear=True) happily creates a directory that does not exist. I ran this patch through the half-finished-virtualenv scenario on both uv 0.9.10 and uv 0.12.3 and it rebuilds cleanly on each.

[antiguru]

Confirmed and fixed in c2cca65. This one is squarely on me, and my previous reply made it worse: I wrote that --force "is right regardless" while reasoning only from the uv on my machine, 0.11.23. That was an untested claim about every other version.

Reproduced exactly as you describe. ci/builder/Dockerfile:72 pins uv 0.9.10, and uv venv --python ... --clear --force there exits with a usage error rather than creating anything:

Usage: uv venv --python <PYTHON> --clear [PATH]

Nonzero exit, so CalledProcessError, so the venv.create fallback never runs. And your point about timing is the part I had not considered: the version probe I added in the previous commit is what puts developers on the recreation path in the first place, so the flag would have been exercised precisely on the machines that had been avoiding it.

I took your patch rather than a narrower fix, because deleting the directory ourselves is the thing both flags were reaching for:

shutil.rmtree(venv_dir, ignore_errors=True)
subprocess.check_call(["uv", "venv", "--python", sys.executable, venv_dir])

It also disposes of finding 3 from your first review properly instead of by flag. A path that exists but is not a virtualenv is simply gone before uv sees it, so there is nothing for --clear and --force to disagree about, and no uv version to track.

Verified against a real uv 0.9.10, not a reasoned argument, by putting a shim ahead of it on PATH. Three scenarios on that version, all of which previously failed: a stale 3.10 virtualenv with dep_stamp, a half-finished directory with no pyvenv.cfg and no dep_stamp, and no directory at all. Each rebuilds onto 3.14.6. Repeated all three on uv 0.11.23, plus a check that a healthy virtualenv is still reused without a rebuild.

The stack on top, #38625 and #38626, has been rebased onto this.

🤖 Posted by Claude Code