Skip to content

CI: Python 3.15 test job is ~3x slower (~137s) than other versions (~45s) #736

Description

@ryancheley

Summary

In the CI workflow (.github/workflows/ci.yml), the "Install dependencies and run tests with tox-uv" step for Python 3.15 consistently takes ~2+ minutes, while every other version finishes in ~20–55s. It's the long pole of the matrix on every PR.

Evidence

Per-version test job durations from a recent green run (run 28987084211):

job duration
test (3.10) 54s
test (3.11) 48s
test (3.12) 48s
test (3.13) 49s
test (3.14) 39s
test (3.14t) 53s
test (3.15) 137s

Where

.github/workflows/ci.yml:

  • Matrix: python-version: ["3.10", ..., "3.14t", "3.15"] with allow-prereleases: true.
  • Step: uvx --with tox-uv tox -e py$(echo ${{ matrix.python-version }} | tr -d '.')
  • setup-uv cache is enabled (enable-cache: true, keyed on uv.lock).

Likely cause (to confirm)

Python 3.15 is a prerelease, so several dependencies with native extensions likely have no prebuilt wheels for 3.15 yet and get built from source under tox-uv (e.g. cryptography, pydantic-core, and other Rust/C-extension deps). The workflow already carries a related note that cryptography doesn't ship 3.15 free-threaded wheels (matrix comment near 3.15t), which supports this theory.

The tox environment build is also likely not hitting the setup-uv cache the same way, since source builds and prerelease resolution differ from the locked wheel set.

Investigation / next steps

  • Confirm the slow phase is dependency building (add timing/verbose output, or inspect uv logs) vs. test execution — the tests themselves run in <2s locally, so it's almost certainly install/build.
  • Identify which packages are compiled from source on 3.15 (uv pip install --verbose / uv's resolution output).
  • Options to speed it up:
    • Cache the built tox env / uv build artifacts across runs for 3.15.
    • Pin/prefer any available 3.15 wheels; wait for upstream wheels.
    • Consider whether 3.15 (prerelease) needs to block every PR, or could run on a reduced cadence (e.g. nightly / main only) until it stabilizes.

Impact

Adds ~1.5 min to every PR's critical path with no added coverage benefit (3.14 already exercises the newest stable behavior). Low urgency, but easy CI-time win.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions