Add Nox configuration, to run tests locally across Python versions

[moreati]

Currently it's awkward to run the testsuite locally across all supported Python versions, so version specific errors are only found in Github CI jobs.

This change allows the wasmtime-py test suite to be run across all supported CPython versions 3.9 - 3.14 at time of writing. It uses Nox a Python CLI tool for running tests or other build/project management tasks. It's invoked as

• nox or python3 -m nox - run every session defined in noxfile.py
• nox --list - list sessions defined
• nox -s <session> - Run a particular session
• nox -p <python,...> - Run sessions the use the given python version(s)

If desired checks such as MyPy or style can also be included.

[moreati]

Example usage (full log attached nox.log)

➜  wasmtime-py git:(nox) nox --list      
Sessions defined in /Users/alex/src/wasmtime-py/noxfile.py:

* tests-3.9
* tests-3.10
* tests-3.11
* tests-3.12
* tests-3.13
* tests-3.14

sessions marked with * are selected, sessions marked with - are skipped.
➜  wasmtime-py git:(nox) nox         
nox > Running session tests-3.9
nox > Creating virtual environment (uv) using python3.9 in .nox/tests-3-9
nox > /Users/alex/.local/share/uv/tools/nox/bin/uv pip install '.[testing]'
nox > coverage run -m pytest
===================================================================================================== test session starts =====================================================================================================
platform darwin -- Python 3.9.23, pytest-8.4.2, pluggy-1.6.0
rootdir: /Users/alex/src/wasmtime-py
configfile: pytest.ini
plugins: mypy-1.0.1
collected 189 items / 1 skipped                                                                                                                                                                                               

ci/build-rust.py ..                                                                                                                                                                                                     [  1%]
ci/cbindgen.py .                                                                                                                                                                                                        [  1%]
ci/download-wasmtime.py .                                                                                                                                                                                               [  2%]

...

wasmtime/bindgen/generated/types.py .                                                                                                                                                                                   [ 99%]
wasmtime/loader.py .                                                                                                                                                                                                    [100%]
============================================================================================================ mypy =============================================================================================================

Success: no issues found in 153 source files
==================================================================================================== 257 passed in 33.87s =====================================================================================================
nox > Session tests-3.14 was successful in 41 seconds.
nox > Ran 6 sessions in 4 minutes:
nox > * tests-3.9: success, took 22 seconds
nox > * tests-3.10: success, took 49 seconds
nox > * tests-3.11: success, took 51 seconds
nox > * tests-3.12: success, took 48 seconds
nox > * tests-3.13: success, took 46 seconds
nox > * tests-3.14: success, took 41 seconds

[moreati]

This PR is offered in the hope it will be useful. No hard feelings if you prefer not to take it, for any reason or no reason.

If you wish a similar capability, but less Python-centric, then Just might fit the bill. I used Nox because it's a hammer I know.

[alexcrichton]

Thanks! Do you know if uv has a mode similar to cargo +nightly ... which can override which Python version is being used to easily run multiple versions locally? If so I might be more inclined to lean towards that to avoid the need to have an extra dep/runner.

If not, though, would it make sense to integrate this into CI to ensure it stays up-to-date?

[moreati]

To be clear the PR already runs the tests across Python 3.9 - 3.14, dynamically downloading Python builds as necessary. The versions to use are read from pyproject.toml as the single source of truth.

To my knowledge uv doesn't have nightly builds of Python, but it does track alpha and beta releases closely - CPython 3.15.0a1 was released on Oct 14. uv 0.9.3 was out the same day with support. I'm not aware of a version specifier that always means "latest pre-release".

There is an astral-uv/setup-uv that can be used in place of setup-python.

nox can be used to generate a GIthub Actions matrix https://nox.thea.codes/en/stable/cookbook.html#generating-a-matrix-with-github-actions

[moreati]

I think there's a way to haveuv be the only dependency, by adding some metadata to noxfile.py and running it with uv run noxfile.py ... rather than nox ... (though this will still work).

➜  wasmtime-py git:(eliminate-star-imports) ✗ git diff noxfile.py      
diff --git a/noxfile.py b/noxfile.py
index 1c95282..88c5e11 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -1,3 +1,6 @@
+# /// script
+# dependencies = ["nox"]
+# ///
 import nox
 
 PYPROJECT = nox.project.load_toml('pyproject.toml')
@@ -16,3 +19,6 @@ nox.options.default_venv_backend = 'uv|virtualenv'
 def tests(session):
     session.install('.[testing]')
     session.run('coverage', 'run', '-m', 'pytest')
+
+if __name__ == "__main__":
+    nox.main()

➜  wasmtime-py git:(eliminate-star-imports) ✗ uv run noxfile.py -p 3.9
/Users/alex/src/wasmtime-py/noxfile.py:18: FutureWarning: The session 'tests' has already been registered; this will be an error in a future version of nox. Overriding the old session for now.
  @nox.session(python=PYTHONS, tags=["test"])
nox > Running session tests-3.9
nox > Creating virtual environment (uv) using python3.9 in .nox/tests-3-9
nox > /Users/alex/.cargo/bin/uv pip install '.[testing]'
nox > coverage run -m pytest
================================================ test session starts =================================================
platform darwin -- Python 3.9.24, pytest-8.4.2, pluggy-1.6.0
rootdir: /Users/alex/src/wasmtime-py
configfile: pytest.ini
plugins: mypy-1.0.1
collected 250 items / 1 skipped                                                                                      

ci/build-rust.py ..                                                                                            [  0%]
ci/cbindgen.py .                                                                                               [  1%]
...
wasmtime/loader.py .                                                                                           [100%]
======================================================== mypy ========================================================

Success: no issues found in 149 source files
=========================================== 250 passed, 1 skipped in 2.85s ===========================================
nox > Session tests-3.9 was successful in 8 seconds.

[alexcrichton]

Oh I agree that CI is still testing a bunch yeah, I mostly would prefer to avoid adding another layer of abstraction if possible. If it's easy to switch python versions locally I'm not too worried about having a manager to auto-run on all versions. For myself at least I'm content to push something to CI, see a failure on a python version that I wasn't running, but then switch locally to debug.

Otherwise though if this is added I'd want to keep CI in sync in terms of ensuring there's not two locations that python versions are listed to test.