Enable tox-uv extension to speed up virtual environments and package installation by tox

[namurphy]

uv is "an extremely fast Python package installer and resolver, written in Rust" that was released by Astral one month ago today. It is a "drop-in replacement for common pip, pip-tools, and virtualenv commands" that is "10-100 times faster than pip & pip-tools." For example, after installing uv we can do uv pip install astropy instead of pip install astropy.

The tox-uv extension to tox makes it so that uv gets used for the creation of virtual environments used by tox.

I have been actively trying to speed up CI checks in PlasmaPy. 🚀 I just opened PlasmaPy/PlasmaPy#2584 to enable tox-uv. (This is in addition to an earlier PR to cache .tox between runs; PlasmaPy/PlasmaPy#2552.) I did some timings, and found that enabling tox-uv for PlasmaPy's CI reduced the non-pytest/non-cache time of a GitHub workflow from ∼155 s to ∼16 s. ⏩ Because of this order of magnitude improvement, I suggest enabling tox-uv in CI for Astropy.

While uv is new, it is described by Astral as production-ready. uv is presently able to handle the most commonly used pip commands, and the production team is very responsive to bug reports.

I'm not as familiar with Astropy's tox setup, so I'm not sure if enabling tox-uv needs to be done here or in OpenAstronomy/github-actions-workflows.

[pllim]

OpenAstronomy side is mostly handled by @ConorMacBride currently, I think.

Thanks for the idea!

[Cadair]

If you just need to install the package that's configurable without editing the workflows

[pllim]

Here?

astropy/tox.ini

Lines 112 to 114 in 12f287a

	install_command =
	!devdeps: python -I -m pip install
	devdeps: python -I -m pip install -v --pre

[namurphy]

Happy to share! I'm a big fan of making it so that the time it takes to run pytest in GitHub Actions is approximately equal to the amount of time it takes to run pytest. 🙃

[eerovaher]

If we can reduce the astropy CI time that much then we really should.

[Cadair]

https://github-actions-workflows.openastronomy.org/en/stable/tox.html#toxdeps

[pllim]

If it is just a few lines, feel free to open an exploratory PR and check the actual run time. Thanks!

[pllim]

I think @jdavies-st been very interested in test run time lately too, so I ping him here.

[jdavies-st]

I think changing from pip is a huge step. And it's likely one-way, much like the switch to ruff has been from flake8. From the same people, btw. Not without its drawbacks. pip is community supported and semi-official. uv is not. Yet. I would be a little wary.

There are plenty of ways to speed up CI. Build platform-specific wheels, cache them, run tox from the wheels. tox has significant overhead for CI, though it gives lots of advantages that make it worth it.

If you want to speed up things in local testing, don't run tox - test from an editable install if it fits your needs. Don't run the full test suite, etc. Rely on CI to do work for you.

When running tox, installation does take a good percentage of the time. For astropy on my machine that's 24 s for pip and 18 s for uv pip. So a reasonable speedup, but limited because much of the time building an editable install is running the C compiler in astropy. The overhead in tox is much higher. Is the speedup for astropy using the tox plugin more significant than ~6 seconds when running via Github actions?

Agree that an exploratory PR would be interesting.

[neutrinoceros]

Something that was pointed out by @namurphy on Slack, but I don't see mentioned here, is that uv is not just a faster replacement for pip: it also brings interesting features like uv pip compile --resolution=lowest-direct, which would make the oldestdeps job easier to implement/maintain as we wouldn't need to manually duplicate/sync metadata from pyproject.toml. I've been using it in my personal repos (not with tox though), and so far so good.

[jdavies-st]

That's super useful to be able to do oldest dependencies. I wrote something similar for jwst that pulls mininum dependencies from the project dependencies in pyproject.toml, and that eventually ended up as

https://github.com/spacetelescope/minimum_dependencies

Having it be a tox plugin would be nice, though it's also easy to use with tox:

https://github.com/spacetelescope/jwst/blob/1c054d892777b5271fb5c3664b80e99a6518e6a8/tox.ini#L74-L76

I'm surprised astropy doesn't do something like this already, and instead pins depedencies in 2 separate places - tox.ini and pyproject.toml.

[namurphy]

I think changing from pip is a huge step. And it's likely one-way, much like the switch to ruff has been from flake8.

The switch would probably involve having tox-uv installed alongside tox, so going back and forth should be straightforward, like pip install tox ⇔ pip install tox tox-uv. Going back and forth between pip and uv is generally straightforward too: pip install astropy ⇔ uv pip install astropy.

pip is community supported and semi-official. uv is not. Yet. I would be a little wary.

For what it's worth, the last couple of issues I raised on the ruff and uv repos have been addressed pretty quickly, typically on the same with with releases soon after. From podcasts, it sounds like Astral employs ∼8 people to work on ruff, uv, and any other projects.

There are plenty of ways to speed up CI. Build platform-specific wheels, cache them, run tox from the wheels. tox has significant overhead for CI, though it gives lots of advantages that make it worth it.

Indeed! I wrote up some of the things I did to speed up PlasmaPy's CI in PlasmaPy/PlasmaPy#2585. Caching the .tox directory between runs sped up CI times by about a minute!

With regards to the overhead of using tox, I just re-ran a tox environment for PlasmaPy with tox-uv installed, and the overhead was only 1.35 s! (noting that we have no pre-compiled code over there).

When running tox, installation does take a good percentage of the time. For astropy on my machine that's 24 s for pip and 18 s for uv pip. So a reasonable speedup, but limited because much of the time building an editable install is running the C compiler in astropy. The overhead in tox is much higher. Is the speedup for astropy using the tox plugin more significant than ~6 seconds when running via Github actions?

If dependencies need to be resolved every time a GitHub Action is run, then using tox-uv should speed things up considerably!

Agree that an exploratory PR would be interesting.

I probably won't get to this for a while since I'm not sure if this needs to be done in Astropy's GitHub Actions or in OpenAstronomy/github-actions-workflows, so if anyone else wants to take this on, please go for it!

[pllim]

I think I would feel better if I see it in action in a smaller Astropy repo first. Maybe asdf-astropy?