Consider switching to uv for package management

[bachase]

Expanding on #867, I propose migrate to uv for package management over poetry.

For more thorough technical arguments, take a look at these recent posts.

• https://martynassubonis.substack.com/p/python-project-management-primer-a55
• https://www.bitecode.dev/p/a-year-of-uv-pros-cons-and-should
  From first hand experience, the speed, consistency in managing python versions, and ability to add ad-hoc tools or libraries is highly beneficial. There's also just the pure momentum behind uv that it feels like where the puck is headed.

What are reasons not to use uv?

1. Although open-source, uv is built by https://astral.sh/ a VC-backed company. Its unclear how/if there VC goals might impact the support of uv. This is a low risk given uv is open source and could be forked, not to mention we've already adopted ruff for formatting and linting, which is also an astral backed project.
2. dependabot support is still coming (https://docs.astral.sh/uv/guides/integration/dependency-bots/#dependabot). This means automated upgrading of uv.lock file is not available quite yet. But there are workarounds
3. Its another tool for users to learn. But it is well documented and similar to poetry and even pip for many use cases.

What do we do about other Unitary Foundation projects?

For consistency, we should use the same approach for all released software. Looking at https://github.com/search?q=org%3Aunitaryfund%20poetry.lock&type=code, I think we could just restrict to software packages we release and leave research packages as is/optional to migrate.

• ucc (from poetry)
• mitiq (from pip)
• metriq (from poetry)
• metriq-client (from `poetry)

[bachase]

tagging @jordandsullivan @Misty-W @cosenal @WrathfulSpatula @natestemen @nathanshammah @willzeng

Starting this discussion here given it most recently originated in regard to mitiq, but would apply to other libraries too. I'd be willing to stub the work for most of this as well, so don't view that as a blocker.

[vprusso]

I'm in favor of switching to uv. I've been using it for a few projects, and I think it has many favorable benefits over poetry (one of the obvious ones being speed of dependency management).

[natestemen]

Considering we rely on dependabot quite heavily to manage dependencies, that is enough of a blocker for me to not want to use uv at the moment. I have no problems switching to uv down the line, but IMO it makes sense to migrate to a pyproject.toml with poetry first, and then switch to uv once dependabot support is added (if UF decides to switch). The process of migrating from poetry to uv is documented here and looks relatively simple.

[bachase]

That's fair. Just to highlight the workaround mentioned above, its fairly straightforward too

xdslproject/xdsl#3932 is an example of using this workaround. Basically it runs normal dependabot to update the pyproject.toml and then there is another github action specified to detect when dependabot opens a PR, and if so, adds one more commit that updates the uv.lock file to reflect the dependabot updates.

[bachase]

I'll add one other anecdote in favor of uv that come up when working on ucc. As part of unitaryfoundation/ucc#251, I wanted to run py-spy to profile the benchmark runs. But as noted, it doesn't work with the system python, and indeed didn't work with the brew installed version of python that I had been using alongside poetry. I could've installed another version of python, maybe using pyenv, but than that starts to be more overhead to manage that global install relative to the virtual environments on each project I'm working on. So then I could add py-spy as a developer dependency to ucc, but I don't really want to force others to install it. So then I need to make a separate repo which I use as a temporary venv, and then keep track of managing it.

ucc fixes this by (1) installing a specific python version alongside the virtual environment it creates for your project and (2) a command uvx py-spy/uv tool run py-spy which adds that package to your existing virtual environment for the duration of the run, but doesn't alter the core venv that matches the pyproject.toml. So, just a much nicer integrated experience for this workflow compared to poetry.

[bachase]

@jordandsullivan @Misty-W in case you have a POV. Although we just switched to poetry, this profiling use case puts me more strongly in the uv camp. For the current stage of UCC, it wouldn't be too painful to switch.

[jordandsullivan]

I haven't used uv, but looking at the documentation, it is more intuitive to me than poetry, in that you can say
uv venv
then uv pip install your dependencies, then
source .venv/bin/activate
which is closer to my normal venv workflow.

https://docs.astral.sh/uv/pip/environments/#creating-a-virtual-environment

Plus this is hard to say no to:

[bachase]

Closing this for now given #867 decision. Projects can revisit as dependabot/dependabot-core#10478 progresses.