chore(development): Switch from poetry to uv #152
Conversation
|
@millin Would like to hear if you have any thoughts. The improvements are already visible in this PR's CI run: It's taking less than a minute to install |
| - name: Code formatting with black | ||
| run: poetry run black --check . | ||
| - name: Code formatting with ruff | ||
| run: uv run ruff format --check . |
There was a problem hiding this comment.
Also getting rid of black. Sometimes we use ruff, sometimes black. This is just consolidating under one tool.
| If you would like to dig a bit into the internals of *airflow-dbt-python* and learn about the development philosophy, it's recommended that you check out the :ref:`how_does_it_work` documentation. | ||
|
|
||
| Project management: Poetry | ||
| Project management: *uv* |
There was a problem hiding this comment.
Parts of this doc are somewhat out of date. I've done some changes here, but a few more are required, I think.
There was a problem hiding this comment.
A lot of the pain that came with resolving dependencies really goes away with uv.
|
|
||
| "Intended Audience :: Developers", | ||
|
|
||
| "License :: OSI Approved :: MIT License", |
There was a problem hiding this comment.
License doesn't change! I just got an error when building, as apparently license is now only under the license key (which we already had, and hasn't changed).
| "black>=22", | ||
| "boto3-stubs[s3]>=1.26.8", | ||
| "coverage[toml]>=7.2", | ||
| # docutils 0.21 causes an error with poetry (https://github.com/python-poetry/poetry/issues/9293) |
There was a problem hiding this comment.
TODO: This may not be needed anymore.
|
I'll ship this as I think the benefits outweigh any costs. There are some doc updates pending but CI should be a breeze. |
|
great improvement! |
Problem
Poetry has always been very slow when resolving dependencies. This was always a pain as both airlfow and dbt are giant packages with a lot of potential conflicts. Combined with the fact that our test matrix includes multiple airflow, dbt, and python combinations, this was making builds take too long.
Changes
uv has recently come up as an alternative to Poetry. It promises to be very fast, and trying it out locally it seems to deliver on those promises. This PR intends to swap Poetry for uv, but I will first monitor the performance of CI to see if the change is worth it. As a baseline, the latest release took more than an hour to build, and about 50 minutes of that was resolving dependencies for Python 3.9/3.10.
The massive line diff is due to the lockfile changes! Do not panic.