Replace pip and pip-tools with uv - #3357
Conversation
|
Deployed to stage. The No change in requirements detected, installing from cache part is not what I expected. |
ca372e3 to
e7c3145
Compare
|
OK, the action is working now.
I was expecting it, because I didn't change this line Interestingly, I didn't realize this is only for STAGE. There is no line for production, so what happens there? Does Heroku use |
e7c3145 to
0470f72
Compare
| coverage/ | ||
| docs/_build/ | ||
| docs/venv/ | ||
| docs/*venv/ |
There was a problem hiding this comment.
By default, uv creates a hidden .venv folder. I thought it would be nice to keep covering also venv, since I got bitten by this (same in .gitignore).
The No change in requirements detected, installing from cache refers to the requirements files.
That line also didn't run... because we've had I've changed the setting now and installed again and now they've installed. |
There are no new packages in the requirements though, only removed, so cache should still be valid. |
|
Yeah, I guess it's smart enough in checking changes. |
| COPY requirements/* /app/requirements/ | ||
| RUN pip install -U 'pip>=8' && \ | ||
| pip install --no-cache-dir --require-hashes -r requirements/default.txt -r requirements/dev.txt -r requirements/test.txt -r requirements/lint.txt | ||
| RUN uv pip install -U 'pip>=8' && \ |
There was a problem hiding this comment.
Where is that uv coming from? We need to install it.
There was a problem hiding this comment.
That's a good question. How was this working locally? 🤔
There was a problem hiding this comment.
I don't see pip being install via apt-get above. Maybe it's already included in the base image?
There was a problem hiding this comment.
Yeah:
Sideshow:pontoon mathjazz$ docker run --rm python:3.11-bookworm pip --version
Unable to find image 'python:3.11-bookworm' locally
3.11-bookworm: Pulling from library/python
56c9b9253ff9: Already exists
364d19f59f69: Already exists
843b1d832182: Already exists
a348c2a8d946: Already exists
353de11681b2: Already exists
29b316cac89e: Already exists
e1860d8ed6fa: Already exists
Digest: sha256:157a371e60389919fe4a72dff71ce86eaa5234f59114c23b0b346d0d02c74d39
Status: Downloaded newer image for python:3.11-bookworm
pip 24.0 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
There was a problem hiding this comment.
But this is running uv pip, not pip.
Trying to figure this out, but it's less trivial than I expected (I can install uv and it works, but then it's not available when running make requirements, I think because of permissions)
There was a problem hiding this comment.
I meant that pip is installed in Docker automatically with the base image we're using, so the old RUN pip install... command just worked.
You probably have uv installed locally, but not in Docker?
Maybe this helps?
https://github.com/astral-sh/uv?tab=readme-ov-file#installation
There was a problem hiding this comment.
My comment How was this working locally? 🤔 was about running the code in this PR (e.g. uv pip install) within the Docker image.
When I tried to run make build this morning on a clean system (all builds/images removed), it failed because uv was not found, but I'm convinced I tested thoroughly this code before. AFAIK, there's no easy way to install uv in the Docker build (no SSH access, make shell is not root and I didn't find a quick way to run commands as root).
| && apt-get autoremove -y | ||
|
|
||
| # Install uv | ||
| COPY --from=ghcr.io/astral-sh/uv:0.4.13 /uv /bin/uv |
There was a problem hiding this comment.
I think I tried all possible ways to extract 0.4.13 into a variable (with both ENV and ARG), and failed.
ENV shouldn't be supported, ARG should work, but something this fails (warnings about UV_VERSION being empty, and then an error).
ARG UV_VERSION=0.4.13
FROM ghcr.io/astral-sh/uv:${UV_VERSION} AS uv_image
COPY --from=uv_image /uv /bin/uv
flodolo
left a comment
flodolo
left a comment
There was a problem hiding this comment.
One thing I noticed is that make requirements is really fast now, compare to pip.
|
This branch: => [server server 6/13] RUN uv pip install --system -r requirements/default.txt -r requirements/dev.txt -r requirements/test.txt 11.7s Main: => [server 5/12] RUN pip install -U 'pip>=8' && pip install --no-cache-dir --require-hashes -r requirements/default.txt -r r 55.3s |
|
Great work, this is a huge performance win for automation and local development. One day we'll benefit from it also on production. |
2 participants
As discussed, this needs to be tested on stage to make sure that the post compile step works as expected on Heroku.