Upgrading python, uv, and dependencies

Every January, we need to upgrade the python version to the latest version. When we upgrade the python version, we also need to upgrade the dependencies of the project to ensure that they are compatible with the new python version.

Steps to upgrade the python version and dependencies:

• Upgrade the python version in the local environment and in the Dockerfile. e.g. 3.12 -> 3.13
• Upgrade uv version
• project dependencies (libs and app)

Avoid duplicate tooling versions across apps. Common packages should be centralized in the pyproject.toml file in the root of the monorepo.

[dependency-groups]
dev = [
  "ruff>=0.4.2,<0.5",
  "mypy>=1.10,<2",
  "pytest>=8,<9"
]

When we upgrade the python version is recommended to check and fix breaking changes first and then upgrade the dependencies. This approach allows us to identify any compatibility issues early on and address them before upgrading all the dependencies and reducing the risk of breaking the code. We should update uv the last, as it is the tool we use to manage the dependencies and virtual environments, and we want to ensure that it is compatible with the new python version before upgrading it. Upgrade the dependencies gradually, starting with the most critical ones, and then upgrading the rest of the dependencies.

Upgrade Python version

Step 1 — (Optional) Upgrade Python on local environment

— Check the current python version python --version — Upgrade python to the latest version brew update brew upgrade brew install python@3.12 — Check the python version again python --version

Step 2 — Update Dockerfile

    ARG PYTHON_VERSION=3.13
    FROM python:${PYTHON_VERSION}-slim-bookworm

Step 3 — Update pyproject.toml (ALL projects)

requires-python = ">=3.13,<4"

Step 4 — Recreate lockfile

uv lock --upgrade

Step 5 — Validate environment

uv sync
uv run python --version

Upgrade uv

##dockerifle# Step 1 — Update Dockerfile

ARG UV_VERSION=0.11.7

FROM ghcr.io/astral-sh/uv:${UV_VERSION} AS uv
FROM python:${PYTHON_VERSION}-slim-bookworm

COPY --from=uv /uv /bin/uv

Step 2 — (Optional) Update locally

curl -Ls https://astral.sh/uv/install.sh | sh

Step 3 — Validate

uv --version

Upgrade dependencies (controlled)

This step is necessary to ensure that the dependencies are compatible with the new python version. It is recommended to upgrade the dependencies selectively (Step 2), starting with the most critical ones, and then upgrading the rest of the dependencies. This approach allows you to identify any compatibility issues early on and address them before upgrading all the dependencies and reducing the risk of breaking the code.

Step 1 — Upgrade selectively (recommended)

uv lock --upgrade-package <package>

Example: uv lock --upgrade-package ruff

In pyproject.toml file, we control the versions of the dependencies e.g. ruff>=0.4.2,<0.5, so when we run the command uv lock --upgrade-package ruff you won't upgrade to the latest version. My recommendation is to upgrade the dependencies one by one updating the pyproject.toml file.

Use the command uv tree to see the dependency tree and check the dependencies that need to be updated. You can use uv tree | grep pytest to check the version of pytest and see if it is compatible with the new python version.

Use pip pages to check the latest version of the dependencies and update the pyproject.toml file accordingly.

Step 2 — Or upgrade everything

uv lock --upgrade

Step 3 — Sync

uv sync

Validate the monorepo

Step 1 — (Optional) Run checks

Run this command for all the applications in the monorepo to check the code style and linting issues, and to fix them if possible.

make check-code APP_PATH=app/ht_indexer
make fix-code APP_PATH=app/ht_indexer

Step 2 — Run typing checks

make type-check APP_PATH=app/ht_indexer

Step 2 — Run tests

make test-all

Build the Docker image and run the container to ensure everything works as expected.

docker compose run --rm ht-indexer-tests
docker compose run --rm solr-query-tests