Migrate from pip/setup.py to uv for dependency management by suhaibmujahid · Pull Request #5913 · mozilla/bugbug

suhaibmujahid · 2026-04-03T00:05:11Z

Resolves #5022

Copilot

Pull request overview

Migrates the project from pip/setup.py/requirements files to uv + pyproject.toml (Hatchling) to improve onboarding and lock dependency resolution (Resolves #5022).

Changes:

Replaces root packaging with pyproject.toml (Hatchling), moving dependencies/extras/scripts and removing setup.py/MANIFEST.in/requirements files.
Updates Taskcluster, Dockerfiles, and docs to install/sync dependencies via uv and use uv.lock.
Migrates http_service to its own pyproject.toml + uv.lock and adjusts compose/Docker build contexts accordingly.

Reviewed changes

Copilot reviewed 22 out of 26 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test-requirements.txt	Removes legacy test requirements file (moved into `pyproject.toml` dependency groups).
requirements.txt	Removes legacy runtime requirements file (moved into `pyproject.toml`).
setup.py	Removes legacy setuptools packaging (replaced by Hatchling in `pyproject.toml`).
MANIFEST.in	Removes legacy manifest handling (replaced by Hatch build config).
pyproject.toml	Adds Hatchling project metadata, dependencies, optional deps, dependency groups, and scripts.
README.md	Updates setup instructions to `uv sync` and documents dependency groups/extras + Python version.
scripts/integration_test.sh	Switches http_service install step from `pip install` to `uv sync`.
infra/spawn_pipeline_requirements.txt	Removes legacy spawn pipeline requirements file (moved into dependency group).
infra/dockerfile.base	Uses uv for dependency sync/install during image build.
infra/dockerfile.base-nlp	Uses uv + `--extra nlp` to provision NLP base image deps.
infra/dockerfile.spawn_pipeline	Uses uv + `--only-group spawn-pipeline` to provision spawn pipeline deps.
http_service/setup.py	Removes legacy setuptools packaging for http_service.
http_service/requirements.txt	Removes legacy http_service requirements file (moved into `http_service/pyproject.toml`).
http_service/pyproject.toml	Adds Hatchling-based packaging + deps + local source reference to `bugbug`.
http_service/MANIFEST.in	Removes legacy manifest rule for templates.
http_service/Dockerfile	Switches build/install flow from pip to uv, uses http_service lockfile.
http_service/Dockerfile.bg_worker	Switches build/install flow from pip to uv, uses http_service lockfile.
http_service/docker-compose.yml	Updates build context/dockerfile paths to reflect root context + new Dockerfile locations.
extra-nlp-requirements.txt	Removes legacy NLP extra requirements file (moved into optional deps).
docker-compose.yml	Updates service build contexts/dockerfiles to root context (needed for bind mounts + lockfiles).
.taskcluster.yml	Migrates CI jobs from pip installs to `uv sync`/`uv run`; removes MCP test task.
.pre-commit-config.yaml	Removes `requirements-txt-fixer` hook (requirements files removed).
.github/dependabot.yml	Moves root Python updates from `pip` to `uv` and expands directories.

Comments suppressed due to low confidence (1)

.taskcluster.yml:262

The release dependency list no longer includes the MCP tests task, even though services/mcp/ has its own test suite. This means MCP tests can regress without being caught by the required CI gate. Consider re-adding the MCP test task (or an equivalent check) to the release dependency chain.

              - dependencies:
                  - { $eval: as_slugid("lint_task") }
                  - { $eval: as_slugid("tests_task") }
                  - { $eval: as_slugid("http_tests_task") }
                  - { $eval: as_slugid("frontend_build") }
                  - { $eval: as_slugid("packaging_test_task") }
                  - { $eval: as_slugid("version_check_task") }
                  - { $eval: as_slugid("integration_test") }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.taskcluster.yml

infra/dockerfile.base

infra/dockerfile.spawn_pipeline

scripts/integration_test.sh

Copilot

Pull request overview

Copilot reviewed 24 out of 27 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

.taskcluster.yml:261

The mcp_tests_task was removed from the Taskcluster config, but services/mcp/ (including tests/) still exists in the repo. This looks unrelated to the stated goal of migrating to uv and reduces CI coverage for the MCP service. Please either restore an equivalent MCP test task using uv or document/justify why MCP tests are no longer required.

        - $if: 'tasks_for == "github-push" && head_branch[:10] == "refs/tags/"'
          then:
            $mergeDeep:
              - { $eval: default_task_definition }
              - dependencies:
                  - { $eval: as_slugid("lint_task") }
                  - { $eval: as_slugid("tests_task") }
                  - { $eval: as_slugid("http_tests_task") }
                  - { $eval: as_slugid("frontend_build") }
                  - { $eval: as_slugid("packaging_test_task") }
                  - { $eval: as_slugid("version_check_task") }
                  - { $eval: as_slugid("integration_test") }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pyproject.toml

http_service/pyproject.toml

infra/dockerfile.base-nlp

infra/dockerfile.base

infra/dockerfile.spawn_pipeline

.taskcluster.yml

marco-c · 2026-04-04T13:11:43Z

.github/dependabot.yml

+      - "/"
+      - "/services/mcp"
+      - "/services/reviewhelper-api"


Is there a way to just find all files in any directory, so we don't have to specify and remember to add/remove them when necessary?

We could use /services/* ¹. I will do that.

Footnotes

Dependabot options reference ↩

Done in 779fe79

marco-c · 2026-04-04T13:12:37Z

bugbug/test_scheduling.py

 import shelve
 import shutil
 import struct
+import tomllib


Why was the change in this file necessary?

Because the project's requires-python changed from >=3.10 (in the old setup.py) to >=3.12 (in the new pyproject.toml).

tomllib was added to the stdlib in Python 3.11. With the old >=3.10 constraint, ruff/isort treated it as a third-party import (since it was not exist in 3.10). With >=3.12, ruff knows it's always stdlib, so it expects it sorted with the other stdlib imports.

marco-c · 2026-04-04T13:14:30Z

http_service/pyproject.toml

+    "flask-cors==6.0.2",
+    "gunicorn==25.1.0",
+    "kombu==5.6.2",
+    "marshmallow>=3.18.0",


This was ==4.2.2

We had a non-obvious conflict — pip was only warning about it rather than failing. The chain was: langchain-community==0.4.1 → dataclasses-json==0.6.7 → marshmallow>=3.18.0,<4.0.0. uv's strict resolver caught this transitive conflict that pip silently ignored.

I'll file a follow-up issue to remove langchain-community, as I don't think it's actually needed.

Filed #5914

marco-c · 2026-04-04T13:17:24Z

.taskcluster.yml

Instead of installing uv in tasks, we could use an image with uv here

I was hesitance to do that, but it is a good idea. I will do that.

Done in f1063e9

Convert root package from setup.py to pyproject.toml with uv

9b12aa8

suhaibmujahid requested a review from Copilot April 3, 2026 00:05

Copilot started reviewing on behalf of suhaibmujahid April 3, 2026 00:05 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

suhaibmujahid force-pushed the use-uv branch 2 times, most recently from 5b457c3 to d5737cf Compare April 3, 2026 02:35

suhaibmujahid added 3 commits April 3, 2026 15:32

Convert http_service from setup.py to pyproject.toml with uv

d6f96fb

Update CI configuration and integration test to use uv

4fc0365

Update README and pre-commit config for uv migration

9d0adb1

suhaibmujahid force-pushed the use-uv branch from 3f9885e to 27e82c2 Compare April 4, 2026 01:08

suhaibmujahid added 3 commits April 3, 2026 21:11

Update dependabot config: replace pip with uv ecosystem

fa85abd

Update all Dockerfiles to use uv instead of pip

a1e6c64

Fix the import ordering for tomllib

ee0d66c

suhaibmujahid force-pushed the use-uv branch from 27e82c2 to ee0d66c Compare April 4, 2026 01:37

suhaibmujahid requested a review from Copilot April 4, 2026 01:37

suhaibmujahid marked this pull request as ready for review April 4, 2026 01:38

suhaibmujahid requested a review from marco-c April 4, 2026 01:38

Copilot started reviewing on behalf of suhaibmujahid April 4, 2026 01:38 View session

Copilot AI reviewed Apr 4, 2026

View reviewed changes

suhaibmujahid enabled auto-merge (rebase) April 4, 2026 02:03

marco-c reviewed Apr 4, 2026

View reviewed changes

marco-c previously approved these changes Apr 4, 2026

View reviewed changes

suhaibmujahid mentioned this pull request Apr 4, 2026

Remove langchain-community from dependencies if it's not used #5914

Open

suhaibmujahid dismissed marco-c’s stale review via 6148ec3 April 4, 2026 14:30

suhaibmujahid added 2 commits April 4, 2026 10:47

Use /services/* wildcard in dependabot config

779fe79

Use uv Docker image instead of installing uv in CI tasks

f1063e9

suhaibmujahid force-pushed the use-uv branch from 6148ec3 to f1063e9 Compare April 4, 2026 14:47

suhaibmujahid disabled auto-merge April 4, 2026 15:58

suhaibmujahid merged commit ea5ae2e into mozilla:master Apr 4, 2026
6 checks passed

suhaibmujahid deleted the use-uv branch April 4, 2026 15:58

Conversation

suhaibmujahid commented Apr 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Footnotes

Uh oh!

suhaibmujahid Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

suhaibmujahid Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

suhaibmujahid Apr 4, 2026 •

edited

Loading

suhaibmujahid Apr 4, 2026 •

edited

Loading