Fix CI: remove nbqa hooks, migrate to Python 3.11 + uv, update vulnerable dependencies - #768
Merged
Conversation
The nbqa-black and nbqa-ruff hooks duplicate what the ruff and ruff-format hooks already do for notebooks (types_or includes jupyter), and black disagrees with ruff-format about blank lines in notebook cells. This caused pre-commit.ci to loop endlessly, reformatting 11 notebooks on every run and blocking unrelated PRs (e.g. #767). With the nbqa hooks removed, the full suite passes on a clean checkout with no notebook changes required. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The lock file had been regenerated with Poetry 2.1.1, which narrowed markers to the locking platform and stamped nearly every package with 'platform_system == "Linux" and platform_machine == "aarch64"'. On CI's x86_64 runners poetry therefore installed only ~14 packages, so pre-commit and sphinx-build were missing and the code-check and docs jobs have failed on every run since. Regenerated with Poetry 2.4.1 (same package versions, no updates). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
It targets self-hosted runners (gpu, db) that are no longer available, so the job queues forever and blocks PR checks from completing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Packaging: - Convert pyproject.toml to PEP 621 with uv dependency groups and the hatchling build backend; replace poetry.lock with uv.lock - Require Python 3.11 (single-version window, >=3.11,<3.12) - Bump torch to ^2.0 (resolves 2.13.0) since torch 1.x has no cp311 wheels; torchvision follows - Security-driven updates now resolvable: urllib3 2.7.0, requests 2.34.2, setuptools 80.x (capped <81 for pybtex's pkg_resources), pytest 9, jupyterlab 4.6 - Pin scikit-learn <1.8 (cyclops imports the private _binary_clf_curve, removed in 1.8) and torchmetrics <1.3 (metric tests are written against its 1.2 zero-division reference behavior) CI and tooling: - Rewrite all workflows to use astral-sh/setup-uv and uv sync; drop poetry everywhere (Dockerfile, README, workflows) - Scope pip-audit ignores to vulnerabilities that are unfixable while alibi/alibi-detect cap pillow<11 and transformers<5, and cycquery caps pyarrow<18 Code fixes for new versions: - sklearn 1.7: precision_recall_curve keyword probas_pred -> y_score - torch 2.9+: index with a tuple of slices in torch_distributed - mypy 1.x updates: remove stale type-ignores, untyped-decorator code, import PIL.Image explicitly, restructure MetricCollection.add_metrics - tests: make OMOP querier construction lazy so importing test_slicer no longer requires a database; use a local RNG in _inject_ignore_index so results don't depend on execution order Full pre-commit suite (ruff, mypy, doctest, nbstripout, pytest: 9879 passed) is green locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
amrit110
changed the title
astral-sh/setup-uv publishes no bare v8 major tag, so pin v8.3.2. The docker workflow now only logs in and pushes outside pull requests; PR runs still build the image for validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…erfile apt package - test_torch_distributed: pool.map does not pin task index to the worker's distributed rank, so assert the rank is valid instead of equal to the task index - unit tests workflow: use 'uv run --no-sync' so pytest does not re-sync with default groups; also set default-groups = [] so the dev group (which includes CUDA-only cupy) is opt-in, matching the old poetry behavior - Dockerfile: drop software-properties-common, which no longer exists in Debian trixie and was unused Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 4, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CI has been red on every PR for months, for several compounding reasons:
nbqa-blackandruff-formatdisagree about blank lines in notebook cells, so autofixes oscillated forever (see Fix datetime day filtering #767).platform_system == "Linux" and platform_machine == "aarch64"markers — on x86_64 runners only ~14 of 368 packages installed, sopre-commitandsphinx-buildwere missing.Fix
nbqa-black/nbqa-ruffhooks (ruff/ruff-formatalready cover notebooks viatypes_or: [python, jupyter]).pyproject.toml+ dependency groups, hatchling build backend,uv.lock); remove poetry from all workflows, the Dockerfile, and the README._binary_clf_curveimport), torchmetrics <1.3 (test reference behavior), setuptools <81 (pybtex needspkg_resources).y_scorerename, torch 2.9+ tuple indexing in the distributed backend, mypy fixes, lazy OMOP querier intest_slicer(module import no longer needs a database), deterministic RNG in_inject_ignore_index(test results no longer depend on execution order).Validation
Full local run of the CI-equivalent suite is green: ruff, ruff-format, mypy, doctest, nbstripout, and pytest (9879 passed, 332 skipped, 0 failed).
🤖 Generated with Claude Code