Skip to content

chore: migrate to uv, update dependencies, fix test markers and CI#29

Open
nllong wants to merge 3 commits into
mainfrom
chore/migrate-to-uv-update-deps
Open

chore: migrate to uv, update dependencies, fix test markers and CI#29
nllong wants to merge 3 commits into
mainfrom
chore/migrate-to-uv-update-deps

Conversation

@nllong

@nllong nllong commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Migrates the project's tooling from Poetry to uv, updates all dependencies to current stable releases, wires ruff/uv into pre-commit, and fixes CI so tests actually run.

This was prompted by CI currently failing on every dependency-bump PR (dependabot/pip/*) — see recent failed runs on main.

What changed

Dependency management: Poetry → uv

  • Converted pyproject.toml from [tool.poetry] to PEP 621 [project] + [dependency-groups], with tool.uv.package = false (this is an app, not a library).
  • Removed the unneeded pathlib PyPI backport dependency — it's dead weight on Python ≥3.12 (stdlib already provides pathlib).
  • Replaced poetry.lock with uv.lock.

Dependency updates

Bumped all deps to current majors, e.g. pandas → 3.0.5, requests → 2.34, geopandas → 1.1.4, pyarrow → 25.0.0, pytest → 9.1.1, mypy → 2.3.0, pre-commit → 4.6.1.

Fixed a real breakage caused by the pandas 3.0 bump: GroupBy.apply() no longer implicitly retains the grouping columns in the sub-frame, which broke 01_data_sampling_example.ipynb with KeyError: 'in.county_name' (confirmed in the failed dependabot CI runs). Switched to the version-safe GroupBy.sample() instead of groupby(...).apply(lambda x: x.sample(...)).

Tests weren't actually connected to CI

7 of the 10 tests in tests/test_comstock_processor.py had no @pytest.mark.unit/@pytest.mark.integration marker. Since CI (and pytest.ini) uses --strict-markers with -m "unit" / -m "integration", those 7 tests were silently never executed by CI. Added the correct marker to every test so the whole suite runs.

mypy config was silently broken

[tool.mypy] pointed at a nonexistent your_module.py and used python_version = "3.8" for a >=3.12 project — mypy was effectively never checking real code. Fixed the config, which surfaced a genuine bug: process_building_time_series() was annotated -> None but actually returns a tuple of lists. Fixed the annotation/type hints; added a per-module override to keep tests loosely typed (standard practice). mypy now passes cleanly.

pre-commit

  • Bumped ruff-pre-commit (v0.11.10 → v0.15.22) and nbstripout (0.8.1 → 0.9.1) to latest.
  • Added astral-sh/uv-pre-commit's uv-lock hook so uv.lock always stays in sync with pyproject.toml.

CI

  • Replaced the Poetry install/run steps with astral-sh/setup-uv + uv sync/uv run. Poetry itself was failing to install on the Python 3.9 job (Installing Poetry ... An error occurred), which is moot anyway since the project requires Python ≥3.12. Swapped the 3.9 job for 3.13 (now testing 3.12/ubuntu, 3.13/ubuntu, 3.12/windows).

README

Updated install/testing/pre-commit instructions to use uv instead of poetry.

Validation

All run locally against the new environment (uv sync --group dev):

  • uv run pytest tests/ -m unit
  • uv run pytest tests/ -m integration (real network downloads from the ComStock S3 bucket) ✅ — all 8 pass
  • uv run mypy ✅ no issues
  • uv run pre-commit run --all-files ✅ all hooks pass (including the new uv-lock hook)
  • Full notebook execution (jupyter nbconvert --execute) ✅ completes, confirming the pandas 3.0 fix

Nicholas Long and others added 3 commits July 22, 2026 20:48
- Migrate pyproject.toml from Poetry to PEP 621 + uv (dependency-groups
  for dev deps, tool.uv package = false since this is an app, not a
  library). Drop the unneeded pathlib backport dependency (stdlib
  covers this on Python 3.12+).
- Bump all runtime and dev dependencies to current stable majors
  (pandas 3.x, requests 2.34, geopandas 1.1, pyarrow 25, pytest 9,
  mypy 2.x, pre-commit 4.6, etc.) and regenerate uv.lock, replacing
  poetry.lock.
- Fix a pandas 3.0 breaking change (GroupBy.apply no longer retains
  grouping columns) that was causing the notebook step to fail with
  KeyError: in.county_name on every dependency-bump CI run; switch
  to GroupBy.sample() which is version-safe and avoids the issue.
- Fix mypy config: it pointed at a nonexistent your_module.py and
  used python_version = 3.8, so it was silently never checking the
  real code. Point it at comstock_processor.py/tests/, target 3.12,
  and relax strict annotations for tests. This surfaced a real bug:
  process_building_time_series() was annotated to return None but
  actually returns a tuple of lists; fixed the annotation and added
  missing type hints.
- Connect all tests to CI: 7 of 10 tests in
  tests/test_comstock_processor.py had no unit/integration marker, so
  --strict-markers plus -m unit / -m integration in CI silently never
  ran them. Added the correct marker to every test so the full suite
  executes.
- Update pre-commit config: bump ruff-pre-commit and nbstripout to
  latest, add astral-sh/uv-pre-commit's uv-lock hook to keep uv.lock
  in sync.
- Update CI workflow to install/run everything through uv
  (astral-sh/setup-uv) instead of Poetry, which was also failing to
  install on the Python 3.9 matrix job; drop the unsupported 3.9 job
  (project requires Python >=3.12) in favor of 3.12/3.13.
- Update README installation/testing/commit instructions for uv.

Verified locally: uv run pytest -m unit, uv run pytest -m integration
(real network downloads), uv run mypy, uv run pre-commit run
--all-files, and a full notebook execution all pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Unpinned major-only v9 tag does not exist for astral-sh/setup-uv,
only full semver tags like v9.0.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The newer ruff-pre-commit (v0.15.22) reformats the notebook cell I
edited after the last local pre-commit run, and warns that the S320
rule referenced in ruff.toml ignore list no longer exists in this
ruff version. Fix both so pre-commit passes cleanly in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nllong
nllong force-pushed the chore/migrate-to-uv-update-deps branch from cef71d1 to aeb54f3 Compare July 23, 2026 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant