chore: migrate to uv, update dependencies, fix test markers and CI#29
Open
nllong wants to merge 3 commits into
Open
chore: migrate to uv, update dependencies, fix test markers and CI#29nllong wants to merge 3 commits into
nllong wants to merge 3 commits into
Conversation
- Migrate pyproject.toml from Poetry to PEP 621 + uv (dependency-groups for dev deps, tool.uv package = false since this is an app, not a library). Drop the unneeded pathlib backport dependency (stdlib covers this on Python 3.12+). - Bump all runtime and dev dependencies to current stable majors (pandas 3.x, requests 2.34, geopandas 1.1, pyarrow 25, pytest 9, mypy 2.x, pre-commit 4.6, etc.) and regenerate uv.lock, replacing poetry.lock. - Fix a pandas 3.0 breaking change (GroupBy.apply no longer retains grouping columns) that was causing the notebook step to fail with KeyError: in.county_name on every dependency-bump CI run; switch to GroupBy.sample() which is version-safe and avoids the issue. - Fix mypy config: it pointed at a nonexistent your_module.py and used python_version = 3.8, so it was silently never checking the real code. Point it at comstock_processor.py/tests/, target 3.12, and relax strict annotations for tests. This surfaced a real bug: process_building_time_series() was annotated to return None but actually returns a tuple of lists; fixed the annotation and added missing type hints. - Connect all tests to CI: 7 of 10 tests in tests/test_comstock_processor.py had no unit/integration marker, so --strict-markers plus -m unit / -m integration in CI silently never ran them. Added the correct marker to every test so the full suite executes. - Update pre-commit config: bump ruff-pre-commit and nbstripout to latest, add astral-sh/uv-pre-commit's uv-lock hook to keep uv.lock in sync. - Update CI workflow to install/run everything through uv (astral-sh/setup-uv) instead of Poetry, which was also failing to install on the Python 3.9 matrix job; drop the unsupported 3.9 job (project requires Python >=3.12) in favor of 3.12/3.13. - Update README installation/testing/commit instructions for uv. Verified locally: uv run pytest -m unit, uv run pytest -m integration (real network downloads), uv run mypy, uv run pre-commit run --all-files, and a full notebook execution all pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Unpinned major-only v9 tag does not exist for astral-sh/setup-uv, only full semver tags like v9.0.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The newer ruff-pre-commit (v0.15.22) reformats the notebook cell I edited after the last local pre-commit run, and warns that the S320 rule referenced in ruff.toml ignore list no longer exists in this ruff version. Fix both so pre-commit passes cleanly in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nllong
force-pushed
the
chore/migrate-to-uv-update-deps
branch
from
July 23, 2026 00:48
cef71d1 to
aeb54f3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates the project's tooling from Poetry to uv, updates all dependencies to current stable releases, wires ruff/uv into pre-commit, and fixes CI so tests actually run.
This was prompted by CI currently failing on every dependency-bump PR (
dependabot/pip/*) — see recent failed runs onmain.What changed
Dependency management: Poetry → uv
pyproject.tomlfrom[tool.poetry]to PEP 621[project]+[dependency-groups], withtool.uv.package = false(this is an app, not a library).pathlibPyPI backport dependency — it's dead weight on Python ≥3.12 (stdlib already providespathlib).poetry.lockwithuv.lock.Dependency updates
Bumped all deps to current majors, e.g. pandas → 3.0.5, requests → 2.34, geopandas → 1.1.4, pyarrow → 25.0.0, pytest → 9.1.1, mypy → 2.3.0, pre-commit → 4.6.1.
Fixed a real breakage caused by the pandas 3.0 bump:
GroupBy.apply()no longer implicitly retains the grouping columns in the sub-frame, which broke01_data_sampling_example.ipynbwithKeyError: 'in.county_name'(confirmed in the failed dependabot CI runs). Switched to the version-safeGroupBy.sample()instead ofgroupby(...).apply(lambda x: x.sample(...)).Tests weren't actually connected to CI
7 of the 10 tests in
tests/test_comstock_processor.pyhad no@pytest.mark.unit/@pytest.mark.integrationmarker. Since CI (andpytest.ini) uses--strict-markerswith-m "unit"/-m "integration", those 7 tests were silently never executed by CI. Added the correct marker to every test so the whole suite runs.mypy config was silently broken
[tool.mypy]pointed at a nonexistentyour_module.pyand usedpython_version = "3.8"for a>=3.12project — mypy was effectively never checking real code. Fixed the config, which surfaced a genuine bug:process_building_time_series()was annotated-> Nonebut actually returns a tuple of lists. Fixed the annotation/type hints; added a per-module override to keep tests loosely typed (standard practice).mypynow passes cleanly.pre-commit
ruff-pre-commit(v0.11.10 → v0.15.22) andnbstripout(0.8.1 → 0.9.1) to latest.astral-sh/uv-pre-commit'suv-lockhook souv.lockalways stays in sync withpyproject.toml.CI
astral-sh/setup-uv+uv sync/uv run. Poetry itself was failing to install on the Python 3.9 job (Installing Poetry ... An error occurred), which is moot anyway since the project requires Python ≥3.12. Swapped the 3.9 job for 3.13 (now testing 3.12/ubuntu, 3.13/ubuntu, 3.12/windows).README
Updated install/testing/pre-commit instructions to use
uvinstead ofpoetry.Validation
All run locally against the new environment (
uv sync --group dev):uv run pytest tests/ -m unit✅uv run pytest tests/ -m integration(real network downloads from the ComStock S3 bucket) ✅ — all 8 passuv run mypy✅ no issuesuv run pre-commit run --all-files✅ all hooks pass (including the newuv-lockhook)jupyter nbconvert --execute) ✅ completes, confirming the pandas 3.0 fix