ci(coverage): scope 70% floor to monolithic run, not per-shard#31
Merged
Conversation
The 70% coverage floor set in 7e37296 was inherited by every per-shard pytest job via pyproject.toml `addopts`. Each shard only exercises its own source, so total coverage across `[tool.coverage.run] source` never gets close to 70% (ebird shard bottoms out at ~19% because noaa/usgs/ databox internals all report 0%). Every merge has been red since. Two changes: 1. Shard jobs (`tests-core`, `tests-ebird`, `tests-noaa`, `tests-usgs`) now override `addopts` to drop `--cov-fail-under=70` while keeping `--cov --cov-report=term-missing` so the coverage artifact is still visible in the log. 2. New `tests-all` job runs `uv run pytest` across the whole workspace and inherits the unchanged `--cov-fail-under=70`. This is the one place the floor is enforced, and it's the one place where every source is actually under test. Local run: 75.86% total. No code behavior change. Preserves the 70% floor intent from 7e37296 while making it actually enforceable. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… take effect Follow-up to the previous commit on this branch. Dropping `--cov-fail-under` from the pytest `addopts` override isn't enough: pytest-cov invokes `coverage.Coverage().report()` which reads `fail_under` directly from `[tool.coverage.report]` in pyproject.toml regardless of whether the CLI flag is present. That's why shard jobs still exited 1 after the previous commit (4 tests passed, coverage.py then raised CoverageException). Removing `fail_under = 70` here keeps the floor intact: the `tests-all` job still passes `--cov-fail-under=70` via the inherited `addopts`, and pytest-cov enforces that flag directly. Also regenerates `docs/dictionary/analytics/platform_health.md` — it was stale on origin/main (missing `main._dlt_loads` and `main.events` as externals), breaking the `generate_docs.py --check` docs job. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…pend Running the whole workspace in one pytest session triggers pytest-recording VCR cassette mismatches because cassette matching is global in-process — a noaa test's request can match an ebird/usgs cassette loaded earlier in the session. Each shard is self-consistent but the combined run isn't. Switch `tests-all` to: coverage erase coverage run --append -m pytest <shard_1> coverage run --append -m pytest <shard_2> ... coverage report --fail-under=70 Per-shard pytest invocations isolate VCR state; `--append` accumulates coverage across them. Local sim: 119 tests pass, 76% total coverage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CI has been red on every merge since
7e37296(feat(ci): strict mypy per-package + 70% coverage floor). The global--cov-fail-under=70inpyproject.toml[tool.pytest.ini_options] addoptsis inherited by every per-shard pytest job, and each shard only exercises its own source — noaa/usgs/databox internals all report 0% in the ebird shard, dragging the total to ~19%.This PR keeps the 70% floor but moves enforcement off the shards:
tests-core,tests-ebird,tests-noaa,tests-usgs) overrideaddoptsto drop--cov-fail-under=70while keeping--cov --cov-report=term-missingso coverage is still visible.tests-alljob runsuv run pytestacross the whole workspace and inherits--cov-fail-under=70— the one place where every source is actually under test. Local run: 75.86% total.No code behavior change. Preserves the 70% floor intent from
7e37296while making it actually enforceable.Test plan
uv run pytestlocally → 75.86% coverage, floor reachedtests-allCI job passes 70% floor🤖 Generated with Claude Code