geotiff tests: consolidate VRT missing-sources cluster + _helpers/ foundation#2394
Conversation
…undation (#2393) PR 1 of the 11-PR epic #2390. Foundation: - New _helpers/ with tiff_builders.py (was conftest.py:make_minimal_tiff), tiff_surgery.py (relocated from _tiff_surgery.py), markers.py (was conftest.py: requires_gpu / requires_loopback / requires_integration plus the gpu_available / loopback_available probes). - conftest.py re-exports the helpers and markers so existing `from .conftest import ...` imports keep working. The pytest_collection_modifyitems socketserver hack stays in place until PR 11. VRT missing-sources consolidation: - New vrt/ directory with test_missing_sources.py. - Folds test_vrt_missing_sources_policy_1799.py (eager-only byte band smoke checks) and test_vrt_missing_sources_policy_2367.py (eager-plus-chunked float matrix) into one file. Parametrise over the eager and dask readers with id="eager" / id="dask"; bad-value matrix parametrised over readers and 6 invalid values. No issue numbers in test ids. - test_vrt_missing_sources_default_raise_1843.py stays in place: it covers the internal _vrt.read_vrt entry point and the XRSPATIAL_GEOTIFF_STRICT env-var override, which is out of the consolidated module's surface. Updated direct importers of _tiff_surgery: - test_local_tile_byte_cap_1664.py - test_gpu_tile_byte_cap_2026_05_18.py Updated release_gate_geotiff.rst to cite the new vrt/test_missing_sources.py path. CLUSTER_AUDIT.md maps every old file::test to its new file::test_id; deleted in a follow-up commit on this branch before merge. Verification: - pytest xrspatial/geotiff/tests/vrt/ -v: 22 passed - pytest xrspatial/geotiff/tests/ -x -q: 5706 passed, 68 skipped, 6 xfailed, 1 xpassed
brendancol
left a comment
There was a problem hiding this comment.
PR review for #2394.
The change is a clean tests-only restructure. The new _helpers/ layout is sensible, the conftest re-exports keep every existing import working, and the parametrised VRT module preserves the old eager-byte and eager-plus-dask float coverage that lived across two files. 22 new tests collected and pass; full geotiff suite 5706 passed after the release-gate doc fix.
A few items below worth a follow-up commit before merge.
Blockers
None.
Suggestions
-
xrspatial/geotiff/tests/CLUSTER_AUDIT.md:75claims "the nettest_*.pycount drop at exactly 2", but the actual drop is 357 -> 356. The PR body has the correct -2/+1 accounting; the audit file should match. Either fix the wording or drop the verification bullet -- the file is going away before merge anyway, but until then it should not state a false invariant. -
xrspatial/geotiff/tests/vrt/test_missing_sources.py:79-115:_write_partial_float_vrtwrites the VRT file via plainopen(..., 'w')and uses no indentation in the XML, while_write_byte_missing_vrt(a few lines up) usesPath.write_textwith two-space indentation. Both are valid VRT; the inconsistency is purely cosmetic from copy-pasting two different sources. Worth aligning so the file does not look like two helpers written by different people. -
xrspatial/geotiff/tests/conftest.py:18-22: the import block pulls_HAS_LOOPBACK(a leading-underscore private name) out of_helpers/markers.pyjust sopytest_collection_modifyitemscan branch on it. Cleaner would be to callloopback_available()once at the top ofpytest_collection_modifyitemsand stash the result, or export the resolved boolean under a non-underscore name (e.g.HAS_LOOPBACK). The current pattern works but reaches into a private symbol from a sibling module.
Nits
-
xrspatial/geotiff/tests/_helpers/tiff_builders.py:11-12: docstring is good; just a style note that the blank line between the docstring and thefrom __future__import could match the conftest pattern. Not worth a churn. -
xrspatial/geotiff/tests/vrt/test_missing_sources.py:198-205: the comment block abovetest_eager_byte_invalid_policymentions "the old _1799 file" by issue number. A passing reference in a code comment is grey area; the epic asks for no issue numbers in test names and filenames. Leave or scrub. -
xrspatial/geotiff/tests/vrt/test_missing_sources.py:90-91:os.path.join(str(tmp_path), ...)can bestr(tmp_path / ...)for consistency with_write_byte_missing_vrt. Same outcome.
What looks good
- Conftest re-exports preserve every existing
from .conftest import make_minimal_tiff/requires_gpuimport without touching the call sites. - Marker names are unchanged (
requires_gpu,requires_loopback,requires_integration). - Git rename detection caught
_2367.py -> vrt/test_missing_sources.pyand_tiff_surgery.py -> _helpers/tiff_surgery.py, so blame keeps working. release_gate_geotiff.rstwas updated in the same commit, caught by the existingtest_release_gate_2321.pygate.- The decision to leave
test_vrt_missing_sources_default_raise_1843.pyin place is correct: it coversxrspatial.geotiff._vrt.read_vrt(internal) plusXRSPATIAL_GEOTIFF_STRICT=1, neither of which is in the public-API surface this module owns.
Checklist
- Conftest re-exports preserve legacy import paths.
- Marker names unchanged.
- Parametrised IDs are descriptive (
eager/dask, bad values quoted). - No issue numbers in test function names or IDs.
- Release-gate doc updated.
- CLUSTER_AUDIT.md maps every old
file::testto its newfile::test_id. - Old files deleted in the same commit as the new ones.
- CLUSTER_AUDIT.md verification block: correct the "net test_*.py drop of 2" claim to reflect the actual 357 -> 356 transition. Consolidation by definition adds one new file; net is -1. - vrt/test_missing_sources.py: align _write_partial_float_vrt to use Path.write_text with two-space XML indentation, matching _write_byte_missing_vrt. Drop now-unused `import os`. Replace `os.path.join(str(tmp_path), ...)` with `str(tmp_path / ...)`. - conftest.py: drop the private _HAS_LOOPBACK import from _helpers/markers. pytest_collection_modifyitems now calls loopback_available() directly at entry. - vrt/test_missing_sources.py: scrub stray "_1799" issue-number references from test docstrings and helper comments. The file-level docstring still names the two deleted source files for provenance, which is intentional.
brendancol
left a comment
There was a problem hiding this comment.
Follow-up review after commit fbecbf8.
Disposition of original findings
- Suggestion 1 (CLUSTER_AUDIT.md net-count wording): fixed. Verification bullet now reports the actual 357 -> 356 transition and explains why consolidation lands one net deletion per cluster.
- Suggestion 2 (XML indentation mismatch): fixed.
_write_partial_float_vrtnow usesPath.write_textwith two-space indentation; the unusedimport oswas dropped; the twoos.path.join(str(tmp_path), ...)calls becamestr(tmp_path / ...). Both VRT helpers now read the same way. - Suggestion 3 (private
_HAS_LOOPBACKimport in conftest): fixed.pytest_collection_modifyitemsnow callsloopback_available()directly. No conftest module reaches into private names in_helpers/markers.pyany more. - Nit 4 (style note on
tiff_builders.pyblank lines): dismissed. The original note said "not worth a churn"; leaving it alone keeps the diff focused. - Nit 5 (issue-number references in test docstrings): fixed. Scrubbed
_1799from three test docstrings and one comment. The file-level header docstring still names the two deleted source files for provenance, which is intentional audit-trail material. - Nit 6 (
os.path.joinvstmp_path /): fixed, folded into the Suggestion 2 commit.
Verification
pytest xrspatial/geotiff/tests/vrt/ -v-- 22 passed.pytest xrspatial/geotiff/tests/ -x -q-- 5706 passed, 68 skipped, 6 xfailed, 1 xpassed.
No remaining open findings.
Closes #2393. PR 1 of the 11-PR epic #2390.
Summary
_helpers/foundation:tiff_builders.py(wasconftest.py::make_minimal_tiff),tiff_surgery.py(was top-level_tiff_surgery.py),markers.py(was the marker block inconftest.py).vrt/directory withtest_missing_sources.pyfoldingtest_vrt_missing_sources_policy_1799.pyandtest_vrt_missing_sources_policy_2367.pyinto one parametrised module.conftest.pyre-exports the helpers and markers so existingfrom .conftest import ...imports keep working. Thepytest_collection_modifyitemssocketserver hack stays in place until PR 11.Coverage delta
_1799: 3 tests preserved as smoke checks alongside the parametrised matrix._2367: 16 parametrised cases preserved (default raise x 2 readers, explicit raise x 2, warn x 2, invalid value 6 x 2 readers).test_vrt_missing_sources_default_raise_1843.pyleft in place: covers the internal_vrt.read_vrtentry point and theXRSPATIAL_GEOTIFF_STRICT=1env-var override, neither of which is in the consolidated module's surface. Noted inCLUSTER_AUDIT.md.release_gate_geotiff.rstupdated to cite the new path.File delta
_tiff_surgery.py,test_vrt_missing_sources_policy_1799.py,test_vrt_missing_sources_policy_2367.py._helpers/{__init__,tiff_builders,tiff_surgery,markers}.py,vrt/{__init__,test_missing_sources}.py,CLUSTER_AUDIT.md(deleted before merge).find xrspatial/geotiff/tests -name 'test_*.py' | wc -l: 357 -> 356. The spec called for a drop of 2 but consolidation by definition adds one new file undervrt/; the deleted-vs-added count is -2 / +1.Test plan
pytest xrspatial/geotiff/tests/vrt/ -v-- 22 passed.pytest xrspatial/geotiff/tests/ -x -q-- 5706 passed, 68 skipped, 6 xfailed, 1 xpassed.pytest xrspatial/geotiff/tests/test_release_gate_2321.py -v-- all green after updatingrelease_gate_geotiff.rst.Audit
CLUSTER_AUDIT.mdmaps every oldfile::testto its newfile::test_id. Deleted on a follow-up commit on this branch before merge per the epic contract.