Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/source/reference/geotiff.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,18 @@ The contract covers:
* Normal CRS, transform, dtype, nodata, band, and
pixel-is-area / pixel-is-point behavior.

What the stable COG contract does NOT cover is the overview pyramid
customisation surface itself. The presence of an internal overview
pyramid is part of the stable COG layout, but the
``overview_levels`` and ``overview_resampling`` knobs on
``to_geotiff`` (and the pyramid bytes the resampling kernels produce)
are tracked as ``advanced`` under
``SUPPORTED_FEATURES['writer.overviews']``. The two registry entries
exist precisely so that the COG layout and the pyramid-generation
knobs can promote independently; the ``to_geotiff`` docstring marks
``cog=True`` as stable but flags ``overview_levels`` and
``overview_resampling`` as advanced for the same reason.

The promotion is backed by the writer compliance suite (#2292), the
cross-backend parity gate (#2293), and the per-tile byte-budget contract
(#2294 / #2298). These tests run on every CI build so a regression in
Expand Down
4 changes: 2 additions & 2 deletions docs/source/reference/geotiff_release_contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,8 @@ category. The `Key` column matches the runtime key.
| Key | Tier | Notes |
| --- | --- | --- |
| `writer.local_file` | stable | Local GeoTIFF write; round-trips bit-exact under every stable codec. |
| `writer.cog` | stable | CPU writer emits a spec-conforming COG layout (IFD-first, tiled, internal overviews, lossless codec). |
| `writer.overviews` | advanced | Internal overview IFD generation. |
| `writer.cog` | stable | CPU writer emits a spec-conforming COG layout (IFD-first, tiled, internal overviews, lossless codec). The COG layout itself is stable; the pyramid-customisation knobs on `to_geotiff` (`overview_levels`, `overview_resampling`) are tracked separately under `writer.overviews`. |
| `writer.overviews` | advanced | Internal overview IFD generation: the `overview_levels` and `overview_resampling` knobs on `to_geotiff` and the resampled pyramid bytes themselves. The presence of an internal pyramid in a stable-codec COG is covered by `writer.cog`; this entry covers the customisation surface and the resampling output. |
| `writer.bigtiff` | advanced | `bigtiff=True` (or auto-promotion above 4 GiB) writes BigTIFF magic, 8-byte offsets, and 20-byte IFD entries. |
| `writer.bigtiff_cog` | advanced | BigTIFF plus COG. Tracked separately because the combination has its own external-interop surface. |
| `writer.gpu` | experimental | GPU write path. |
Expand Down
31 changes: 20 additions & 11 deletions xrspatial/geotiff/_writers/eager.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,13 @@ def to_geotiff(data: xr.DataArray | np.ndarray,
* [stable] Local-file output on an axis-aligned grid with
``compression`` in ``{'none', 'deflate', 'lzw', 'packbits',
'zstd'}``; CRS / transform / nodata attrs round-trip; ``bigtiff``
auto-promotion.
* [advanced] ``cog=True`` and overview generation; explicit
``bigtiff=True``; ``photometric=`` overrides; ``extra_tags``
pass-through.
auto-promotion; ``cog=True`` (the IFD-first tiled COG layout with
a stable codec, covered by ``SUPPORTED_FEATURES['writer.cog']``).
* [advanced] Internal overview pyramid generation
(``SUPPORTED_FEATURES['writer.overviews']``): the
``overview_levels`` and ``overview_resampling`` knobs and the
pyramid bytes themselves. Also explicit ``bigtiff=True``;
``photometric=`` overrides; ``extra_tags`` pass-through.
* [experimental] GPU dispatch via ``gpu=True``;
``compression`` in ``{'lerc', 'jpeg2000', 'j2k', 'lz4'}`` behind
the explicit ``allow_experimental_codecs=True`` opt-in;
Expand Down Expand Up @@ -186,13 +189,19 @@ def to_geotiff(data: xr.DataArray | np.ndarray,
* ``3`` -> floating-point predictor (float dtypes only; typically
gives better deflate/zstd ratios on float data than predictor 2).
cog : bool
[advanced] COG output materialises the full array because
overview pyramids need it, and the all-IFDs-at-file-start
layout only round-trips through readers that honour the COG
layout contract. Write as Cloud Optimized GeoTIFF. Requires
``tiled=True`` (the default): the COG specification mandates a
tiled internal layout, so ``cog=True, tiled=False`` raises
``ValueError``.
[stable] Write as Cloud Optimized GeoTIFF. The CPU writer
emits the spec-conforming COG layout (IFD-first, tiled,
internal overviews, lossless codec) covered by
``SUPPORTED_FEATURES['writer.cog']``. Requires ``tiled=True``
(the default): the COG specification mandates a tiled internal
layout, so ``cog=True, tiled=False`` raises ``ValueError``.
COG output also materialises the full array, because the
overview pyramid needs random access to every pixel; the
``streaming_buffer_bytes`` kwarg is a no-op on this path.
Customisation of the overview pyramid itself
(``overview_levels``, ``overview_resampling``) is tracked
separately as advanced under
``SUPPORTED_FEATURES['writer.overviews']``.
overview_levels : list[int] or None
[advanced] Overview pyramids are an optional COG feature; the
decimation factors and resampling choice affect downstream
Expand Down
136 changes: 136 additions & 0 deletions xrspatial/geotiff/tests/release_gates/test_stable_features.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
from __future__ import annotations

import importlib.util
import inspect
import os
import re
import struct
Expand Down Expand Up @@ -2581,3 +2582,138 @@ def test_release_gate_vrt_rows_point_at_real_test_functions() -> None:
"file was emptied or the row should be removed: "
f"{empty}"
)


# =========================================================================== #
# Section: COG stability contract parity (issue #2513) #
# =========================================================================== #
#
# Three surfaces have to agree on whether COG writes are stable:
#
# 1. The runtime tier registry ``SUPPORTED_FEATURES`` in ``_attrs.py``.
# 2. The release contract / reference docs in ``docs/source/reference/``.
# 3. The ``to_geotiff`` docstring's release-contract tier block and the
# per-parameter ``[tier]`` marker on ``cog``.
#
# Issue #2513 documented that surface (3) declared ``cog=True`` as
# ``[advanced]`` while (1) and (2) said ``stable``. This gate locks the
# resolution: ``writer.cog`` is the stable COG layout contract;
# ``writer.overviews`` is the separately tracked advanced pyramid
# customisation surface. If a future change demotes ``writer.cog`` or
# re-introduces an ``[advanced]`` framing on the ``cog`` docstring, this
# gate fails so the drift is caught before release.


@pytest.mark.release_gate
def test_release_gate_writer_cog_stays_stable() -> None:
"""``SUPPORTED_FEATURES['writer.cog']`` is the stable COG layout entry."""
assert SUPPORTED_FEATURES.get("writer.cog") == "stable", (
"release gate: SUPPORTED_FEATURES['writer.cog'] is no longer "
"'stable'. The release contract and the to_geotiff docstring "
"promise stable COG writes; demoting the registry entry breaks "
"that contract. See issue #2513."
)


@pytest.mark.release_gate
def test_release_gate_writer_overviews_stays_advanced() -> None:
"""``writer.overviews`` is the advanced sub-behaviour of COG writes.

Issue #2513: the stable COG layout (``writer.cog``) and the advanced
overview-customisation surface (``writer.overviews``) are tracked as
two separate registry entries so they can promote independently. If
overview customisation gets promoted, the docstring and contract
have to be updated together.
"""
assert SUPPORTED_FEATURES.get("writer.overviews") == "advanced", (
"release gate: SUPPORTED_FEATURES['writer.overviews'] is no "
"longer 'advanced'. If overview customisation has been promoted "
"(or demoted), update the to_geotiff docstring's "
"[advanced]/[stable] markers on overview_levels and "
"overview_resampling and the matching rows in "
"docs/source/reference/geotiff_release_contract.md and "
"docs/source/reference/geotiff.rst together. See issue #2513."
)


@pytest.mark.release_gate
def test_release_gate_to_geotiff_docstring_marks_cog_stable() -> None:
"""The ``to_geotiff`` docstring marks ``cog=True`` as ``[stable]``.

Tripwire for issue #2513: an earlier version of the docstring
described ``cog=True`` as ``[advanced]``, contradicting the registry
and the release contract. This assertion fails if the contradiction
creeps back in. The check is deliberately strict on the wording so
a copy-paste from another parameter cannot satisfy it accidentally.

``inspect.getdoc`` normalises docstring indentation across every
supported Python (3.12 keeps the source-level indent on ``__doc__``;
3.13+ strips the common leading whitespace at compile time). The
regexes anchor at column 0 of the cleaned text so the assertion
matches on every supported version.
"""
doc = inspect.getdoc(to_geotiff) or ""
# The function-level tier block must list cog=True under [stable].
# Match the bullet body across line wraps without taking a hard
# dependency on a single line layout.
stable_bullet_re = re.compile(
r"\*\s+\[stable\][^*]*?cog=True",
re.DOTALL,
)
assert stable_bullet_re.search(doc), (
"release gate: the to_geotiff docstring's [stable] tier bullet "
"no longer mentions ``cog=True``. The COG layout is stable per "
"SUPPORTED_FEATURES['writer.cog']; the docstring has to agree. "
"See issue #2513."
)

# The per-parameter marker on the ``cog`` parameter must be [stable].
# ``inspect.getdoc`` already removed the common indent, so the
# parameter line is at column 0 and the body is indented one level.
cog_param_re = re.compile(
r"^cog : bool\n \[(?P<tier>[\w-]+)\]",
re.MULTILINE,
)
match = cog_param_re.search(doc)
assert match is not None, (
"release gate: cannot find the ``cog`` parameter docstring "
"block in to_geotiff. The tier-marker regex needs updating, or "
"the parameter was renamed."
)
assert match.group("tier") == "stable", (
"release gate: to_geotiff's ``cog`` parameter is marked "
f"[{match.group('tier')}] in its docstring. "
"SUPPORTED_FEATURES['writer.cog'] is stable, so the docstring "
"marker has to be [stable] too. See issue #2513."
)


@pytest.mark.release_gate
def test_release_gate_to_geotiff_docstring_marks_overview_knobs_advanced() -> None:
"""``overview_levels`` and ``overview_resampling`` stay ``[advanced]``.

The pyramid-customisation surface is the advanced sub-behaviour of
COG writes (``SUPPORTED_FEATURES['writer.overviews'] == 'advanced'``).
If those knobs ever get promoted, this gate fails together with the
registry gate above so the change is forced through both surfaces.
Uses ``inspect.getdoc`` for the same cross-version reason as the
``cog`` gate above.
"""
doc = inspect.getdoc(to_geotiff) or ""
for param in ("overview_levels", "overview_resampling"):
param_re = re.compile(
rf"^{param} : [^\n]+\n \[(?P<tier>[\w-]+)\]",
re.MULTILINE,
)
match = param_re.search(doc)
assert match is not None, (
f"release gate: cannot find the ``{param}`` parameter "
"docstring block in to_geotiff. The tier-marker regex "
"needs updating, or the parameter was renamed."
)
assert match.group("tier") == "advanced", (
f"release gate: to_geotiff's ``{param}`` parameter is "
f"marked [{match.group('tier')}] in its docstring. "
"SUPPORTED_FEATURES['writer.overviews'] is advanced, so the "
"docstring marker has to be [advanced] too. See issue #2513."
)
Loading