Use sidecar header for remote .ovr decode byte order (#2314)#2319
Merged
Conversation
When the selected overview IFD lives in an external .tif.ovr sidecar, _parse_cog_http_meta now returns the sidecar's TIFFHeader instead of the base file's. The eager HTTP path and the dask HTTP path already unpack this header and feed header.byte_order into _decode_strip_or_tile, so the swap is enough to make a big-endian .ovr paired with a little-endian base (or the reverse) decode against the right endianness. Mirrors the local sidecar path in _reader.py:223 which already handled this case. Adds test_remote_sidecar_byte_order_2314.py covering the eager HTTP, dask HTTP, fsspec memory, and direct _parse_cog_http_meta contracts for both byte-order directions. Eight of the ten test cases fail on the unfixed branch.
brendancol
commented
May 22, 2026
Contributor
Author
brendancol
left a comment
There was a problem hiding this comment.
PR Review: Use sidecar header for remote .ovr decode byte order (#2314)
Blockers
None.
Suggestions
None.
Nits
xrspatial/geotiff/tests/test_remote_sidecar_byte_order_2314.py: the fixture builder relies on tifffile's default layout, which is stripped (untiled). The eager and dask HTTP paths both flow through_fetch_decode_cog_http_stripsfor these fixtures, so the tiled decode path'sheader.byte_orderplumbing is exercised only transitively. The fix lives at the metadata layer so the result is the same, but a second parametrized fixture withtile=...would close the last coverage gap. Optional.xrspatial/geotiff/_cog_http.py:265:return_header = sidecar.header if used_sidecar else headeris safe becauseused_sidecarcan only be True whensidecaris not None (sidecar_ifd_idsstays empty otherwise). A defensive comment tying those two invariants together would make the relationship obvious to a future reader. Optional.
What looks good
- The fix is minimum-surface: one line in
_parse_cog_http_metareturns the sidecar'sTIFFHeaderwhenused_sidecaris True. Downstream code in_read_cog_httpand_backends/dask.pyalready readsheader.byte_orderoff the unpacked header and picks up the right value with no further plumbing. - Docstring updated to describe the new contract.
- Comment in
dask.pyupdated so the code matches what the comment claims. - Tests fail on 8 of 10 cases without the fix (parametrized LE/BE and BE/LE across eager HTTP, dask HTTP, fsspec memory, and a direct contract test on
_parse_cog_http_meta). The two passing-before cases are the local sidecar path, included as a parity sanity check. - Test fixtures use uuid-suffixed paths so parallel runs do not collide.
- No public API change; no docs / README update needed.
Checklist
- Algorithm matches reference behaviour (parity with
_reader.py:223local sidecar swap) - All implemented backends produce consistent results (numpy and dask HTTP both exercised)
- NaN handling is correct (untouched; uint16 fixtures have no NaN)
- Edge cases are covered by tests (both byte-order directions)
- Dask chunk boundaries handled correctly (
chunks=16forces multi-chunk read) - No premature materialization or unnecessary copies
- Benchmark exists or is not needed (pure correctness fix)
- README feature matrix updated (not applicable)
- Docstrings present and accurate
* Add a parametrized tiled-layout HTTP eager test so the _fetch_decode_cog_http_tiles call site is exercised directly, not just through the stripped path. * Document the used_sidecar => sidecar is not None invariant in the comment above the sidecar.header read so future readers can see why the conditional access is safe.
brendancol
commented
May 22, 2026
Contributor
Author
brendancol
left a comment
There was a problem hiding this comment.
PR Review (follow-up after c1dbfb9)
Both nits from the previous review have been addressed:
_cog_http.py: invariant comment added above thesidecar.headerread, explaining whyused_sidecar=Trueimpliessidecar is not None.test_remote_sidecar_byte_order_2314.py: new parametrizedtest_http_eager_mixed_endian_sidecar_tiledexercises the tiled HTTP decode path directly. Test count is now 12 (was 10), all passing.
No remaining blockers, suggestions, or nits. Ready for CI to land.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2314.
Summary
_parse_cog_http_metanow returns the sidecar'sTIFFHeaderwhen the selected overview IFD came from the.tif.ovr. The eager HTTP path and the dask HTTP path already feed the unpackedheader.byte_orderinto_decode_strip_or_tile, so the swap is enough to make a big-endian sidecar paired with a little-endian base (or the reverse) decode against the right endianness._reader.py:223already did this; the remote paths now reach parity.test_remote_sidecar_byte_order_2314.pycovering the eager HTTP, dask HTTP, fsspec memory, and direct_parse_cog_http_metacontracts for both byte-order directions. Eight of ten test cases fail on the unfixed branch and pass after the change.Backend coverage
_read_cog_http)_backends/dask.py)memory://)Test plan
pytest xrspatial/geotiff/tests/test_remote_sidecar_byte_order_2314.py(10 tests pass after fix, 8 fail before)pytest xrspatial/geotiff/tests/(5224 pass, 68 skipped)test_remote_sidecar_chunked_2239.pystill passes (parity with same-endian sidecar fixture)