geotiff: close HTTP source when COG tile fetch raises#1819
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses issue #1816 by ensuring the HTTP COG reader path closes its _HTTPSource even when tile fetching/decoding or subsequent post-processing raises, preventing future resource leaks if _HTTPSource.close() later becomes non-trivial.
Changes:
- Wrap
_fetch_decode_cog_http_tiles+ post-processing in_read_cog_httpwithtry/finallyto guaranteesource.close(). - Add regression tests that wrap
_HTTPSourceto trackclose()calls on both success and exception paths (fetch failure + post-processing failure).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
xrspatial/geotiff/_reader.py |
Adds try/finally around the HTTP tile fetch and post-processing so source.close() runs on exceptions. |
xrspatial/geotiff/tests/test_cog_http_close_on_error_1816.py |
Introduces regression tests validating source.close() is invoked on both success and failure paths. |
Comments suppressed due to low confidence (1)
xrspatial/geotiff/tests/test_cog_http_close_on_error_1816.py:206
- Same continuation-indentation issue as above: this wrapped function signature uses an 8-space indent on the continuation line and may fail flake8/pycodestyle continuation-indent checks. Reformat the parameters using a 4-space hanging indent (or alignment) to match PEP8/flake8 expectations.
def test_http_source_closed_when_post_processing_raises(
single_band_cog, monkeypatch):
"""An exception from the orientation/photometric step also runs
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1938
to
+1946
| # Issue #1816: wrap the tile fetch and post-processing in try/finally | ||
| # so ``source.close()`` runs even when the fetch/decode or the | ||
| # orientation/photometric step raises. ``_HTTPSource.close()`` is a | ||
| # no-op today, but a future resource-holding source would leak on | ||
| # the error path without this. The explicit ``source.close()`` calls | ||
| # in the validation block above stay as-is; ``close()`` is idempotent. | ||
| try: | ||
| arr = _fetch_decode_cog_http_tiles( | ||
| source, header, ifd, max_pixels=max_pixels, window=window) |
Comment on lines
+171
to
+173
| def test_http_source_closed_when_tile_fetch_raises( | ||
| single_band_cog, monkeypatch): | ||
| """When ``_fetch_decode_cog_http_tiles`` raises, ``_read_cog_http`` |
PR #1803 forwarded the caller's max_pixels to read_to_array inside read_vrt's source loop so a tiny VRT output cannot force a huge source decode (#1796). The output-window check at the source read enforces that correctly. A separate per-tile dimension check at the same call sites also consumed the caller's max_pixels, so a caller setting max_pixels as an output budget (e.g. 10_000) failed the per-tile sanity check on any normal source whose default tile size is 256x256 (= 65_536 pixels). Use MAX_PIXELS_DEFAULT for the per-tile dim check at the two call sites in _read_tiles (local) and _read_tiles_cog_http (HTTP). The output-window check at the same functions continues to enforce the user-supplied max_pixels, preserving the #1796 protection.
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
Co-authored-by: brendancol <433221+brendancol@users.noreply.github.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1816.
Wraps the tile fetch and post-processing block in _read_cog_http in try/finally so source.close() runs even when _fetch_decode_cog_http_tiles raises. _HTTPSource.close() is currently a no-op so the fix is structural: a future resource-holding source would otherwise leak on the error path.
Tested: pytest xrspatial/geotiff/tests/test_cog_http_close_on_error_1816.py