The HTTP COG reader at xrspatial/geotiff/_reader.py already fetches tile bytes concurrently via read_ranges_coalesced (issues #1480 and #1487). The Pass 3 decode loop, however, runs sequentially:
for (band_idx, tr, tc), tile_data in zip(placements, tile_bytes_list):
tile_pixels = _decode_strip_or_tile(...)
...The local-file path in the same module (_read_tiles around line 2017) parallelises decode whenever tile_pixels >= 64 * 1024 and n_tiles > 1:
use_parallel = (n_tiles > 1 and tile_pixels >= 64 * 1024)
if use_parallel:
with ThreadPoolExecutor(max_workers=n_workers) as pool:
decoded = list(pool.map(_decode_one, tile_jobs))Codec extensions (deflate, zstd, LZW) release the GIL, so the thread pool actually overlaps decode work across cores. For wide windowed reads of multi-tile COGs the HTTP path was leaving that parallelism on the table after a concurrent fetch.
Pass 10 of the geotiff performance sweep flagged this asymmetry. Fix mirrors the local-path threshold and pool inside _fetch_decode_cog_http_tiles; the placement loop that writes pixels into result stays serial to avoid contended writes.
Found by /sweep-performance.
The HTTP COG reader at
xrspatial/geotiff/_reader.pyalready fetches tile bytes concurrently viaread_ranges_coalesced(issues #1480 and #1487). The Pass 3 decode loop, however, runs sequentially:The local-file path in the same module (
_read_tilesaround line 2017) parallelises decode whenevertile_pixels >= 64 * 1024andn_tiles > 1:Codec extensions (deflate, zstd, LZW) release the GIL, so the thread pool actually overlaps decode work across cores. For wide windowed reads of multi-tile COGs the HTTP path was leaving that parallelism on the table after a concurrent fetch.
Pass 10 of the geotiff performance sweep flagged this asymmetry. Fix mirrors the local-path threshold and pool inside
_fetch_decode_cog_http_tiles; the placement loop that writes pixels intoresultstays serial to avoid contended writes.Found by /sweep-performance.