[GH-3286] Bound multi-bbox STAC client searches with max_items - #3287
Merged
Conversation
A search with a positive max_items and multiple bounding boxes previously fell back to fully enumerating the collection with 10-Item pages. Issue one API-native capped search per bbox, union the results, remove duplicate (collection, id) pairs, and then apply the global limit. Each bbox may safely use the full max_items cap: if any bbox alone holds that many Items it can satisfy the limit, otherwise every bbox is fully represented. Residual Spark-side fallbacks stay uncapped but use 200-Item request pages to reduce pagination overhead.
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Did you read the Contributor Guide?
Is this PR related to a ticket?
[GH-XXX] my subject. Closes STAC Python client: multi-bbox search with max_items enumerates the entire collection #3286What changes were proposed in this PR?
GH-3283 added API-native capped searches, but only for at most one bbox: a search with multiple bounding boxes and a positive
max_itemsstill fell back to fully enumerating the collection at 10 Items per request before Spark applied the bbox predicates and the final limit.Each bbox maps directly to a STAC API Collection Items request, and the list-of-bboxes extension represents their union, so a bounded multi-bbox search is now decomposed instead of falling back:
_api_search_option_setsbuilds one API-native capped search per bbox (itemsLimitMax=max_items,itemsLimitPerRequest=min(200, max_items));load_items_dfunions the per-bbox DataFrames, deduplicates by(collection, id), and then applies one final globalmax_itemslimit.max_itemscap: if any bbox alone contains at least that many Items it can satisfy the limit; otherwise every bbox is fully represented, so deduplicate-then-limit yields the requested global result size. A single bbox skips the union and dedup entirely.How was this patch tested?
_apply_spatial_temporal_filtersasserts API-owned bboxes are never re-evaluated by Spark.python/tests/stac/test_collection_client.py: 33 passed; the one failure (test_expand_date_filter_includes_final_fractional_second) also fails on unmodified master in this environment (stale local jar) and is unrelated to this change.black-jupyter,blacken-docs,pyupgrade, andcodespellhooks pass.Did this PR include necessary documentation updates?