Skip to content

[GH-3286] Bound multi-bbox STAC client searches with max_items - #3287

Merged
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:stac-multi-bbox-search
Aug 26, 2026
Merged

[GH-3286] Bound multi-bbox STAC client searches with max_items#3287
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:stac-multi-bbox-search

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Aug 26, 2026

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

GH-3283 added API-native capped searches, but only for at most one bbox: a search with multiple bounding boxes and a positive max_items still fell back to fully enumerating the collection at 10 Items per request before Spark applied the bbox predicates and the final limit.

Each bbox maps directly to a STAC API Collection Items request, and the list-of-bboxes extension represents their union, so a bounded multi-bbox search is now decomposed instead of falling back:

  • _api_search_option_sets builds one API-native capped search per bbox (itemsLimitMax=max_items, itemsLimitPerRequest=min(200, max_items)); load_items_df unions the per-bbox DataFrames, deduplicates by (collection, id), and then applies one final global max_items limit.
  • Each request may safely use the full max_items cap: if any bbox alone contains at least that many Items it can satisfy the limit; otherwise every bbox is fully represented, so deduplicate-then-limit yields the requested global result size. A single bbox skips the union and dedup entirely.
  • If any bbox is unrepresentable (invalid coordinates), the whole search falls back rather than partially capping.
  • Shapes that still require a Spark-side predicate (multiple datetime intervals, geometry filters, IDs) keep the uncapped fallback so a raw fetch cap cannot starve residual predicates — but now use 200-Item request pages to reduce pagination overhead without changing which rows Spark evaluates.

How was this patch tested?

  • New Python tests: multi-bbox decomposition into per-bbox option sets (including invalid-bbox fallback), union/deduplication/global-limit wiring with per-search reader options, and the fallback page-size bump; a patched _apply_spatial_temporal_filters asserts API-owned bboxes are never re-evaluated by Spark. python/tests/stac/test_collection_client.py: 33 passed; the one failure (test_expand_date_filter_includes_final_fractional_second) also fails on unmodified master in this environment (stale local jar) and is unrelated to this change.
  • Repository pinned black-jupyter, blacken-docs, pyupgrade, and codespell hooks pass.

Did this PR include necessary documentation updates?

  • Yes, I have updated the documentation. The STAC tutorial (EN/ZH) documents the multi-bbox union/deduplication semantics and the fallback page size.

A search with a positive max_items and multiple bounding boxes previously
fell back to fully enumerating the collection with 10-Item pages. Issue
one API-native capped search per bbox, union the results, remove
duplicate (collection, id) pairs, and then apply the global limit. Each
bbox may safely use the full max_items cap: if any bbox alone holds that
many Items it can satisfy the limit, otherwise every bbox is fully
represented. Residual Spark-side fallbacks stay uncapped but use 200-Item
request pages to reduce pagination overhead.
@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Aug 26, 2026
@jiayuasu
jiayuasu merged commit 37e64f3 into apache:master Aug 26, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

STAC Python client: multi-bbox search with max_items enumerates the entire collection

1 participant