-
Notifications
You must be signed in to change notification settings - Fork 30
Push down all spatial predicates when reading GeoParquet files #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Push down all spatial predicates when reading GeoParquet files #39
Conversation
…ter evaluation function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This patch enhances GeoParquet predicate pushdown capabilities by adding support for all spatial predicates including distance predicates and spatial relation operators. The key improvements include expanding bounding box functionality with containment checking and distance-based expansion, implementing spatial predicate parsing utilities, and extending the spatial filter system to handle additional spatial operations.
- Adds bounding box containment and expansion methods to support spatial predicate evaluation
- Implements comprehensive spatial predicate pushdown for functions like
ST_Contains,ST_Within,ST_DWithin, etc. - Introduces utility functions for parsing distance predicates from physical expressions
Reviewed Changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| rust/sedona-geometry/src/interval.rs | Adds containment checking and expansion methods to interval types |
| rust/sedona-geometry/src/bounding_box.rs | Implements contains and expand_by methods for bounding box operations |
| rust/sedona-expr/src/utils.rs | New utility module for parsing distance predicates from expressions |
| rust/sedona-expr/src/spatial_filter.rs | Extends spatial filter support to handle all spatial relation operators and distance predicates |
| rust/sedona-spatial-join/src/optimizer.rs | Refactors distance predicate matching to use new utility functions |
| rust/sedona-spatial-join/src/operand_evaluator.rs | Adds type casting for distance values |
| rust/sedona-geoparquet/src/format.rs | Fixes typo and updates test cases for expanded predicate support |
| rust/sedona-geoparquet/Cargo.toml | Adds rstest dependency for parameterized tests |
| rust/sedona-expr/src/lib.rs | Exports new utils module |
| rust/sedona-expr/Cargo.toml | Adds rstest dependency for parameterized tests |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
paleolimbot
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! (Particularly for handling the wraparound case 😬 )
Agreed. This requires porting spherical bounding to Rust (which we probably need anyway for Rust Parquet to properly support geography), or injecting it from the session somehow to avoid depending on s2 C++. |
This patch pushes all spatial predicates when reading GeoParquet files, including distance predicate and all spatial relation operators such as
ST_Contains,ST_Within, etc. To implement this, we have added the following methods toBoundingBox:contains: Check for bounding box containmentexpand_by: Expand the bounding box by the specified distanceThe interval types also have corresponding functions implemented to work with 1-D plain and wrapping-around intervals.
Currently GeoParquet predicate pushdown only work with planar geometries correctly. This limitation already exists prior to developing this patch. I suggest that we submit another PR to disable spatial predicate pushdown for GeoParquet files containing spherical geography objects.