[GH-3156][GH-3157][GH-3158] Implement distributed GeoSeries methods - #3159
Merged
jiayuasu merged 11 commits intoJul 24, 2026
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands Sedona’s GeoPandas compatibility layer with distributed implementations of GeoSeries.explode, GeoSeries.get_coordinates, and GeoSeries.geom_equals_exact, while also introducing a new native ST_EqualsExact predicate end-to-end (common -> Spark SQL/Scala API -> Python API) with documentation and test coverage.
Changes:
- Added native
ST_EqualsExactpredicate with tolerance, including Spark SQL registration, Scala DataFrame API wrappers, Python API wrapper, docs, and tests. - Implemented distributed
GeoSeries.explodeandGeoSeries.get_coordinatesusing a shared Spark-side row-expansion path (ST_Dump/ST_DumpPoints+posexplode) with GeoPandas-compatible index modes and ordering. - Implemented distributed
GeoSeries.geom_equals_exact(scalar and series operands) including GeoPandas-like alignment behavior and focused parity tests.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| spark/common/src/test/scala/org/apache/sedona/sql/predicateTestScala.scala | Adds SQL + expression-level Scala predicate tests for ST_EqualsExact, including null handling. |
| spark/common/src/test/scala/org/apache/sedona/sql/dataFrameAPITestScala.scala | Adds Scala DataFrame API test coverage for ST_EqualsExact. |
| spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/st_predicates.scala | Exposes Scala DataFrame API wrappers for ST_EqualsExact. |
| spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Predicates.scala | Introduces Spark expression ST_EqualsExact bound to Predicates.equalsExact. |
| spark/common/src/main/scala/org/apache/sedona/sql/UDF/Catalog.scala | Registers ST_EqualsExact in the Sedona SQL function catalog. |
| python/tests/sql/test_predicate.py | Adds Python SQL predicate parity tests for ST_EqualsExact. |
| python/tests/sql/test_dataframe_api.py | Adds Python DataFrame API tests + null-argument cases for ST_EqualsExact. |
| python/tests/geopandas/test_match_geopandas_series.py | Adds GeoPandas parity tests for explode, get_coordinates, and geom_equals_exact. |
| python/tests/geopandas/test_geoseries.py | Adds focused GeoSeries tests for explode, get_coordinates (incl. index modes), and geom_equals_exact alignment/error cases. |
| python/sedona/spark/sql/st_predicates.py | Adds Python predicate API wrapper ST_EqualsExact. |
| python/sedona/spark/geopandas/geoseries.py | Implements distributed explode, get_coordinates, and geom_equals_exact, plus shared expansion helper and CRS preservation for all-empty explode results. |
| python/sedona/spark/geopandas/base.py | Adds GeoDataFrame delegation and public docstrings for get_coordinates and geom_equals_exact. |
| docs/api/sql/Predicates/ST_EqualsExact.md | Adds SQL reference documentation for ST_EqualsExact. |
| docs/api/sql/Geometry-Functions.md | Adds ST_EqualsExact to the SQL predicate function index table. |
| common/src/test/java/org/apache/sedona/common/PredicatesTest.java | Adds common-layer predicate tests for equalsExact semantics and edge cases. |
| common/src/main/java/org/apache/sedona/common/Predicates.java | Adds Predicates.equalsExact(Geometry, Geometry, double) backed by JTS equalsExact. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
670
to
+674
| if not is_aggr: | ||
| # We always select NATURAL_ORDER_COLUMN_NAME, to avoid having to regenerate it in the result. | ||
| # We always select SPARK_DEFAULT_INDEX_NAME, to retain series index info. | ||
|
|
||
| exprs.append(scol_for(df, SPARK_DEFAULT_INDEX_NAME)) | ||
| exprs.append(scol_for(df, NATURAL_ORDER_COLUMN_NAME)) | ||
|
|
||
| index_spark_columns = [scol_for(df, SPARK_DEFAULT_INDEX_NAME)] | ||
| index_fields = [self._internal.index_fields[0]] | ||
| # Preserve every index level and the natural order in the result. | ||
| index_spark_columns = [ | ||
| scol_for(df, name) for name in self._internal.index_spark_column_names | ||
| ] |
Comment on lines
+3409
to
+3410
| from pyspark.pandas.internal import InternalField | ||
|
|
| :param b: Other geometry column to check. | ||
| :type b: ColumnOrName | ||
| :param tolerance: Maximum distance allowed between corresponding coordinates. | ||
| :type tolerance: ColumnOrName or float |
Member
Author
There was a problem hiding this comment.
Fixed in 15cc146: the tolerance type documentation now includes int, matching the function signature.
Comment on lines
+22
to
+24
| Introduction: Return true if A and B have the same structure and their corresponding coordinates are equal within a tolerance. | ||
|
|
||
| Unlike `ST_Equals`, this predicate requires geometry types, component order, ring order, and vertex order to match. The tolerance is the maximum distance allowed between each pair of corresponding coordinates. The comparison uses x and y coordinates and ignores z and m coordinates. |
Member
Author
There was a problem hiding this comment.
Fixed in 15cc146: changed the introduction to use Returns true and capitalized Z and M.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
python/sedona/spark/geopandas/geoseries.py:789
- When
index_parts=True,_expand_geometry_arrayappendsNoneintoindex_fields.index_fieldsshould containInternalFieldentries aligned withindex_spark_columns; insertingNonerisks breakingInternalFramemetadata and downstream MultiIndex behavior.
if index_parts:
part_index_col = SPARK_INDEX_NAME_FORMAT(len(output_index_cols))
output_index_cols.append(part_index_col)
index_names.append(None)
index_fields.append(None)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Did you read the Contributor Guide?
Is this PR related to a ticket?
[GH-XXX] my subject. Closes GeoPandas: implement distributed GeoSeries.get_coordinates #3156. Closes GeoPandas: implement distributed GeoSeries.explode #3157. Closes GeoPandas: implement distributed GeoSeries.geom_equals_exact #3158.What changes were proposed in this PR?
GeoSeries.explodewith GeoPandas-compatible index modes, deterministic component ordering, geometry metadata preservation, and empty/null handling.GeoSeries.get_coordinatesand itsGeoDataFramedelegation, including optional Z/M ordinates and GeoPandas-compatible index modes.ST_Dump/ST_DumpPointsplusposexploderow-expansion path between both expansion methods.ST_EqualsExactpredicate with coordinate tolerance to Sedona common; expose it through Spark SQL, Flink SQL, and Snowflake BINARY/GEOMETRY/GEOGRAPHY functions; and add the Scala and Python DataFrame APIs.GeoSeries.geom_equals_exactand itsGeoDataFramedelegation for scalar geometry and distributed Series operands, including duplicate-index and MultiIndex alignment behavior.How was this patch tested?
Did this PR include necessary documentation updates?
v1.9.1format.ST_EqualsExactSQL references for Spark, Flink, and Snowflake.