Skip to content

GeoPandas: implement distributed coverage validation #3273

Description

@jiayuasu

Part of #2230.

GeoPandas 1.1 added GeoSeries.is_valid_coverage and
GeoSeries.invalid_coverage_edges for validating polygonal coverages. A valid coverage has
non-overlapping polygon interiors and exactly edge-matched shared boundaries; callers may also
request detection of gaps up to a specified width.

Sedona already depends on JTS 1.20, which provides CoveragePolygonValidator, but validating an
entire distributed GeoSeries cannot be implemented by collecting every polygon into one task.

The implementation should:

  • add a focused internal Spark primitive that validates one target geometry against a
    self-including array of candidate coverage geometries;
  • discover candidate neighbors with a distributed spatial self-join using polygonal envelopes
    expanded by gap_width, matching JTS coverage-index semantics;
  • validate each distinct represented geometry against its candidates on executors, then map the
    diagnostic back to every matching physical input row;
  • return one invalid-edge geometry per original row while preserving its index, order, CRS, and
    output naming;
  • reduce the distributed invalid-edge result to one Python boolean for is_valid_coverage;
  • preserve GeoPandas behavior for valid coverages, overlaps, edge mismatches, duplicate polygons,
    narrow gaps, holes, multipolygons, empty sources, and empty/non-polygonal values;
  • preserve null geometries in their physical input rows instead of reproducing Shapely 2.1's
    result-compaction bug, which moves null outputs to the end;
  • accept finite, non-negative numeric gap_width scalars and reject negative or non-finite values,
    whose behavior is inconsistent between GEOS and JTS (and for positive infinity would require a
    non-scalable all-pairs join); and
  • avoid Python UDFs, global geometry collection, and driver materialization of geometry rows.

The per-target primitive is implementation plumbing for the distributed GeoPandas methods, not a
public Spark SQL API.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions