Skip to content

[DOCS] Add blog post: spatial statistics with sedona.stats - #3219

Merged
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:blog/spatial-stats
Jul 31, 2026
Merged

[DOCS] Add blog post: spatial statistics with sedona.stats#3219
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:blog/spatial-stats

Conversation

@jiayuasu

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

  • No: this is a documentation update. The PR name follows the format [DOCS] my subject

What changes were proposed in this PR?

A new blog post on the spatial statistics module (sedona.stats), which currently has no blog coverage. It works through the classic exploratory-spatial-analysis workflow as one investigation on the OSM POI shapefile bundled at docs/usecases/data (12,873 points):

  • Moran's I — global spatial autocorrelation (is anything clustered at all?)
  • Getis-Ord Gi* — hotspot significance on a geohash grid (where, exactly?)
  • DBSCAN — density clustering with spheroidal distances (what are the groupings?)
  • Local Outlier Factor — spatial anomalies (who doesn't belong?)
  • A closing section on how each primitive distributes (distance-join weighting, connected-components clustering, ST_KNN-based neighbor search)

Includes a cover image and three figures rendered from the actual analysis results (hotspot z-score map, cluster map, outlier map). The post also notes the Spark checkpoint-directory requirement for DBSCAN and uses a CAST(... AS DOUBLE) on the count column for Moran's I — the integer-column failure behind that workaround is filed as #3218.

How was this patch tested?

  • Every snippet was executed against Apache Sedona (PySpark, current snapshot jars) on the bundled shapefile; all numbers shown (Moran I = 0.2519 / z = 12.69, 26 significant Gi* hotspots with top z = 14.94, 72 DBSCAN clusters, LOF top outliers) are the real outputs of those runs.
  • The three figures are plotted directly from the exported results of the same runs; the implementation claims in the scale section were checked against the Scala sources (Weighting, DBSCAN, LocalOutlierFactor).
  • The site was built with mkdocs serve and the rendered post verified (all four images load, code highlighting, blog index entry).
  • SVG validated as well-formed XML; pre-commit hooks pass on the changed files.

Did this PR include necessary documentation updates?

  • Yes, I have updated the documentation.

Works through sedona.stats as one investigation on the bundled OSM POI shapefile: Moran's I global autocorrelation, Getis-Ord Gi* hotspot significance, DBSCAN density clustering, and Local Outlier Factor anomalies, with real-data figures rendered from the verified runs. Notes the Spark checkpoint requirement for DBSCAN and the double-cast workaround for integer value columns (apacheGH-3218).
@jiayuasu
jiayuasu marked this pull request as ready for review July 31, 2026 08:13
@jiayuasu
jiayuasu merged commit ef02187 into apache:master Jul 31, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant