Skip to content

[GH-2640] Simplify warning messages of SedonaContext.create()#2698

Merged
jiayuasu merged 1 commit intomasterfrom
fix-rs-function-warning-messages
Mar 9, 2026
Merged

[GH-2640] Simplify warning messages of SedonaContext.create()#2698
jiayuasu merged 1 commit intomasterfrom
fix-rs-function-warning-messages

Conversation

@jiayuasu
Copy link
Member

@jiayuasu jiayuasu commented Mar 9, 2026

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

When SedonaContext.create() is called a second time, it prints warning messages about functions and UDTs being already registered. This was partially fixed for ST functions but RS functions and UDT registrations still produced warnings.

This PR adds existence checks before registering:

  1. RS aggregate functions (RS_Union_Aggr): Check functionRegistry.functionExists() before calling sparkSession.udf.register() in RasterRegistrator
  2. UDT types (Geometry, Geography, SpatialIndex, GridCoverage2D): Check UDTRegistration.exists() before calling UDTRegistration.register() in UdtRegistratorWrapper and RasterUdtRegistratorWrapper
  3. Viz UDT types (ImageSerializableWrapper, Pixel): Same check in viz UdtRegistratorWrapper

How was this patch tested?

Tested by calling SedonaContext.create() twice and verifying no warning messages are produced. Also ran all 166 raster algebra tests to ensure RS functions remain functional.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

Skip duplicate registration of UDT types and raster aggregate functions when SedonaContext.create() is called multiple times, eliminating spurious warning messages.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses repeated-registration warnings emitted when SedonaContext.create() is invoked multiple times by adding “already registered” guards for raster aggregate UDFs and several UDT registrations.

Changes:

  • Guard Sedona Viz UDT registration (ImageSerializableWrapper, Pixel) with UDTRegistration.exists.
  • Refactor Sedona SQL UDT registration (Geometry, Geography, SpatialIndex) to register only if missing.
  • Guard raster UDT registration and raster aggregate UDAF registration to avoid duplicate function registration warnings.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
spark/common/src/main/scala/org/apache/spark/sql/sedona_viz/UDT/UdtRegistratorWrapper.scala Adds UDTRegistration.exists checks before registering viz UDTs.
spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/UDT/UdtRegistratorWrapper.scala Adds a helper to conditionally register core Sedona SQL UDTs only when missing.
spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/UDT/RasterUdtRegistratorWrapper.scala Adds UDTRegistration.exists guard for raster UDT registration.
spark/common/src/main/scala/org/apache/sedona/sql/RasterRegistrator.scala Checks the function registry before registering the raster aggregate UDAF to avoid duplicate-registration warnings.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jiayuasu jiayuasu merged commit 4e8c317 into master Mar 9, 2026
45 checks passed
@jiayuasu jiayuasu deleted the fix-rs-function-warning-messages branch March 9, 2026 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simply the warning messages of SedonaContext.create()

2 participants