Provide a way to do spatial joins with custom judgement/iterator class.

[cakilandeswari]

Expected behavior

I have a use case where I want to join based on geodesic area overlap of 90% or more and I would like to define my own judgment.

In class org.datasyslab.geospark.spatialOperator.JoinQuery,

Add as below to allow users to specify their own operator for join and generating matching pairs.
public static final class JoinParams<U extends Object, T extends Object>
{
public final boolean useIndex;
public final boolean considerBoundaryIntersection;
public final boolean allowDuplicates;
public final IndexType indexType;
public final JoinBuildSide joinBuildSide;
public final FlatMapFunction2<Iterator, Iterator, Pair<U, T>> customJudgement;
}

and allow user to use it with

public static <U extends Geometry, T extends Geometry> JavaPairRDD<U, T> spatialJoin(
SpatialRDD leftRDD,
SpatialRDD rightRDD,
JoinParams joinParams) {
....
}

Actual behavior

JudgementBase is private and user does not have a way to specify their own judgement to use for spatial joins.

Steps to reproduce the problem

N/A

Settings

GeoSpark version = 1.2.3.1

Apache Spark version = 2.4.1

JRE version = 1.8?

API type = Scala or Java? Java

[jiayuasu]

@cakilandeswari I believe what you described can be easily done by adding one more filtering step after getting the spatial join query result. You just keep the pairs that have geodesic area overlap of 90%.

[cakilandeswari]

@jiayuasu When i started, I did try to add a UDF to compute geodesic area to use after converting the results from join RDD to a data frame, my workflow could never complete due to to memory issues. It doesn't help that shapes need to be converted to strings when converting from RDD to data frames due to the removal of user defined data types in a sql row.

The shapes that i have are pretty huge and of bigger numbers (imagine all possible places in united states) and filtering early helps me complete the join successfully.

By adding my own judgement, I could filter out early and i also optimized on just returning the centroid+metadata in my shape pairs rather than returning complete shapes.

Since this is custom optimization, developers like me would highly benefit from specifying a custom iterator/judgment. Also the spatial join api is touted as for developers who want to do this sort of detailed optimization and will not affect other APIs.