Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SEDONA-498] Add Spheroidal ST_Buffer #1251

Merged
merged 60 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
2af48a3
Init: ST_Buffer Spheroidal variant
prantogg Feb 8, 2024
62908bb
Merge branch 'master' into spheroidal-ST_Buffer
prantogg Feb 8, 2024
cbfd51d
Add tests
prantogg Feb 9, 2024
d01819e
make BestSRID ST function; Add tests
prantogg Feb 20, 2024
27f1f6b
Implement ST_BestSRID
prantogg Feb 20, 2024
0f89438
fix typos
prantogg Feb 20, 2024
6881f06
fix test case; Add comments
prantogg Feb 20, 2024
412a8d4
Add docs; Fix test case
prantogg Feb 20, 2024
244a348
Docs: fix typo
prantogg Feb 20, 2024
b66dc2a
Docs: add Snowflake doc
prantogg Feb 21, 2024
12c1af6
add flink and scala tests
prantogg Feb 21, 2024
600ebe1
Docs: Fix indentation
prantogg Feb 21, 2024
c59b8b9
Fix Snowflake test case
prantogg Feb 21, 2024
1ece852
Fix Snowflake compile issue; Update doc
prantogg Feb 21, 2024
6e8b513
refactor snowflake test
prantogg Feb 21, 2024
c1d8b67
Docs: update
prantogg Feb 21, 2024
06fc006
Merge branch 'apache:master' into spheroidal-ST_Buffer
prantogg Feb 21, 2024
22b6aed
Merge branch 'add-ST_BestSRID' into spheroidal-ST_Buffer
prantogg Feb 21, 2024
fe685fd
Update docs; Add python tests
prantogg Feb 22, 2024
982be17
nit: update docs, change useSpheroidal to useSpheroid
prantogg Feb 22, 2024
79dc271
docs: update
prantogg Feb 22, 2024
3f57294
Merge branch 'master' into spheroidal-ST_Buffer
prantogg Feb 22, 2024
69a2352
Fix duplicates
prantogg Feb 22, 2024
d71f9c5
Fix lint; fix scala/java test
prantogg Feb 22, 2024
2c2d97e
Change function signature; fix tests
prantogg Feb 23, 2024
977ee84
move spheroid calculation to FunctionGeotools; Fix python test
prantogg Feb 24, 2024
ab72359
Enforce lon/lat input geometries for ST_BestSRID and ST_Buffer sphero…
prantogg Feb 25, 2024
8f994fc
Docs: Update
prantogg Feb 25, 2024
7542d75
Fix typo
prantogg Feb 25, 2024
ddb795e
Fix incorrect wrong type config
prantogg Feb 25, 2024
754b263
Fix python test
prantogg Feb 25, 2024
632a878
Add handling for geometries crossing the dateline
prantogg Feb 26, 2024
1c07550
add geometryChanged() to normalizeLongitude()
prantogg Feb 27, 2024
9dbe575
Use CoordinateSequenceFilter for ST_ShiftLongitude
prantogg Feb 28, 2024
fd3712f
change return type to Geometry for shiftLongitude
prantogg Feb 28, 2024
06ba679
Merge branch 'master' into spheroidal-ST_Buffer
prantogg Feb 28, 2024
0c882c0
Remove duplicate functions and tests
prantogg Feb 28, 2024
64ce098
Fix ST_Buffer definition in python
prantogg Feb 28, 2024
1a4e9f5
Add support in snowflake; add snowflake docs
prantogg Feb 28, 2024
afb528c
Fix class definition for snowflake
prantogg Feb 28, 2024
a1674ac
update python test case
prantogg Feb 29, 2024
8af5ec3
Fix python test
prantogg Feb 29, 2024
12c5760
Fix python test
prantogg Feb 29, 2024
0f52875
Merge branch 'master' into spheroidal-ST_Buffer
prantogg Feb 29, 2024
91a7845
Update
prantogg Feb 29, 2024
2227a88
Update
prantogg Feb 29, 2024
b73d2ad
Fix snowflake error
prantogg Feb 29, 2024
0d9071d
Update snowflake test
prantogg Feb 29, 2024
3282d5e
fix snowflake test
prantogg Feb 29, 2024
05815bd
fix snowflake test
prantogg Feb 29, 2024
48b9aaf
fix snowflake test
prantogg Feb 29, 2024
52d2ad8
fix snowflake test
prantogg Feb 29, 2024
5b60a38
fix snowflake test
prantogg Feb 29, 2024
94272c5
fix snowflake test
prantogg Feb 29, 2024
17768ce
fix snowflake test
prantogg Feb 29, 2024
cc3a1c4
fix snowflake test
prantogg Feb 29, 2024
2d05970
Update Docs
prantogg Feb 29, 2024
c05cc4e
Address comments
prantogg Mar 3, 2024
877db2d
Address comments
prantogg Mar 4, 2024
8bef013
Address comments - update docs
prantogg Mar 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
59 changes: 46 additions & 13 deletions common/src/main/java/org/apache/sedona/common/Functions.java
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@
import org.locationtech.jts.operation.valid.TopologyValidationError;
import org.locationtech.jts.precision.GeometryPrecisionReducer;
import org.locationtech.jts.simplify.TopologyPreservingSimplifier;
import org.opengis.referencing.FactoryException;
import org.opengis.referencing.operation.TransformException;
import org.wololo.jts2geojson.GeoJSONWriter;
import java.util.ArrayList;
import java.util.Arrays;
Expand Down Expand Up @@ -86,26 +88,57 @@ public static Geometry boundary(Geometry geometry) {
return boundary;
}

public static Geometry buffer(Geometry geometry, double radius) {
return buffer(geometry, radius, "");
public static Geometry buffer(Geometry geometry, double radius) throws FactoryException, TransformException {
return buffer(geometry, radius, "", false);
}

public static Geometry buffer(Geometry geometry, double radius, String params) {
if (params.isEmpty()) {
return BufferOp.bufferOp(geometry, radius);
}
public static Geometry buffer(Geometry geometry, double radius, String params) throws FactoryException, TransformException {
return buffer(geometry, radius, params, false);
}

public static Geometry buffer(Geometry geometry, double radius, String params, boolean useSpheroid) throws FactoryException, TransformException {
BufferParameters bufferParameters = new BufferParameters();

BufferParameters bufferParameters = parseBufferParams(params);
// Processing parameters
if (!params.isEmpty()) {
bufferParameters = parseBufferParams(params);

// convert the sign to the appropriate direction
// left - radius should be positive
// right - radius should be negative
if (bufferParameters.isSingleSided() &&
(params.toLowerCase().contains("left") && radius < 0 || params.toLowerCase().contains("right") && radius > 0)) {
// convert the sign to the appropriate direction
jiayuasu marked this conversation as resolved.
Show resolved Hide resolved
// left - radius should be positive
// right - radius should be negative
if (bufferParameters.isSingleSided() &&
(params.toLowerCase().contains("left") && radius < 0 || params.toLowerCase().contains("right") && radius > 0)) {
radius = -radius;
}
}

return BufferOp.bufferOp(geometry, radius, bufferParameters);
if (useSpheroid) {
// Spheroidal buffering logic

// Determine the best SRID for spheroidal calculations
int bestCRS = bestSRID(geometry);
int originalCRS = geometry.getSRID();
final int WGS84CRS = 4326;

// If originalCRS is not set, use WGS84 as the originalCRS for transformation
String sourceCRSCode = (originalCRS == 0) ? "EPSG:" + WGS84CRS : "EPSG:" + originalCRS;
String targetCRSCode = "EPSG:" + bestCRS;

// Transform the geometry to the selected SRID
Geometry transformedGeometry = FunctionsGeoTools.transform(geometry, sourceCRSCode, targetCRSCode);

// Apply the buffer operation in the selected SRID
Geometry bufferedGeometry = BufferOp.bufferOp(transformedGeometry, radius, bufferParameters);

// Transform back to the original SRID or to WGS 84 if original SRID was not set
int backTransformCRSCode = (originalCRS == 0) ? WGS84CRS : originalCRS;
Geometry bufferedResult = FunctionsGeoTools.transform(bufferedGeometry, targetCRSCode, "EPSG:" + backTransformCRSCode);
bufferedResult.setSRID(backTransformCRSCode);
return bufferedResult;
} else {
// Existing planar buffer logic with params handling
return BufferOp.bufferOp(geometry, radius, bufferParameters);
}
}

private static BufferParameters parseBufferParams(String params) {
Expand Down

Large diffs are not rendered by default.

23 changes: 19 additions & 4 deletions docs/api/flink/Function.md
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,9 @@ Output: `LINESTRING Z(-1 -1 0, 10 5 5)`

## ST_Buffer

Introduction: Returns a geometry/geography that represents all points whose distance from this Geometry/geography is less than or equal to distance.
Introduction: Returns a geometry/geography that represents all points whose distance from this Geometry/geography is less than or equal to distance. The function supports both Planar/Euclidean and Spheroidal/Geodesic buffering (Since v1.6.0).

Buffer Style Parameters:

The optional third parameter controls the buffer accuracy and style. Buffer accuracy is specified by the number of line segments approximating a quarter circle, with a default of 8 segments. Buffer style can be set by providing blank-separated key=value pairs in a list format.

Expand All @@ -563,18 +565,31 @@ The optional third parameter controls the buffer accuracy and style. Buffer accu
- `mitre_limit=#.#` : mitre ratio limit and it only affects mitred join style. `miter_limit` is an accepted synonym for `mitre_limit`.
- `side=both|left|right` : The option `left` or `right` enables a single-sided buffer operation on the geometry, with the buffered side aligned according to the direction of the line. This functionality is specific to LINESTRING geometry and has no impact on POINT or POLYGON geometries. By default, square end caps are applied.

Mode of buffer calculation (Since: `v1.6.0`):

- Planar Buffering (default): When `useSpheroid` is false, `ST_Buffer` performs standard planar buffering based on the provided parameters.
- Spheroidal Buffering:
- When `useSpheroid` is set to true, the function returns the spheroidal buffer polygon for more accurate representation over the Earth.
- ST_Buffer first determines the most appropriate Spatial Reference Identifier (SRID) for a given geometry, based on its spatial extent and location, using `ST_BestSRID`.
- The geometry is then transformed from its original SRID to the selected SRID. If the input geometry does not have a set SRID, `ST_Buffer` defaults to using WGS 84 (SRID 4326) as its original SRID.
- The standard planar buffer operation is then applied in this coordinate system.
- Finally, the buffered geometry is transformed back to its original SRID, or to WGS 84 if the original SRID was not set.

!!!note
`ST_Buffer` throws an `IllegalArgumentException` if the correct format, parameters, or options are not provided.
`ST_Buffer` throws an `IllegalArgumentException` if the correct format, parameters, or options are not provided.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pay attention to the indent here. This will not render properly.


Format:

```
ST_Buffer (A: Geometry, buffer: Double, bufferStyleParameters: String [Optional])
```
```
ST_Buffer (A: Geometry, buffer: Double, bufferStyleParameters: String [Optional], useSperoidal: Boolean)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UseSpheroid parameter should be the 3rd param.

```

Since: `v1.5.1`

Example:
Spark SQL Example:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's stop calling any example Spark SQL example, even if in Spark. Let's use SQL Example everywhere.


```sql
SELECT ST_Buffer(ST_GeomFromWKT('POINT(0 0)'), 10)
Expand All @@ -588,7 +603,7 @@ Output:

8 Segments &ensp; 2 Segments

Example:
Spark SQL Example:

```sql
SELECT ST_Buffer(ST_GeomFromWKT('LINESTRING(0 0, 50 70, 100 100)'), 10, 'side=left')
Expand Down
16 changes: 15 additions & 1 deletion docs/api/sql/Function.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,7 +551,8 @@ Output: `LINESTRING Z(-1 -1 0, 10 5 5)`

## ST_Buffer

Introduction: Returns a geometry/geography that represents all points whose distance from this Geometry/geography is less than or equal to distance.
Introduction: Returns a geometry/geography that represents all points whose distance from this Geometry/geography is less than or equal to distance. The function supports both Planar/Euclidean and Spheroidal/Geodesic buffering (Since v1.6.0).
Buffer Style Parameters:

The optional third parameter controls the buffer accuracy and style. Buffer accuracy is specified by the number of line segments approximating a quarter circle, with a default of 8 segments. Buffer style can be set by providing blank-separated key=value pairs in a list format.

Expand All @@ -561,6 +562,16 @@ The optional third parameter controls the buffer accuracy and style. Buffer accu
- `mitre_limit=#.#` : mitre ratio limit and it only affects mitred join style. `miter_limit` is an accepted synonym for `mitre_limit`.
- `side=both|left|right` : The option `left` or `right` enables a single-sided buffer operation on the geometry, with the buffered side aligned according to the direction of the line. This functionality is specific to LINESTRING geometry and has no impact on POINT or POLYGON geometries. By default, square end caps are applied.

Mode of buffer calculation (Since: `v1.6.0`):

- Planar Buffering (default): When `useSpheroid` is false, `ST_Buffer` performs standard planar buffering based on the provided parameters.
- Spheroidal Buffering:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing. Please add that, when useSpheroid is set to true, the unit of the buffer is meter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- When `useSpheroid` is set to true, the function returns the spheroidal buffer polygon for more accurate representation over the Earth.
- ST_Buffer first determines the most appropriate Spatial Reference Identifier (SRID) for a given geometry, based on its spatial extent and location, using `ST_BestSRID`.
jiayuasu marked this conversation as resolved.
Show resolved Hide resolved
- The geometry is then transformed from its original SRID to the selected SRID. If the input geometry does not have a set SRID, `ST_Buffer` defaults to using WGS 84 (SRID 4326) as its original SRID.
- The standard planar buffer operation is then applied in this coordinate system.
- Finally, the buffered geometry is transformed back to its original SRID, or to WGS 84 if the original SRID was not set.

!!!note
`ST_Buffer` throws an `IllegalArgumentException` if the correct format, parameters, or options are not provided.

Expand All @@ -569,6 +580,9 @@ Format:
```
ST_Buffer (A: Geometry, buffer: Double, bufferStyleParameters: String [Optional])
```
```
ST_Buffer (A: Geometry, buffer: Double, bufferStyleParameters: String [Optional], useSpheroid: Boolean)
```

Since: `v1.5.1`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,16 @@
*/
package org.apache.sedona.flink.expressions;

import org.apache.calcite.runtime.Geometries;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.flink.table.annotation.DataTypeHint;
import org.apache.flink.table.annotation.InputGroup;
import org.apache.flink.table.functions.ScalarFunction;
import org.apache.sedona.common.FunctionsGeoTools;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.operation.buffer.BufferParameters;
import org.opengis.referencing.FactoryException;
import org.opengis.referencing.operation.TransformException;

import java.util.Arrays;

Expand Down Expand Up @@ -69,17 +72,24 @@ public Geometry eval(@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.j
public static class ST_Buffer extends ScalarFunction {
@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
public Geometry eval(@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
Object o, @DataTypeHint("Double") Double radius) {
Object o, @DataTypeHint("Double") Double radius) throws FactoryException, TransformException {
Geometry geom = (Geometry) o;
return org.apache.sedona.common.Functions.buffer(geom, radius);
}

@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
public Geometry eval(@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
Object o, @DataTypeHint("Double") Double radius, @DataTypeHint("String") String params) {
Object o, @DataTypeHint("Double") Double radius, @DataTypeHint("String") String params) throws FactoryException, TransformException {
Geometry geom = (Geometry) o;
return org.apache.sedona.common.Functions.buffer(geom, radius, params);
}

@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
public Geometry eval(@DataTypeHint(value = "RAW", bridgedTo = org.locationtech.jts.geom.Geometry.class)
Object o, @DataTypeHint("Double") Double radius, @DataTypeHint("String") String params, @DataTypeHint("Boolean") Boolean useSpheroid) throws FactoryException, TransformException {
Geometry geom = (Geometry) o;
return org.apache.sedona.common.Functions.buffer(geom, radius, params, useSpheroid);
}
}

public static class ST_BestSRID extends ScalarFunction {
Expand Down
10 changes: 10 additions & 0 deletions flink/src/test/java/org/apache/sedona/flink/FunctionTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,14 @@ public void testBuffer() {
actual = (String) first(tableEnv.sqlQuery("SELECT ST_AsText(ST_ReducePrecision(ST_Buffer(ST_Point(100, 90), 200, 'quad_segs=4'), 4))")).getField(0);
expected = "POLYGON ((284.7759 13.4633, 241.4214 -51.4214, 176.5367 -94.7759, 100 -110, 23.4633 -94.7759, -41.4214 -51.4214, -84.7759 13.4633, -100 90, -84.7759 166.5367, -41.4214 231.4214, 23.4633 274.7759, 100 290, 176.5367 274.7759, 241.4214 231.4214, 284.7759 166.5367, 300 90, 284.7759 13.4633))";
assertEquals(expected, actual);

actual = (String) first(tableEnv.sqlQuery("SELECT ST_AsText(ST_ReducePrecision(ST_Buffer(ST_GeomFromWKT('LINESTRING(-91.185 30.4505, -91.187 30.452, -91.189 30.4535)'), 10, 'side=left', true), 4))")).getField(0);
expected = "POLYGON ((-91.187 30.452, -91.185 30.4505, -91.1851 30.4504, -91.1871 30.4519, -91.1891 30.4534, -91.189 30.4535, -91.187 30.452))";
assertEquals(expected, actual);

actual = (String) first(tableEnv.sqlQuery("SELECT ST_AsText(ST_ReducePrecision(ST_Buffer(ST_GeomFromWKT('POLYGON((-120 30, -80 30, -80 50, -120 50, -120 30))'), 200, 'quad_segs=4', true), 4))")).getField(0);
expected = "POLYGON ((-120.0018 50, -120.0017 50.0004, -120.0013 50.0008, -120.0007 50.0011, -120 50.0012, -80 50.0012, -79.9993 50.0011, -79.9987 50.0008, -79.9983 50.0004, -79.9982 50, -79.9982 30, -79.9983 29.9994, -79.9987 29.9989, -79.9993 29.9986, -80 29.9984, -120 29.9984, -120.0007 29.9986, -120.0013 29.9989, -120.0017 29.9994, -120.0018 30, -120.0018 50))";
assertEquals(expected, actual);
}

@Test
Expand Down Expand Up @@ -122,6 +130,8 @@ public void testClosestPoint() {
Geometry result = (Geometry) first(table).getField(0);
assertEquals("POINT (160 40)", result.toString());
}

@Test
public void testCentroid() {
Table polygonTable = tableEnv.sqlQuery("SELECT ST_GeomFromText('POLYGON ((2 2, 0 0, 2 0, 0 2, 2 2))') as geom");
Table resultTable = polygonTable.select(call(Functions.ST_Centroid.class.getSimpleName(), $("geom")));
Expand Down
8 changes: 5 additions & 3 deletions python/sedona/sql/st_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ def ST_Boundary(geometry: ColumnOrName) -> Column:


@validate_argument_types
def ST_Buffer(geometry: ColumnOrName, buffer: ColumnOrNameOrNumber, parameters: Optional[Union[ColumnOrName, str]] = None) -> Column:
def ST_Buffer(geometry: ColumnOrName, buffer: ColumnOrNameOrNumber, parameters: Optional[Union[ColumnOrName, str]] = None, useSpheroid: Optional[Union[ColumnOrName, bool]] = None) -> Column:
"""Calculate a geometry that represents all points whose distance from the
input geometry column is equal to or less than a given amount.

Expand All @@ -341,10 +341,12 @@ def ST_Buffer(geometry: ColumnOrName, buffer: ColumnOrNameOrNumber, parameters:
:return: Buffered geometry as a geometry column.
:rtype: Column
"""
if parameters is None:
if parameters is None and useSpheroid is None:
args = (geometry, buffer)
else:
elif useSpheroid is None:
args = (geometry, buffer, parameters)
else:
args = (geometry, buffer, parameters, useSpheroid)

return _call_st_function("ST_Buffer", args)

Expand Down
1 change: 1 addition & 0 deletions python/tests/sql/test_dataframe_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
(stf.ST_BestSRID, ("geom",), "triangle_geom", "", 3395),
(stf.ST_Boundary, ("geom",), "triangle_geom", "", "LINESTRING (0 0, 1 0, 1 1, 0 0)"),
(stf.ST_Buffer, ("point", 1.0), "point_geom", "ST_ReducePrecision(geom, 2)", "POLYGON ((0.98 0.8, 0.92 0.62, 0.83 0.44, 0.71 0.29, 0.56 0.17, 0.38 0.08, 0.2 0.02, 0 0, -0.2 0.02, -0.38 0.08, -0.56 0.17, -0.71 0.29, -0.83 0.44, -0.92 0.62, -0.98 0.8, -1 1, -0.98 1.2, -0.92 1.38, -0.83 1.56, -0.71 1.71, -0.56 1.83, -0.38 1.92, -0.2 1.98, 0 2, 0.2 1.98, 0.38 1.92, 0.56 1.83, 0.71 1.71, 0.83 1.56, 0.92 1.38, 0.98 1.2, 1 1, 0.98 0.8))"),
(stf.ST_Buffer, ("point", 1.0, "", True), "point_geom", "ST_ReducePrecision(geom, 2)", "POLYGON ((0.98 0.8, 0.92 0.62, 0.83 0.44, 0.71 0.29, 0.56 0.17, 0.38 0.08, 0.2 0.02, 0 0, -0.2 0.02, -0.38 0.08, -0.56 0.17, -0.71 0.29, -0.83 0.44, -0.92 0.62, -0.98 0.8, -1 1, -0.98 1.2, -0.92 1.38, -0.83 1.56, -0.71 1.71, -0.56 1.83, -0.38 1.92, -0.2 1.98, 0 2, 0.2 1.98, 0.38 1.92, 0.56 1.83, 0.71 1.71, 0.83 1.56, 0.92 1.38, 0.98 1.2, 1 1, 0.98 0.8))"),
(stf.ST_BuildArea, ("geom",), "multiline_geom", "ST_Normalize(geom)", "POLYGON ((0 0, 1 1, 1 0, 0 0))"),
(stf.ST_BoundingDiagonal, ("geom",), "square_geom", "ST_BoundingDiagonal(geom)", "LINESTRING (1 0, 2 1)"),
(stf.ST_Centroid, ("geom",), "triangle_geom", "ST_ReducePrecision(geom, 2)", "POINT (0.67 0.33)"),
Expand Down