Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Read SQL Servers binary format directly #174

Closed
wants to merge 4 commits into from

3 participants

@bakk

Read SQL Server binary directly into JTS Geometries, instead of going
via WKB. While geotools will use a little longer to decode (about 50% more
compared to WKB), SQL Server won't have to encode at all, resulting in a performance
improvement.

A simple benchmark showed a performance increase around 50% on a
layer with many small geometries (a grid), while another layer only showed
7% performance increase.

The binary reader only support v1 of the specification. V2 adds support for CircularString, CompoundCurve, CurvePolygon, FullGlobe (http://go.microsoft.com/fwlink/?LinkId=226407), and is not supported by today's WKB implementation either (as far as I can see).

It would be nice if the added classes (for parsing the sql server binary) could be licenced freely (eg. apache/bsd), but if you would like it to be LGPL like the rest, that's OK too.

It's important to review this thoroughly so that everything that today's implementation supports still works :)

Anders Bakkevold Read SQL Servers binary format directly
Read SQL Server binary directly into JTS Geometries, instead of going
via WKB. While geotools will use a little longer to decode (than WKB),
SQL Server won't have to encode at all, resulting in a performance
improvement.
acd8e36
@aaime
Owner

Everything that becomes part of GeoTools has to be licenced under LGPL. I'll have a look at the patches as spare time allows (or maybe Justin wants to?). Wondering, are V1 and V2 specs compatible when it comes to the same geometry type? Like, Point, Line and Polygon are encoded the same way?
Given this is new code, I'm wondering if having a configurable choice between WKB and native binary would be better

@bakk

About the license: that's ok.
I forgot to include a link to the actual spec: http://msdn.microsoft.com/en-us/library/ee320529(v=sql.105).aspx
The differences between V1 and V2 are:

  • new geometry types, as mentioned
  • a new flag to indicate that a geography is larger than a hemisphere
  • a segments-structure (used for compound curves)
  • the definition of "figure attribute" has been changed

To support points, lines, polygons in V2, you would at least have to support the new definition of "figure attribute". Otherwise the formats are the same. However, I downloaded the latest SQL Server 2012 release, and when I created a new geometry, I got version 1. I've also tested with SQL Server 2008 R2. So the fact that I wasn't able to produce test-data with SQL Server stopped me from implementing partial support for V2.

If you could point me in the right direction of how to make the choice configurable, I can implement it. How would the user specify it - in the GUI?

@aaime
Owner

Just add a new configuration parameter in the factory, that gets eventually set in the dialect, like the one for supporting native paging. The GUI in GeoServer is automatically generated from the parameter list (you'll have to also add it to the list of params advertised by the factory, and its jndi equivalent).

Anders Bakkevold Make it optional to use sql server binary serialization format
Added a parameter to switch between WKB and SQL Server binary as
serialization format
9f9c70b
@bakk

I've added a new configuration parameter. It defaults to using WKB. If the new code proves to be stable, I suggest switching default settings later.

...n/java/org/geotools/data/sqlserver/Reader/Figure.java
@@ -0,0 +1,25 @@
+package org.geotools.data.sqlserver.Reader;
@jdeolive Owner
jdeolive added a note

We try to stick to the convention of keeping package names lowercase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jdeolive
Owner

First off great work! This is something i have wanted to see for a while since as you note it can have a huge performance impact.

I think the patch looks great for the most part, except for the optional configuration parameter, which as implemented will still change the default for people upgrading. I think instead of explicitly having to enable wkb serialization we should have to explicitly enable native serializastion. So i would change the parameter to "NATIVE_SERIALIZATION" and have it by default set to false.

As you say if it shows to be stable we can make it the default in a future version.

Anders Bakkevold Changed WKB_SERIALIZATION to NATIVE_SERIALIZATION
Additionally the parameter is moved up, which groups it together with
two other boolean params in Geoserver's GUI. Corrected package name to
lowercase.
05fbd8c
@bakk

You are correct - changing default for people upgrading was unintended. I've corrected it, and I also took the liberty to move the parameter up by one position in the list of advertised parameters. I just thought it looked better in GeoServer. The casing of the parameters (as they appear in GeoServer) is inconsistent, so I'm not sure if there is a correct way.

Another thing: I believe all the tests now run with the default setting (WKB), so the native implementation is not tested (other than the unit tests which test serialization directly). Not sure if this is a problem.

@jdeolive
Owner

Great, thanks for changing that. And yeah, the casing is inconsistent so i wouldn't worry about that.

Indeed it would be nice to have at least one of the tests run with the native serialization, ideally one of the tests that involves reading geometries, like SQLServerFeatureSourceTest. It should just be a matter of extending the existing class and overriding the setupDataStore method to set the native serialization. But I will leave that up to you, i don't see it as a blocker for this change since it already has a good test that exercise the changes.

So as far as i am concerned the patch is good but i'll give Andrea a chance to weigh in as well before merge.

@aaime aaime closed this
@aaime
Owner

Thanks a lot, looking good. I've manually merged it and added a few of changes:

  • there was a package named Reader instead of reader, as a result the code would not even compile on a case sensitive file system like Linux has, fixed it
  • made it clear that the serialization format is for the geometries ("use native geometry serialization format")
  • tested it with several geometry types with success, observed the speedup by the naked eye on large enough data sets
  • created a ticket in jira (http://jira.codehaus.org/browse/GEOT-4442), squashed all commits to one and reworded the commit message to refer to the jira ticket

The full result is here:
9d7f87d

Given that the behavior is optional and I've found no issues in the code or tests, I've also backported the patch to the stable series (9.x).

@bakk

That's great! Thank you both for reviewing it so quickly!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 4, 2013
  1. Read SQL Servers binary format directly

    Anders Bakkevold authored
    Read SQL Server binary directly into JTS Geometries, instead of going
    via WKB. While geotools will use a little longer to decode (than WKB),
    SQL Server won't have to encode at all, resulting in a performance
    improvement.
  2. Make it optional to use sql server binary serialization format

    Anders Bakkevold authored
    Added a parameter to switch between WKB and SQL Server binary as
    serialization format
  3. Changed WKB_SERIALIZATION to NATIVE_SERIALIZATION

    Anders Bakkevold authored
    Additionally the parameter is moved up, which groups it together with
    two other boolean params in Geoserver's GUI. Corrected package name to
    lowercase.
Commits on Apr 5, 2013
  1. Use native serialization in SQLServerFeatureSourceTest

    Anders Bakkevold authored
This page is out of date. Refresh to see the latest.
Showing with 670 additions and 38 deletions.
  1. +25 −0 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Figure.java
  2. +31 −0 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Shape.java
  3. +103 −0 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinary.java
  4. +15 −0 ...gin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinaryParseException.java
  5. +268 −0 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinaryReader.java
  6. +36 −0 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Type.java
  7. +12 −3 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerDataStoreFactory.java
  8. +37 −33 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerDialect.java
  9. +1 −0  ...les/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerJNDIDataStoreFactory.java
  10. +2 −2 modules/plugin/jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/SQLServerFeatureSourceTest.java
  11. +20 −0 ...in/jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/SqlServerNativeSerializationTestSetup.java
  12. +120 −0 .../plugin/jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/reader/SQLServerBinaryReaderTest.java
View
25 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Figure.java
@@ -0,0 +1,25 @@
+package org.geotools.data.sqlserver.reader;
+
+/**
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+public class Figure {
+
+ private int attribute;
+ private int pointOffset;
+
+ public Figure(int attribute, int pointOffset) {
+ this.attribute = attribute;
+ this.pointOffset = pointOffset;
+ }
+
+ public int getAttribute() {
+ return attribute;
+ }
+
+ public int getPointOffset() {
+ return pointOffset;
+ }
+}
View
31 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Shape.java
@@ -0,0 +1,31 @@
+package org.geotools.data.sqlserver.reader;
+
+/**
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+public class Shape {
+
+ private int parentOffset;
+ private int figureOffset;
+ private Type type;
+
+ public Shape(int parentOffset, int figureOffset, int type) {
+ this.parentOffset = parentOffset;
+ this.figureOffset = figureOffset;
+ this.type = Type.findType(type);
+ }
+
+ public int getParentOffset() {
+ return parentOffset;
+ }
+
+ public int getFigureOffset() {
+ return figureOffset;
+ }
+
+ public Type getType() {
+ return type;
+ }
+}
View
103 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinary.java
@@ -0,0 +1,103 @@
+package org.geotools.data.sqlserver.reader;
+
+import com.vividsolutions.jts.geom.Coordinate;
+import com.vividsolutions.jts.geom.CoordinateSequence;
+
+/**
+ * Represents the information from a binary sqlserver geometry
+ *
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+class SqlServerBinary {
+
+ private int srid;
+ private int numberOfPoints;
+ private Coordinate[] coordinates;
+ private Shape[] shapes;
+ private Figure[] figures;
+ private CoordinateSequence[] sequences;
+
+ public int getSrid() {
+ return srid;
+ }
+
+ public void setSrid(int srid) {
+ this.srid = srid;
+ }
+
+ public void setSerializationProperties(byte serializationProperties) {
+ this.serializationProperties = serializationProperties;
+ }
+
+ public boolean hasZ() {
+ return (serializationProperties & 1) == 1;
+ }
+
+ public boolean hasM() {
+ return (serializationProperties & 2) == 2;
+ }
+
+ public boolean isValid(){
+ return (serializationProperties & 4) == 4;
+ }
+
+ public boolean isSinglePoint() {
+ return (serializationProperties & 8) == 8;
+ }
+
+ public boolean hasSingleLineSegment() {
+ return (serializationProperties & 16) == 16;
+ }
+
+ private byte serializationProperties;
+
+ public int getNumberOfPoints() {
+ return numberOfPoints;
+ }
+
+ public void setNumberOfPoints(int numberOfPoints) {
+ this.numberOfPoints = numberOfPoints;
+ }
+
+ public Coordinate[] getCoordinates() {
+ return coordinates;
+ }
+
+ public void setCoordinates(Coordinate[] coordinates) {
+ this.coordinates = coordinates;
+ }
+
+ public void setShapes(Shape[] shapes) {
+ this.shapes = shapes;
+ }
+
+ public void setFigures(Figure[] figures) {
+ this.figures = figures;
+ }
+
+ public Figure[] getFigures() {
+ return figures;
+ }
+
+ public void setSequences(CoordinateSequence[] sequences) {
+ this.sequences = sequences;
+ }
+
+ public Shape[] getShapes() {
+ return shapes;
+ }
+
+ public Shape getShape(int index) {
+ return shapes[index];
+ }
+
+ public Figure getFigure(int index) {
+ return figures[index];
+ }
+
+ public CoordinateSequence getSequence(int index) {
+ return sequences[index];
+ }
+}
View
15 .../jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinaryParseException.java
@@ -0,0 +1,15 @@
+package org.geotools.data.sqlserver.reader;
+
+import java.io.IOException;
+
+/**
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+public class SqlServerBinaryParseException extends IOException {
+
+ public SqlServerBinaryParseException(String message) {
+ super(message);
+ }
+}
View
268 ...s/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/SqlServerBinaryReader.java
@@ -0,0 +1,268 @@
+package org.geotools.data.sqlserver.reader;
+
+import com.vividsolutions.jts.geom.*;
+import com.vividsolutions.jts.io.ByteArrayInStream;
+import com.vividsolutions.jts.io.ByteOrderDataInStream;
+import com.vividsolutions.jts.io.ByteOrderValues;
+import com.vividsolutions.jts.io.InStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+/**
+ * Decode Sql Server binary format to JTS
+ *
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+public class SqlServerBinaryReader {
+
+ private ByteOrderDataInStream dis = new ByteOrderDataInStream();
+ private GeometryFactory gf = new GeometryFactory();
+ private SqlServerBinary binary;
+
+ public SqlServerBinaryReader() {
+ this.gf = new GeometryFactory();
+ }
+
+ public SqlServerBinaryReader(GeometryFactory gf) {
+ this.gf = gf;
+ }
+
+ public Geometry read(byte[] bytes) throws IOException {
+ this.binary = new SqlServerBinary();
+ return read(new ByteArrayInStream(bytes));
+ }
+
+ public Geometry read(InStream is) throws IOException {
+ parse(is);
+ readCoordinateSequences();
+ Type type = getTypeFromBinary();
+ Geometry geometry = decode(0, type);
+ geometry.setSRID(binary.getSrid());
+ return geometry;
+ }
+
+ private Geometry decode(int shapeIndex, Type type) throws SqlServerBinaryParseException {
+ switch (type) {
+ case GEOMETRYCOLLECTION:
+ return decodeGeometryCollection(shapeIndex);
+ case POINT:
+ return decodePoint(shapeIndex);
+ case LINESTRING:
+ return decodeLinestring(shapeIndex);
+ case POLYGON:
+ return decodePolygon(shapeIndex);
+ case MULTILINESTRING:
+ return decodeMultiLinestring(shapeIndex);
+ case MULTIPOINT:
+ return decodeMultiPoint(shapeIndex);
+ case MULTIPOLYGON:
+ return decodeMultiPolygon(shapeIndex);
+ default:
+ throw new SqlServerBinaryParseException("Geometry type unsupported " + type);
+ }
+ }
+
+ private Geometry decodeMultiPolygon( int shapeIndex) {
+ Collection<Geometry> polygons = new ArrayList<Geometry>();
+ for (int i = shapeIndex; i < binary.getShapes().length; i++) {
+ if (binary.getShape(i).getParentOffset() == shapeIndex) {
+ polygons.add(gf.createPolygon(binary.getSequence(binary.getShape(i).getFigureOffset())));
+ }
+ }
+ return gf.createMultiPolygon(polygons.toArray(new Polygon[polygons.size()])); }
+
+ private Geometry decodeMultiPoint(int shapeIndex) {
+ Collection<Geometry> points = new ArrayList<Geometry>();
+ for (int i = shapeIndex; i < binary.getShapes().length; i++) {
+ if (binary.getShape(i).getParentOffset() == shapeIndex) {
+ points.add(gf.createPoint(binary.getSequence(binary.getShape(i).getFigureOffset())));
+ }
+ }
+ return gf.createMultiPoint(points.toArray(new Point[points.size()]));
+ }
+
+ private Geometry decodeMultiLinestring(int shapeIndex) {
+ Collection<Geometry> linestrings = new ArrayList<Geometry>();
+ for (int i = shapeIndex; i < binary.getShapes().length; i++) {
+ if (binary.getShape(i).getParentOffset() == shapeIndex) {
+ linestrings.add(gf.createLineString(binary.getSequence(binary.getShape(i).getFigureOffset())));
+ }
+ }
+ return gf.createMultiLineString(linestrings.toArray(new LineString[linestrings.size()]));
+ }
+
+ private Geometry decodePolygon(int shapeIndex) {
+ Shape shape = binary.getShape(shapeIndex);
+ int figureOffset = shape.getFigureOffset();
+ int figureStopIndex = binary.getFigures().length-1;
+ if (shapeIndex +1 < binary.getShapes().length) {
+ Shape nextShape = binary.getShape(shapeIndex + 1);
+ figureStopIndex = nextShape.getFigureOffset()-1;
+ }
+ List<LinearRing> linearRings = new ArrayList<LinearRing>();
+ if (figureOffset <= -1) {
+ return gf.createPolygon(new Coordinate[0]);
+ }
+ for (int i = figureOffset; i <= figureStopIndex; i++) {
+ CoordinateSequence sequence = binary.getSequence(i);
+ linearRings.add(gf.createLinearRing(sequence));
+ }
+ LinearRing outerShell = linearRings.remove(0);
+ LinearRing[] holes = linearRings.toArray(new LinearRing[linearRings.size()]);
+ return gf.createPolygon(outerShell, holes);
+ }
+
+ private Geometry decodeLinestring(int shapeIndex) {
+ Shape shape = binary.getShape(shapeIndex);
+ CoordinateSequence sequence = binary.getSequence(shape.getFigureOffset());
+ return gf.createLineString(sequence);
+ }
+
+ private Geometry decodePoint(int shapeIndex) {
+ Shape shape = binary.getShapes()[shapeIndex];
+ Coordinate coordinate;
+ if (binary.isSinglePoint()) {
+ coordinate = binary.getCoordinates()[0];
+ } else if (shape.getParentOffset() != -1) {
+ Figure figure = binary.getFigure(shape.getFigureOffset());
+ coordinate = binary.getCoordinates()[figure.getPointOffset()];
+ } else {
+ coordinate = null;
+ }
+ return gf.createPoint(coordinate);
+ }
+
+ private Geometry decodeGeometryCollection(int shapeIndex) throws SqlServerBinaryParseException {
+ Collection<Geometry> geometries = new ArrayList<Geometry>();
+ for (int i = shapeIndex +1; i < binary.getShapes().length; i++) {
+ Shape subShape = binary.getShapes()[i];
+ if (subShape.getParentOffset() == shapeIndex) {
+ geometries.add(decode(i, subShape.getType()));
+ }
+ }
+ return gf.buildGeometry(geometries);
+ }
+
+ private Type getTypeFromBinary() {
+ if (binary.isSinglePoint()) {
+ return Type.POINT;
+ }
+ if (binary.hasSingleLineSegment()) {
+ return Type.LINESTRING;
+ }
+ return binary.getShapes()[0].getType();
+ }
+
+ private void readCoordinateSequences() {
+ Figure[] figures = binary.getFigures();
+ CoordinateSequence[] sequences = new CoordinateSequence[figures.length];
+ for (int i = 0; i < figures.length; i++) {
+ int figurePointOffset = figures[i].getPointOffset();
+ int nextPointOffset = figures.length >= i+2 ? figures[i+1].getPointOffset() : binary.getCoordinates().length;
+ Coordinate[] coordinates = Arrays.copyOfRange(binary.getCoordinates(), figurePointOffset, nextPointOffset);
+ int attribute = figures[i].getAttribute();
+ if ((attribute == 0 || attribute == 2) && !coordinates[0].equals(coordinates[coordinates.length-1]) ) {
+ coordinates = Arrays.copyOf(coordinates, coordinates.length + 1);
+ coordinates[coordinates.length-1] = coordinates[0];
+ }
+ sequences[i] = gf.getCoordinateSequenceFactory().create(coordinates);
+ }
+ binary.setSequences(sequences);
+ }
+
+ private void parse(InStream is) throws IOException {
+ dis.setInStream(is);
+ dis.setOrder(ByteOrderValues.LITTLE_ENDIAN);
+ binary.setSrid(dis.readInt());
+ byte version = dis.readByte();
+ if (version != 1) {
+ throw new SqlServerBinaryParseException("Unsupported version (only supports version 1): " + version);
+ }
+ binary.setSerializationProperties(dis.readByte());
+
+ readNumberOfPoints();
+ readCoordinates();
+ readZValues();
+ readMValues();
+
+ if (binary.isSinglePoint()) {
+ binary.setFigures(new Figure[] { new Figure(1,0) });
+ binary.setShapes(new Shape[] { new Shape(-1,0,2)});
+ } else if (binary.hasSingleLineSegment()) {
+ binary.setFigures(new Figure[] { new Figure(1,0) });
+ binary.setShapes(new Shape[] { new Shape(-1,0,1)});
+ } else {
+ readFigures();
+ readShapes();
+ }
+ }
+
+ private void readNumberOfPoints() throws IOException {
+ if (binary.isSinglePoint()) {
+ binary.setNumberOfPoints(1);
+ } else if (binary.hasSingleLineSegment()) {
+ binary.setNumberOfPoints(2);
+ } else {
+ binary.setNumberOfPoints(dis.readInt());
+ }
+ }
+
+ private void readCoordinates() throws IOException {
+ Coordinate[] coordinates = new Coordinate[binary.getNumberOfPoints()];
+ for(int i = 0; i<binary.getNumberOfPoints(); i++) {
+ coordinates[i] = readCoordinate();
+ }
+ binary.setCoordinates(coordinates);
+ }
+
+ private void readShapes() throws IOException {
+ int numberOfShapes;Shape[] shapesMetadata;
+ numberOfShapes = dis.readInt();
+ shapesMetadata = new Shape[numberOfShapes];
+ for (int i = 0; i < numberOfShapes; i++) {
+ int parentOffset = dis.readInt();
+ int figureOffset = dis.readInt();
+ int shapeType = dis.readByte();
+ shapesMetadata[i] = new Shape(parentOffset, figureOffset, shapeType);
+ }
+ binary.setShapes(shapesMetadata);
+ }
+
+ private void readFigures() throws IOException {
+ int numberOfFigures;Figure[] figuresMetadata;
+ numberOfFigures = dis.readInt();
+ figuresMetadata = new Figure[numberOfFigures];
+ for (int i = 0; i < numberOfFigures; i++) {
+ byte figureAttribute = dis.readByte();
+ int figurePointOffset = dis.readInt();
+ figuresMetadata[i] = new Figure(figureAttribute,figurePointOffset);
+ }
+ binary.setFigures(figuresMetadata);
+ }
+
+ private void readMValues() throws IOException {
+ //measure values are currently discarded, as they cannot be represented in a JTS Geometry
+ if (binary.hasM()) {
+ for (int i = 0; i < binary.getNumberOfPoints(); i++) {
+ dis.readDouble();
+ }
+ }
+ }
+
+ private void readZValues() throws IOException {
+ if (binary.hasZ()) {
+ for (int i = 0; i < binary.getNumberOfPoints(); i++) {
+ binary.getCoordinates()[i].z = dis.readDouble();
+ }
+ }
+ }
+
+ private Coordinate readCoordinate() throws IOException {
+ return new Coordinate(dis.readDouble(), dis.readDouble());
+ }
+}
View
36 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/Reader/Type.java
@@ -0,0 +1,36 @@
+package org.geotools.data.sqlserver.reader;
+
+/**
+ * @author Anders Bakkevold, Bouvet
+ *
+ * @source $URL$
+ */
+public enum Type {
+
+ POINT (1),
+ LINESTRING(2),
+ POLYGON(3),
+ MULTIPOINT(4),
+ MULTILINESTRING(5),
+ MULTIPOLYGON(6),
+ GEOMETRYCOLLECTION(7),
+ CIRCULARSTRING(8),
+ COMPOUNDCURVE(9),
+ CURVEPOLYGON(10),
+ FULLGLOBE(11);
+
+ private int value;
+
+ private Type(int value) {
+ this.value = value;
+ }
+
+ public static Type findType(int value) {
+ for (Type type : Type.values()) {
+ if (type.value == value) {
+ return type;
+ }
+ }
+ throw new IllegalArgumentException();
+ }
+}
View
15 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerDataStoreFactory.java
@@ -19,7 +19,6 @@
import java.io.IOException;
import java.util.Map;
-import org.geotools.data.DataAccessFactory.Param;
import org.geotools.jdbc.JDBCDataStore;
import org.geotools.jdbc.JDBCDataStoreFactory;
import org.geotools.jdbc.SQLDialect;
@@ -47,7 +46,10 @@
/** Metadata table providing information about primary keys **/
public static final Param GEOMETRY_METADATA_TABLE = new Param("Geometry metadata table", String.class,
"The optional table containing geometry metadata (geometry type and srid). Can be expressed as 'schema.name' or just 'name'", false);
-
+
+ /** parameter for using WKB or Sql server binary directly. Setting to true will use WKB */
+ public static final Param NATIVE_SERIALIZATION = new Param("Use native serialization", Boolean.class,
+ "Use native SQL Server serialization, or WKB serialization.", false, Boolean.FALSE);
@Override
protected SQLDialect createSQLDialect(JDBCDataStore dataStore) {
@@ -80,6 +82,7 @@ protected void setupParameters(Map parameters) {
parameters.put(DBTYPE.key, DBTYPE);
parameters.put(INTSEC.key, INTSEC);
parameters.put(NATIVE_PAGING.key, NATIVE_PAGING);
+ parameters.put(NATIVE_SERIALIZATION.key, NATIVE_SERIALIZATION);
parameters.put(GEOMETRY_METADATA_TABLE.key, GEOMETRY_METADATA_TABLE);
}
@@ -115,7 +118,13 @@ protected JDBCDataStore createDataStoreInternal(JDBCDataStore dataStore, Map par
// check native paging
Boolean useNativePaging = (Boolean) NATIVE_PAGING.lookUp(params);
dialect.setUseOffSetLimit(useNativePaging == null || Boolean.TRUE.equals(useNativePaging));
-
+
+ // check serialization format
+ Boolean useNativeSerialization = (Boolean) NATIVE_SERIALIZATION.lookUp(params);
+ if (useNativeSerialization != null) {
+ dialect.setUseNativeSerialization(useNativeSerialization);
+ }
+
return dataStore;
}
View
70 modules/plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerDialect.java
@@ -16,20 +16,12 @@
*/
package org.geotools.data.sqlserver;
-import java.io.IOException;
-import java.sql.Connection;
-import java.sql.Date;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.sql.Time;
-import java.sql.Types;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.UUID;
-import java.util.logging.Level;
-
+import com.vividsolutions.jts.geom.*;
+import com.vividsolutions.jts.io.ParseException;
+import com.vividsolutions.jts.io.WKBReader;
+import com.vividsolutions.jts.io.WKTReader;
import org.geotools.data.jdbc.FilterToSQL;
+import org.geotools.data.sqlserver.reader.SqlServerBinaryReader;
import org.geotools.factory.Hints;
import org.geotools.geometry.jts.ReferencedEnvelope;
import org.geotools.jdbc.BasicSQLDialect;
@@ -42,19 +34,12 @@
import org.opengis.referencing.cs.CoordinateSystem;
import org.opengis.referencing.cs.CoordinateSystemAxis;
-import com.vividsolutions.jts.geom.Envelope;
-import com.vividsolutions.jts.geom.Geometry;
-import com.vividsolutions.jts.geom.GeometryCollection;
-import com.vividsolutions.jts.geom.GeometryFactory;
-import com.vividsolutions.jts.geom.LineString;
-import com.vividsolutions.jts.geom.MultiLineString;
-import com.vividsolutions.jts.geom.MultiPoint;
-import com.vividsolutions.jts.geom.MultiPolygon;
-import com.vividsolutions.jts.geom.Point;
-import com.vividsolutions.jts.geom.Polygon;
-import com.vividsolutions.jts.io.ParseException;
-import com.vividsolutions.jts.io.WKBReader;
-import com.vividsolutions.jts.io.WKTReader;
+import java.io.IOException;
+import java.sql.*;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+import java.util.logging.Level;
/**
* Dialect implementation for Microsoft SQL Server.
@@ -77,8 +62,9 @@
*/
private String geometryMetadataTable;
-
private Boolean useOffsetLimit = false;
+
+ private Boolean useNativeSerialization = false;
final static Map<String, Class> TYPE_TO_CLASS_MAP = new HashMap<String, Class>() {
{
@@ -414,7 +400,9 @@ public Integer getGeometrySRID(String schemaName, String tableName,
public void encodeGeometryColumn(GeometryDescriptor gatt, String prefix,
int srid, Hints hints, StringBuffer sql) {
encodeColumnName( prefix, gatt.getLocalName(), sql );
- sql.append( ".STAsBinary()");
+ if (!useNativeSerialization) {
+ sql.append( ".STAsBinary()");
+ }
}
@Override
@@ -437,11 +425,19 @@ public Geometry decodeGeometryValue(GeometryDescriptor descriptor,
if(bytes == null) {
return null;
}
- try {
- return new WKBReader(factory).read(bytes);
- } catch ( ParseException e ) {
- throw (IOException) new IOException().initCause( e );
- }
+ if (useNativeSerialization) {
+ try {
+ return new SqlServerBinaryReader(factory).read(bytes);
+ } catch ( IOException e ) {
+ throw (IOException) new IOException().initCause( e );
+ }
+ } else {
+ try {
+ return new WKBReader(factory).read(bytes);
+ } catch ( ParseException e ) {
+ throw (IOException) new IOException().initCause( e );
+ }
+ }
}
Geometry decodeGeometry( String s, GeometryFactory factory ) throws IOException {
@@ -636,5 +632,13 @@ public void setGeometryMetadataTable(String geometryMetadataTable) {
public void setUseOffSetLimit(Boolean useOffsetLimit) {
this.useOffsetLimit = useOffsetLimit;
}
+
+ /**
+ * Sets whether to use native SQL Server binary serialization or WKB serialization
+ * @param useNativeSerialization
+ */
+ public void setUseNativeSerialization(Boolean useNativeSerialization) {
+ this.useNativeSerialization = useNativeSerialization;
+ }
}
View
1  .../plugin/jdbc/jdbc-sqlserver/src/main/java/org/geotools/data/sqlserver/SQLServerJNDIDataStoreFactory.java
@@ -44,6 +44,7 @@ protected void setupParameters(Map parameters) {
super.setupParameters(parameters);
parameters.put(INTSEC.key, INTSEC);
parameters.put(NATIVE_PAGING.key, NATIVE_PAGING);
+ parameters.put(NATIVE_SERIALIZATION.key, NATIVE_SERIALIZATION);
parameters.put(GEOMETRY_METADATA_TABLE.key, GEOMETRY_METADATA_TABLE);
}
}
View
4 ...les/plugin/jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/SQLServerFeatureSourceTest.java
@@ -20,7 +20,7 @@
import org.geotools.jdbc.JDBCTestSetup;
/**
- *
+ *
*
* @source $URL$
*/
@@ -28,7 +28,7 @@
@Override
protected JDBCTestSetup createTestSetup() {
- return new SQLServerTestSetup();
+ return new SqlServerNativeSerializationTestSetup();
}
}
View
20 ...jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/SqlServerNativeSerializationTestSetup.java
@@ -0,0 +1,20 @@
+package org.geotools.data.sqlserver;
+
+import org.apache.commons.dbcp.BasicDataSource;
+import org.geotools.jdbc.JDBCDataStore;
+
+import java.util.Properties;
+
+/**
+ *
+ *
+ * @source $URL$
+ */
+public class SqlServerNativeSerializationTestSetup extends SQLServerTestSetup {
+
+ protected void setUpDataStore(JDBCDataStore dataStore) {
+ SQLServerDialect dialect = (SQLServerDialect) dataStore.getSQLDialect();
+ dialect.setUseNativeSerialization(Boolean.TRUE);
+ super.setUpDataStore(dataStore);
+ }
+}
View
120 ...ugin/jdbc/jdbc-sqlserver/src/test/java/org/geotools/data/sqlserver/reader/SQLServerBinaryReaderTest.java
@@ -0,0 +1,120 @@
+package org.geotools.data.sqlserver.reader;
+
+import com.vividsolutions.jts.geom.Geometry;
+import com.vividsolutions.jts.geom.GeometryFactory;
+import com.vividsolutions.jts.geom.PrecisionModel;
+import com.vividsolutions.jts.io.ParseException;
+import com.vividsolutions.jts.io.WKBReader;
+import com.vividsolutions.jts.io.WKTReader;
+import junit.framework.TestCase;
+
+import java.io.IOException;
+
+/**
+ * Binary test data have been produced by executing this query in SQL Server, and then removing the prefix '0x':
+ * select geometry::STGeomFromText('wktstring',srid);
+ *
+ * @source $URL$
+ */
+public class SQLServerBinaryReaderTest extends TestCase {
+
+ public void testPoint() throws Exception {
+ String geometryPointWKT = "POINT (5 10)";
+ String geometryPointBinary = "E6100000010C00000000000014400000000000002440";
+ testGeometry(geometryPointBinary, geometryPointWKT, 4326);
+ }
+
+ public void testEmptyPointGeometry() throws Exception {
+ String geometryEmptyWKT = "POINT EMPTY";
+ String geometryEmptyBinary = "000000000104000000000000000001000000FFFFFFFFFFFFFFFF01";
+ testGeometry(geometryEmptyBinary, geometryEmptyWKT);
+ }
+
+ public void testEmptyPolygonGeometry() throws Exception {
+ String geometryEmptyWKT = "POLYGON EMPTY";
+ String geometryEmptyBinary = "000000000104000000000000000001000000FFFFFFFFFFFFFFFF03";
+ testGeometry(geometryEmptyBinary, geometryEmptyWKT);
+ }
+
+ public void testEmptyGeometryCollection() throws Exception {
+ String geometryEmptyWKT = "GEOMETRYCOLLECTION EMPTY";
+ String geometryEmptyBinary = "000000000104000000000000000001000000FFFFFFFFFFFFFFFF07";
+ testGeometry(geometryEmptyBinary, geometryEmptyWKT);
+ }
+
+ public void testPolygon() throws Exception {
+ String geometryPolygonWkt = "POLYGON ((-680000 6100000, -670000 6100000, -670000 6090000, -680000 6090000, -680000 6100000))"; //32633
+ String geometryPolygonBinary = "797F00000104050000000000000080C024C1000000000845574100000000607224C1000000000845574100000000607224C100000000443B57410000000080C024C100000000443B57410000000080C024C1000000000845574101000000020000000001000000FFFFFFFF0000000003";
+ testGeometry(geometryPolygonBinary, geometryPolygonWkt, 32633);
+ }
+
+ public void testLineStringWithZ() throws Exception {
+ String geometryLineStringWithZWKT = "LINESTRING (0 1 1, 3 2 2, 4 5 NaN)";
+ String geometryLineStringWithZBinary = "E61000000105030000000000000000000000000000000000F03F0000000000000840000000000000004000000000000010400000000000001440000000000000F03F0000000000000040000000000000F8FF01000000010000000001000000FFFFFFFF0000000002";
+ Geometry geometry = testGeometry(geometryLineStringWithZBinary, geometryLineStringWithZWKT, 4326);
+ assertEquals(1.0, geometry.getCoordinates()[0].z, 0);
+ assertEquals(2.0, geometry.getCoordinates()[1].z, 0);
+ assertEquals(Double.NaN, geometry.getCoordinates()[2].z, 0);
+ }
+
+ public void testGeometryCollection() throws Exception {
+ String geometryCollectionWKT = "GEOMETRYCOLLECTION (POINT (4 0), LINESTRING (4 2, 5 3), POLYGON ((0 0, 3 0, 3 3, 0 3, 0 0),(1 1, 1 2, 2 2, 2 1, 1 1)))";
+ String geometryCollectionBinary = "0000000001040D0000000000000000001040000000000000000000000000000010400000000000000040000000000000144000000000000008400000000000000000000000000000000000000000000008400000000000000000000000000000084000000000000008400000000000000000000000000000084000000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F0000000000000040000000000000004000000000000000400000000000000040000000000000F03F000000000000F03F000000000000F03F04000000010000000001010000000203000000000800000004000000FFFFFFFF0000000007000000000000000001000000000100000002000000000200000003";
+ testGeometry(geometryCollectionBinary, geometryCollectionWKT,0);
+
+ String wkt2 = "GEOMETRYCOLLECTION (POINT (4.2585 0), MULTILINESTRING ((10 10, 20 20, 10 40.999), (40 40, 30 30, 40 20, 30 10)), POLYGON ((0 0, 3 0, 3 3, 0 3, 0 0),(1 1, 1 2, 2 2, 2 1, 1 1)))";
+ String binary2 = "0000000001041200000062105839B40811400000000000000000000000000000244000000000000024400000000000003440000000000000344000000000000024401D5A643BDF7F4440000000000000444000000000000044400000000000003E400000000000003E40000000000000444000000000000034400000000000003E4000000000000024400000000000000000000000000000000000000000000008400000000000000000000000000000084000000000000008400000000000000000000000000000084000000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F0000000000000040000000000000004000000000000000400000000000000040000000000000F03F000000000000F03F000000000000F03F050000000100000000010100000001040000000208000000000D00000006000000FFFFFFFF0000000007000000000000000001000000000100000005020000000100000002020000000200000002000000000300000003";
+ testGeometry(binary2, wkt2, 0);
+ }
+
+ public void testMultiLinestring() throws Exception {
+ String wkt = "MULTILINESTRING ((10 10, 20 20, 10 40), (40 40, 30 30, 40 20, 30 10)) "; //4326
+ String binary = "00000000010407000000000000000000244000000000000024400000000000003440000000000000344000000000000024400000000000004440000000000000444000000000000044400000000000003E400000000000003E40000000000000444000000000000034400000000000003E400000000000002440020000000100000000010300000003000000FFFFFFFF0000000005000000000000000002000000000100000002";
+ testGeometry(binary, wkt, 0);
+ }
+
+ public void testMultiPoint() throws Exception {
+ String wkt = "MULTIPOINT ((10 40), (40 30), (20 20), (30 10))";
+ String binary = "000000000104040000000000000000002440000000000000444000000000000044400000000000003E40000000000000344000000000000034400000000000003E40000000000000244004000000010000000001010000000102000000010300000005000000FFFFFFFF0000000004000000000000000001000000000100000001000000000200000001000000000300000001";
+ testGeometry(binary, wkt, 0);
+ }
+
+ public void testMultiPolygon() throws Exception {
+ String wkt = "MULTIPOLYGON (((30 20, 10 40, 45 40, 30 20)),((15 5, 40 10, 10 20, 5 10, 15 5)))";
+ String binary = "000000000104090000000000000000003E40000000000000344000000000000024400000000000004440000000000080464000000000000044400000000000003E4000000000000034400000000000002E4000000000000014400000000000004440000000000000244000000000000024400000000000003440000000000000144000000000000024400000000000002E400000000000001440020000000200000000020400000003000000FFFFFFFF0000000006000000000000000003000000000100000003";
+ testGeometry(binary, wkt, 0);
+ }
+
+ public void testSingleLineSegment() throws Exception {
+ String geometryPointWKT = "LINESTRING (5 10, 10 10)";
+ String geometryPointBinary = "0000000001140000000000001440000000000000244000000000000024400000000000002440";
+ testGeometry(geometryPointBinary, geometryPointWKT, 0);
+ }
+
+ public void testGeographyExtremeValues() throws Exception {
+ String wkt = "LINESTRING (-90 -15069, 90 15069)";
+ String binary = "E6100000011400000000008056C000000000806ECDC0000000000080564000000000806ECD40";
+ testGeometry(binary,wkt,4326);
+ }
+
+ private Geometry testGeometry(String geometryBinary, String geometryWKT) throws Exception {
+ WKTReader readerWkt = new WKTReader((new GeometryFactory(new PrecisionModel(), 0)));
+ return testGeometry(geometryBinary, geometryWKT, readerWkt);
+ }
+
+ private Geometry testGeometry(String geometryBinary, String geometryWKT, WKTReader wktReader) throws ParseException, IOException {
+ byte[] bytes = WKBReader.hexToBytes(geometryBinary);
+ Geometry geometryFromWkt = wktReader.read(geometryWKT);
+ SqlServerBinaryReader reader = new SqlServerBinaryReader();
+ Geometry geometryFromBinary = reader.read(bytes);
+ assertEquals(geometryFromWkt, geometryFromBinary);
+ return geometryFromBinary;
+ }
+
+ private Geometry testGeometry(String geometryBinary, String geometryWKT, int srid) throws Exception {
+ WKTReader readerWkt = new WKTReader((new GeometryFactory(new PrecisionModel(), srid)));
+ Geometry geometry = testGeometry(geometryBinary, geometryWKT, readerWkt);
+ assertEquals(srid, geometry.getSRID());
+ return geometry;
+ }
+}
Something went wrong with that request. Please try again.