[GEOS-8069] Add documentation and tests about using MongoDB as a data store for app-schema #2205

nmco · 2017-04-05T21:19:32Z

This pull request adds documentation bout how to use MongoDB with App-Schema and adds online tests (integration test).

Depends on pull request: geotools/geotools#1546

Associated issue: https://osgeo-org.atlassian.net/browse/GEOS-8069

nmco · 2017-04-06T09:39:04Z

...ma/app-schema-mongo-test/src/test/java/org/geoserver/test/onlineTest/ComplexMongoDBTest.java

@@ -0,0 +1,273 @@
+package org.geoserver.test.onlineTest;


Missing header.

bencaradocdavies

@nmco I do not know much about MongoDB but this looks like a tidy and thorough piece of work.

bencaradocdavies · 2017-04-06T23:39:24Z

doc/en/user/source/data/app-schema/mongo-tutorial.rst

@@ -0,0 +1,436 @@
+.. _app-schema.tutorial:


Duplicate label (same as the app-schema tutorial). Should be:

.. _app-schema.mongo-tutorial:

bencaradocdavies · 2017-04-06T23:41:18Z

doc/en/user/source/data/app-schema/data-stores.rst

+        </DataStore>
+    </sourceDataStores>
+
+Check the MongoDB tutorial for a more detailed description about how to use MongoDB with app-schema.


Please link to the mongo tutorial.

Correct thanks, fixed.

bencaradocdavies · 2017-04-06T23:41:45Z

doc/en/user/source/data/app-schema/data-stores.rst

+
+Check the MongoDB tutorial for a more detailed description about how to use MongoDB with app-schema.
+
+.. note:: You must install the Oracle plugin to connect to Oracle Spatial databases.


I think your mean MongoDB here. Cut and paste error.

Yup fixed thanks.

bencaradocdavies · 2017-04-06T23:42:54Z

doc/en/user/source/data/app-schema/mongo-tutorial.rst

+
+    db.stations.insert({
+        "id": "1",
+        "name": "station 1"


Missing comma at end of line here. Hooray for GitHub syntax highlighting.

bencaradocdavies · 2017-04-06T23:43:03Z

doc/en/user/source/data/app-schema/mongo-tutorial.rst

+
+    db.stations.insert({
+        "id": "2",
+        "name": "station 2"


Missing comma at end of line here. Hooray for GitHub syntax highlighting.

bencaradocdavies · 2017-04-06T23:44:35Z

doc/en/user/source/data/app-schema/mongo-tutorial.rst

+    })
+
+    db.stations.createIndex({
+        geometry: "2dsphere"


GitHub JSON syntax highlighting suggests that something is wrong with geometry. Missing quotes perhaps?

bencaradocdavies · 2017-04-06T23:56:27Z

src/extension/app-schema/app-schema-mongo-test/src/test/resources/data/stations1.json

@@ -0,0 +1,36 @@
+{
+  "id": "1",
+  "name": "station 1",


Aha! Unlike the tutorial, this line has a comma at the end.

Yeah no sure what happen, I copy paste test cases to doc 😥.

bencaradocdavies · 2017-04-07T00:06:15Z

...ma/app-schema-mongo-test/src/test/java/org/geoserver/test/onlineTest/ComplexMongoDBTest.java

+    @BeforeClass
+    public static void setup() throws Exception {
+        // check that we have access to a mongodb and instantiate the client
+        String hostAsString = getProperty("mongo.host", "127.0.0.1");


This test fixture should use settings stored in a properties file ~/.geoserver/mongodb.properties or similar. GeoTools fixture configuration support is more reusable, but GeoServer is messier. ReferenceDataPostgisSetup uses JDBCTestSetup but I expect that this will be less useful for Mongo. GeoTools MongoTestSupport does the right thing.

Agree, fixed.

bencaradocdavies · 2017-04-07T00:08:36Z

doc/en/developer/source/programming-guide/app-schema/index.rst

+Running MongoDB Online Tests 
+----------------------------
+
+MongoDB online tests expected by default that a MongoDB instance can be reached using host `127.0.0.1` and port `27017`. A different host can be provided using environment variable `mongo.host` and a different port can be provided using variable `mongo.port`. If no MongoDB instance can be reached the online tests will be skipped.


As noted elsewhere, it would be preferable that these setting were stored in a mongodb.properties file.

bencaradocdavies · 2017-04-07T01:00:54Z

With no MongoDB installed, this PR causes app-schema online tests (-Papp-schema-online-test) to fail with:

Tests in error: 
  org.geoserver.test.onlineTest.ComplexMongoDBTest: got: <false>, expected: is <true>
  org.geoserver.test.onlineTest.ComplexMongoDBTest: Timed out after 2000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]

Looking a little deeper, one fails in setup:

org.geoserver.test.onlineTest.ComplexMongoDBTest  Time elapsed: 4518 sec  <<< ERROR!
org.junit.internal.AssumptionViolatedException: got: <false>, expected: is <true>
	at org.junit.Assume.assumeThat(Assume.java:95)
	at org.geoserver.test.onlineTest.ComplexMongoDBTest.setup(ComplexMongoDBTest.java:84)
[...]

and the other fails in tearDown:

org.geoserver.test.onlineTest.ComplexMongoDBTest  Time elapsed: 4520 sec  <<< ERROR!
com.mongodb.MongoTimeoutException: Timed out after 2000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
	at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:375)
	at com.mongodb.connection.BaseCluster.selectServer(BaseCluster.java:104)
	at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:75)
	at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:71)
	at com.mongodb.binding.ClusterBinding.getWriteConnectionSource(ClusterBinding.java:68)
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:159)
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:134)
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:124)
	at com.mongodb.operation.CommandWriteOperation.execute(CommandWriteOperation.java:55)
	at com.mongodb.Mongo.execute(Mongo.java:827)
	at com.mongodb.Mongo$2.execute(Mongo.java:810)
	at com.mongodb.DB.executeCommand(DB.java:648)
	at com.mongodb.DB.dropDatabase(DB.java:216)
	at org.geoserver.test.onlineTest.ComplexMongoDBTest.tearDown(ComplexMongoDBTest.java:111)

nmco · 2017-04-08T23:57:23Z

Thanks for the review @bencaradocdavies ! I amend has requested and online should work has expected now.

bencaradocdavies · 2017-04-09T04:43:04Z

...ma/app-schema-mongo-test/src/test/java/org/geoserver/test/onlineTest/ComplexMongoDBTest.java

+ * tests hence they require a MongoDB instance. Host 127.0.0.1 and port 27017 will
+ * be used by default but is possible to provide another host and port using Java
+ * variables mongo.host and mongo.port respectively.
+ */


Should mention mongodb.properties not Java properties.

Good catch, I forget about this Java doc. Fixed.

bencaradocdavies · 2017-04-09T04:48:22Z

@nmco that looks great. Tidy work. Full builds and app-schema online tests allpass (without a mongodb instance). I plan to merge this and geotools/geotools#1546 as soon as the geoserver-master build on ares Jenkins has recovered (I think its last failure was some intermittent resource problem). There is the one little note about the test javadoc that could be fixed to mention mongodb.properties but this is not enough to hold up the merge. Feel free to fix now if you can beat me to the merge or fix later at your convenience.

… app-schema

nmco · 2017-04-09T09:39:00Z

Java doc amended and a mean-full message will be displayed pointing to the mongodb.properties when tests are skipped.

nmco mentioned this pull request Apr 5, 2017

[GEOT-5690] Make MongoDB usable as a data store in app-schema geotools/geotools#1546

Merged

nmco commented Apr 6, 2017

View reviewed changes

bencaradocdavies suggested changes Apr 7, 2017

View reviewed changes

nmco force-pushed the mongo-doc-and-tests branch from f9f36e5 to addb080 Compare April 8, 2017 23:56

bencaradocdavies reviewed Apr 9, 2017

View reviewed changes

bencaradocdavies approved these changes Apr 9, 2017

View reviewed changes

[GEOS-8069] Add documentation about using MongoDB as a data store for…

d24d2af

… app-schema

nmco force-pushed the mongo-doc-and-tests branch from addb080 to d24d2af Compare April 9, 2017 09:36

bencaradocdavies merged commit 1ac6737 into geoserver:master Apr 9, 2017

nmco mentioned this pull request Nov 14, 2017

Backport of MongoDB integration with App-Schema to 2.11.x #2640

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GEOS-8069] Add documentation and tests about using MongoDB as a data store for app-schema #2205

[GEOS-8069] Add documentation and tests about using MongoDB as a data store for app-schema #2205

nmco commented Apr 5, 2017 •

edited

Loading

nmco Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies left a comment

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 6, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 7, 2017

nmco Apr 8, 2017

bencaradocdavies Apr 7, 2017

nmco Apr 8, 2017

bencaradocdavies commented Apr 7, 2017 •

edited

Loading

nmco commented Apr 8, 2017 •

edited

Loading

bencaradocdavies Apr 9, 2017

nmco Apr 9, 2017

bencaradocdavies commented Apr 9, 2017

nmco commented Apr 9, 2017


		Check the MongoDB tutorial for a more detailed description about how to use MongoDB with app-schema.

		.. note:: You must install the Oracle plugin to connect to Oracle Spatial databases.

[GEOS-8069] Add documentation and tests about using MongoDB as a data store for app-schema #2205

[GEOS-8069] Add documentation and tests about using MongoDB as a data store for app-schema #2205

Conversation

nmco commented Apr 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bencaradocdavies left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bencaradocdavies commented Apr 7, 2017 • edited Loading

nmco commented Apr 8, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bencaradocdavies commented Apr 9, 2017

nmco commented Apr 9, 2017

nmco commented Apr 5, 2017 •

edited

Loading

bencaradocdavies commented Apr 7, 2017 •

edited

Loading

nmco commented Apr 8, 2017 •

edited

Loading