[Server][FEATURE][needs-docs] Support WFS 1.1.0 #5297

rldhont · 2017-10-04T14:16:09Z

Description

QGIS Server already supports WMS 1.3.0 and 1.1.1, WFS 1.0.0 and WCS 1.0.0
WFS 1.1.0 adds parameters like: SrsName, to transform GML geometry and SortBy to order features.

Funded by Ifremer

Checklist

Commit messages are descriptive and explain the rationale for changes
Commits which add new features are tagged with [FEATURE] in the commit message
Commits which change the UI or existing user workflows are tagged with [needs-docs] in the commit message and containt sufficient information in the commit message to be documented
I have read the QGIS Coding Standards and this PR complies with them
This PR passes all existing unit tests (test results will be reported by travis-ci after opening this PR)
New unit tests have been added for core changes
I have run the scripts/prepare-commit.sh script before each commit

pblottiere · 2017-10-11T08:03:04Z

src/server/services/wfs/qgswfs.cpp

+          // Supports WFS 1.0.0
+          if ( QSTR_COMPARE( versionString, "1.0.0" ) )
+          {
+            v1_0_0::writeGetCapabilities( mServerIface, project, versionString, request, response );


I'm not a big fan of this notation : v1_0_0::.

Moreover, I've the feeling that most of the code is duplicated between qgswfsgetcapabilities.cpp and qgswfsgetcapabilities_1_0_0.cpp. And it's not homogeneous with other services (for example, we don't have a qgswmsrenderer.cpp and a qgswmsrenderer_1_3_0.cpp).

No? Or am I missing something?

The WFS capabilities documents are not the same at all and it will be easiest to enhance or maintain with separate files. For the other request like DescribeFeature or GetFeature, we reused the same code.

It's true that it's not as similar as I thought... However, I think we could factor a little more.

For example, writeGetCapabilities is identical and there's some redundant content in createGetCapabilitiesDocument (WFS_Capabilities, ogc:SpatialOperators) or getFeatureTypeListElement.

That being said, your comment about maintainability is justified...

@nyalldawson @m-kuhn What do you think about adding files for specific service versions (like those introduced in this PR : qgswfsgetcapabilities.cpp and qgswfsgetcapabilities_1_0_0.cpp) ?

FWIW, owslib made the same thing as you (separate versions in files) : https://github.com/geopython/OWSLib/tree/master/owslib/feature

So why not, but in this case, maybe we should have a qgswfsgetcapabilities_common.x, qgswfsgetcapabilities_100.x and qgswfsgetcapabilities_110.x files to be more explicit?

Otherwise (I'm just thinking out loud, so it may probably be a bad idea...), maybe a composite/visitor pattern could be used:

1 node of the tree for 1 node of the document

one or several validity versions for each node (the same node may be valid for 1.0.0 or 1.1.0)

a visitor collects only nodes for a specific version

when we want to add a new version, we just have to add some node and retrieve existing nodes

Hi @pblottiere

There is several ways to add new versions:

Handle several versions in one module (all in one, only one default version registred but assumed to handle everything, or register multiple versions in the same module.)

Add new versions with different modules python or c++ (register them with there proper versions)

In the present the case we use a different namespace juste to separates units specific to a given version: I do not care about the name of the namespace as long it is meaningful in the context - IMO, the advantage of using namespace over is that you don't have to rename all your fonctions/objects.

IMHO, we should consider the current implementation as a easy way to implement quickly the 1_1_0 without taking the chance to break actual implementation with more or less heavy refactoring: we will have time to factorize things when all the stuff will be stabilized a little more.

Hi @dmarteau
I don't have a strong opinion against the use of a namespaces here as long as it does not introduce redundancy in the code.
And I would also explicit versions: it is a bit strange to have qgswfsgetcapabilities_1_0_0 and qgswfsgetcapabilities without suffix (which version ?), no ?

@mhugo it's because 1.1.0 will be the default version instead of 1.0.0 which become a special version.

@mhugo, @rldhont there is no points to make a debat about it, I really do not care about it as long as naming is not ambiguous.

pblottiere · 2017-10-11T08:05:48Z

src/server/qgsrequesthandler.cpp

+          continue;
+
+        QDomNode attrNode = map.item( i );
+        QDomAttr attr = attrNode.toAttr();


I think attrNode is not used directly. Maybe const QDomAttr attr = map.item( i ).toAttr(); is enough?

pblottiere · 2017-10-11T08:06:01Z

src/server/qgsrequesthandler.cpp

+        if ( attr.isNull() )
+          continue;
+
+        QString attrName = attr.name();


pblottiere · 2017-10-11T08:08:01Z

src/server/services/wfs/qgswfs.cpp

@@ -71,7 +72,15 @@ namespace QgsWfs

        if ( QSTR_COMPARE( req, "GetCapabilities" ) )
        {
-          writeGetCapabilities( mServerIface, project, versionString, request, response );
+          // Supports WFS 1.0.0
+          if ( QSTR_COMPARE( versionString, "1.0.0" ) )


For now there's only 2 versions to manage. But something like QgsProjectVersion (for minor versions and so on) could be used instead of comparing string. What do you think?

Not need to use a QgsProjectVersion, here the versionSTring has to be "1.0.0", if it's not the default version has to be returned.

pblottiere · 2017-10-11T08:12:09Z

src/server/services/wfs/qgswfsgetcapabilities.cpp

-      QDomText keywordsText = doc.createTextNode( keywords.join( QStringLiteral( ", " ) ) );
-      keywordsElem.appendChild( keywordsText );
+      QDomElement keywordsElem = doc.createElement( QStringLiteral( "ows:Keywords" ) );
+      for ( QString keyword : keywords )


pblottiere · 2017-10-11T08:12:46Z

src/server/services/wfs/qgswfsgetcapabilities.cpp

-    requestElement.appendChild( getCapabilitiesElement );
+      QString contactMail = QgsServerProjectUtils::owsServiceContactMail( *project );
+      QString contactPhone = QgsServerProjectUtils::owsServiceContactPhone( *project );
+      QString onlineResource = QgsServerProjectUtils::owsServiceOnlineResource( *project );


pblottiere · 2017-10-11T08:14:06Z

src/server/services/wfs/qgswfsgetcapabilities.cpp

-      QDomElement srsElem = doc.createElement( QStringLiteral( "SRS" ) );
-      QDomText srsText = doc.createTextNode( layer->crs().authid() );
+      //create DefaultSRS element
+      QString defaultSrs = layer->crs().authid();


pblottiere · 2017-10-11T08:19:23Z

src/server/services/wfs/qgswfsgetcapabilities_1_0_0.cpp

+      QStringList wfstUpdateLayersId = QgsServerProjectUtils::wfstUpdateLayerIds( *project );
+      QStringList wfstInsertLayersId = QgsServerProjectUtils::wfstInsertLayerIds( *project );
+      QStringList wfstDeleteLayersId = QgsServerProjectUtils::wfstDeleteLayerIds( *project );
+      for ( int i = 0; i < wfsLayerIds.size(); ++i )


for ( const QString &wfsLayerId : wfsLayerIds ) {}

pblottiere · 2017-10-11T08:19:35Z

src/server/services/wfs/qgswfsgetcapabilities_1_0_0.cpp

+      QStringList wfsLayerIds = QgsServerProjectUtils::wfsLayerIds( *project );
+      QStringList wfstUpdateLayersId = QgsServerProjectUtils::wfstUpdateLayerIds( *project );
+      QStringList wfstInsertLayersId = QgsServerProjectUtils::wfstInsertLayerIds( *project );
+      QStringList wfstDeleteLayersId = QgsServerProjectUtils::wfstDeleteLayerIds( *project );


pblottiere · 2017-10-11T08:29:06Z

src/server/services/wfs/qgswfsgetfeature.cpp

+            geom = transformed;
+            crs = params.outputCrs;
+            if ( crs.isGeographic() && !params.crs.isGeographic() )
+              prec = std::min( params.precision + 3, 6 );


Is it standard to manage precision this way?

In the ISO GeoJSON standard, the max precision has been fixed to 6 for EPSG:4326, so I think I can homogenize the precision to 6 for all the Geographic requested outputCrs.

Mm, after checking the standard, 6 digit is only a recommandation to avoid heavy files with 15 digits.
As there is a per layer precision configuration windows, I would reuse the precision used for GML output. If really you need a different precision between GML an GeoJSON, we could add another column there, but I don't think it's worth it.
Also we could discuss if setting by default 6 digits there woudn't be a better behavior than the current digit default

Sorry, last sentence was "the current 8 digit default"

@haubourg the problem is not between GML or GeoJSON, is between meters and degrees. In WFS 1.1.0, the GML can be requested in an other CRS than the source. So if the features ares requested in a Geographic CRS, I can't used directly the precision, for exemple 1 for a CRS in meters. The test here is only for this type of transform.
The other point is that I have updated the current default precision to 6 digits to avoid heavy files.

I can add the inverse, from Geographic to Planar, I can limit to 3 digits.

pblottiere · 2017-10-11T08:38:25Z

src/server/services/wfs/qgswfsparameters.h

+#include "qgsserverrequest.h"
+#include "qgsprojectversion.h"
+
+/** QgsWmsParameters provides an interface to retrieve and manipulate WFS


which one ?

QgsWmsParameters -> QgsWfsParameters

pblottiere · 2017-10-11T09:23:58Z

@rldhont Thank you for this PR, it's good to see WFS 1.1.0 landing in server!

Just some comments:

Do you think that QgsWmsParameters and QgsWfsParameters classes could inherit to avoid some duplication (like load or toXXX, and so on)?
Just by curiosity, did you try to run OGC tests on WFS services?
Check const on variables

rldhont · 2017-10-11T09:48:49Z

I think that QgsWmsParameters and QgsWfsParameters classes could inherit to avoid some duplication (like load or toXXX, and so on), but I'm not sure how to do it with namespace.
I did not try to run OGC tests on WFS services, because I firstly like to finish it and test it with QGIS.
I think it will be great to check const in all the QGIS Server Capabilities writer.

rldhont · 2017-10-11T14:13:27Z

@pblottiere have you some comment on tests/src/python/test_qgsserver.py changes ?

… parameters

rldhont · 2017-10-14T13:59:21Z

@pblottiere I'd like to merge it, we can enhance the code after the feature freeze.

rldhont added Feature Server labels Oct 4, 2017

rldhont added this to the QGIS 3 milestone Oct 4, 2017

rldhont requested review from elpaso, sbrunner and pblottiere October 4, 2017 14:16

rldhont force-pushed the wfs_1_1_0 branch 2 times, most recently from c113d48 to fd1a4c4 Compare October 5, 2017 14:31

pblottiere reviewed Oct 11, 2017

View reviewed changes

src/server/qgsrequesthandler.cpp Outdated

if ( attr.isNull() )

continue;

QString attrName = attr.name();

Copy link

Member

pblottiere Oct 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const

pblottiere reviewed Oct 11, 2017

View reviewed changes

rldhont force-pushed the wfs_1_1_0 branch 2 times, most recently from f5e42b6 to 1d38203 Compare October 11, 2017 14:12

rldhont added 6 commits October 12, 2017 13:45

[Server][WFS,1.1.0] Add layout for WFS supports

1b9d8ee

[Server][WFS] Update GetCapabilities 1.1.0

c0117df

[Server][WFS] add QgsWfsParameters

c8a4407

[Server][WFS] Update DescribeFeatureType 1.1.0

d7a48c6

[Server][Tests] Add PyQgsServerWFS test

b8ebbc1

[Server] Request handler reads the document root attributes to expose…

5d2c035

… parameters

rldhont added 8 commits October 12, 2017 13:49

[Server][WFS] Update QgsWfsParameters for GetFeature

44754ed

[Server][WFS] Update GetFeature 1.1.0

c7a396b

[Server][Tests] Enhance assertXMLEqual

96ec4bb

[Server][Tests] Update PyQgsServerWFS test for 1.1.0

1e009b4

[Server][WFS][Feature] Support resultType=hits from GetFeature 1.1.0

0d350b8

[Server][WFS] Update GML GetFeature document

24a6854

[Server][WFS] Update Transaction 1.1.0

1558b03

[Server][WFS] check const and indetation

2f29e16

rldhont force-pushed the wfs_1_1_0 branch from 1d38203 to 2f29e16 Compare October 12, 2017 11:50

rldhont mentioned this pull request Oct 13, 2017

[server][bugfix] Fix OGC test getfeatureinfo:each-queryable-layer #5303

Merged

8 tasks

rldhont merged commit ab107d0 into qgis:master Oct 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Server][FEATURE][needs-docs] Support WFS 1.1.0 #5297

[Server][FEATURE][needs-docs] Support WFS 1.1.0 #5297

rldhont commented Oct 4, 2017 •

edited

Loading

pblottiere Oct 11, 2017

rldhont Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 13, 2017

dmarteau Oct 13, 2017 •

edited

Loading

mhugo Oct 13, 2017

rldhont Oct 13, 2017

dmarteau Oct 13, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

rldhont Oct 13, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere Oct 11, 2017

rldhont Oct 11, 2017

haubourg Oct 13, 2017

haubourg Oct 13, 2017

rldhont Oct 13, 2017

rldhont Oct 13, 2017

pblottiere Oct 11, 2017

rldhont Oct 11, 2017

pblottiere Oct 11, 2017

pblottiere commented Oct 11, 2017

rldhont commented Oct 11, 2017

rldhont commented Oct 11, 2017

rldhont commented Oct 14, 2017

[Server][FEATURE][needs-docs] Support WFS 1.1.0 #5297

[Server][FEATURE][needs-docs] Support WFS 1.1.0 #5297

Conversation

rldhont commented Oct 4, 2017 • edited Loading

Description

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmarteau Oct 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pblottiere commented Oct 11, 2017

rldhont commented Oct 11, 2017

rldhont commented Oct 11, 2017

rldhont commented Oct 14, 2017

rldhont commented Oct 4, 2017 •

edited

Loading

dmarteau Oct 13, 2017 •

edited

Loading