WFS: content detected as NAS without geometries, but dataset contains them

[tobwen]

Expected behavior and actual behavior.

This issue stays in connection with #719, since the same WFS is affected.

When trying to raw data downloaded from a WFS (see below), the content gets interpreted as NAS without geometries. But the data contains geometries, which can be parsed as normal GML.

[...]
Warning 1: NAS: Skipping geometry without feature
Warning 1: NAS: Skipping geometry without feature
Warning 1: NAS: Skipping geometry without feature
Warning 1: NAS: Skipping geometry without feature
Warning 1: NAS: Skipping geometry without feature
More than 1000 errors or warnings have been reported. No more will be reported from now.
INFO: Open of wfsdemo.gml' using driver NAS' successful.
1: ALKIS_beziehungen (None)

Steps to reproduce the problem.

wget -o wfsdemo.gml https://www.wfs.nrw.de/geobasis/wfs_nw_alkis_vereinfacht?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAMES=ave:Nutzung&STARTINDEX=0&COUNT=1000

ogrinfo -ro -al wfsdemo.gml

Operating system

Microsoft Windows 7, Version 6.1.7601 (64-bit)

GDAL version and provenance

GDAL 2.4.0dev, released 2018/99/99 from OSGeo4W (64-bit)

[rouault]

@jef-n This file is currently recognized by the NAS driver because the header contains the "aaa-suite" string. Do you think this is appropriate ? Or should it not identify itself for this file to let the GML driver deal with it ? In which case how to relax OGRNASDriverIdentify() ?

<?xml version="1.0" encoding="utf-8"?>
<wfs:FeatureCollection xmlns="http://repository.gdi-de.org/schemas/adv/produkt/alkis-vereinfacht/1.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:wfs="http://www.opengis.net/wfs/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" timeStamp="2018-06-27T22:14:45.073+02:00" numberReturned="1000" numberMatched="unknown" xsi:schemaLocation="http://repository.gdi-de.org/schemas/adv/produkt/alkis-vereinfacht/1.0 http://www.wfs.nrw.de/aaa-suite/schema/AAA-vereinfacht/1.0/schema/ALKIS-vereinfacht.xsd http://www.opengis.net/wfs/2.0 http://www.wfs.nrw.de/aaa-suite/schema/ogc/wfs/2.0/wfs.xsd http://www.opengis.net/gml/3.2 http://www.wfs.nrw.de/aaa-suite/schema/ogc/gml/3.2.1/gml.xsd http://www.isotc211.org/2005/gco http://www.wfs.nrw.de/aaa-suite/schema/ogc/iso/19139/20070417/gco/gco.xsd http://www.isotc211.org/2005/gmd http://www.wfs.nrw.de/aaa-suite/schema/ogc/iso/19139/20070417/gmd/gmd.xsd">
  <wfs:boundedBy>
    <gml:Envelope srsName="urn:ogc:def:crs:EPSG::25832" srsDimension="2">
      <gml:lowerCorner>478023.127 5703007.006</gml:lowerCorner>
      <gml:upperCorner>489982.701 5707785.497</gml:upperCorner>
    </gml:Envelope>
  </wfs:boundedBy>
  <wfs:member>
    <Nutzung gml:id="DENW19AL0000ggHXTN">
      <gml:identifier codeSpace="urn:adv:oid:">urn:adv:oid:DENW19AL0000ggHXTN</gml:identifier>
      <oid>DENW19AL0000ggHXTN</oid>
      <aktualit>2016-02-29</aktualit>
      <geometrie>
        <gml:MultiSurface gml:id="tn.id_tn.1025.position.Geom_0" srsName="urn:ogc:def:crs:EPSG::25832" srsDimension="2">
          <gml:surfaceMember>
            <gml:Polygon gml:id="tn.id_tn.1025.position.Geom_1">
              <gml:exterior>
                <gml:LinearRing>
                  <gml:posList>478614.892 5705476.391 478634.79 5705460.534 478656.089 5705486.656 478639.462 5705502.756 478631.812 5705494.547 478614.892 5705476.391</gml:posList>
                </gml:LinearRing>
              </gml:exterior>
            </gml:Polygon>
          </gml:surfaceMember>
        </gml:MultiSurface>
      </geometrie>
      <nutzart>Wohnbauflaeche</nutzart>
    </Nutzung>
  </wfs:member>
[...]

[tobwen]

FYI: This is the simplified WFS for clients, which are not capable of loading NAS over WFS (there's also a version with complex objects). QGIS can load the simple WFS. Perhaps they also had a discussion about this topic?

[rouault]

Perhaps they also had a discussion about this topic?

No, QGIS uses its own WFS implementation that has only a simple features GML parser. The OGR WFS driver passes the GetFeature response to its other drivers (GML, GeoJSON, etc... and possibly NAS), and let them decode it. Here the NAS driver believes that this is for it, but this isn't probably the best decision.

[tobwen]

Here the NAS driver believes that this is for it, but this isn't probably the best decision.

How about a switch to let the user force the format -oo GML=TRUE?

[rouault]

You can already use --config OGR_SKIP NAS .

[tobwen]

Great, thanks.

[jef-n]

@rouault oops, missed this one. I guess the problem is that we don't have a --oo NAS=TRUE - ie. a way to set which driver to use. I don't know of any safe way to recognize a NAS file. We could drop the default for NAS_INDICATOR. Where I use it NAS_INDICATOR is explicitly set to the current default anyway as well as OGR_SKIP=GML,SEGY.

[tobwen]

Can anyone give a quick roundup, what has changed now? --config OGR_SKIP NAS did it for me.

[rouault]

9ec6971 breaks the tests. See 9ec6971#commitcomment-29848859

[jef-n]

@rouault does https://gist.github.com/jef-n/6e121d62770380047f75a827805b421c look ok to you?

[rouault]

@rouault does https://gist.github.com/jef-n/6e121d62770380047f75a827805b421c look ok to you?

That stil means that users have to set NAS_INDICATOR to esoteric value that they are unlikely to guess. I think the situation before the changes attempted in that ticket was not so bad after all, wasn't it ?

[jef-n]

Depends on the point of view.

To me the NAS driver is meant to import the data into a pre-created schema that also has the triggers to handle deletions in place and a corresponding schema.gfs (like implemented in alkisimport or described on the PostNAS site).

Using it in other circumstances is an edge case to me as it might lead to unexpected results. The driver will deduce attributes and their types from the first appearing element - and silently ignore attributes from following attributes or data that doesn't match the type the first element suggested the attribute would have.

The data is usually spread over several files so scanning the one you're currently importing wouldn't help either. And the datasets are usually large, so reading them directly from XML doesn't make much sense to me either.

So the commit fixes this by only enabling the NAS driver in known territory and the changes to the test put it into the environment that it (probably) commonly used in.

OTH just using OGR_SKIP=NAS also handles this case and would not break anything that might be using the driver in other cases (like loading NAS files directly into QGIS - although in that case you might get very similar results using the GML driver).

Not sure what's actually better.

[rouault]

@jef-n OK i'll defer on your judgment having no practical experience with this driver. probably some more prominent notice is needed in the driver doc

[rouault]

@jef-n I've disabled the nas tests for now per 5a4d3f4 as they break the build

I tried https://gist.github.com/jef-n/6e121d62770380047f75a827805b421c but it broke with

Traceback (most recent call last):
  File "../pymod/gdaltest_python2.py", line 43, in run_func
    result = func()
  File "ogr_nas.py", line 322, in ogr_nas_5
    field = feat.GetField('zustaendigestelle_stelle')
  File "/home/even/gdal/git/gdal/gdal/swig/python/build/lib.linux-x86_64-2.7/osgeo/ogr.py", line 4724, in GetField
    raise KeyError("Illegal field requested in GetField()")
KeyError: 'Illegal field requested in GetField()'

And your fix will only work when ogr_nas.py is run individually, and not when run with run_all.py

[tobwen]

Even on GDAL 3.1.0dev, the WFS from my first post is recognized as NAS. --config OGR_SKIP NASactually works. So the fuzzing stuff might not work as expected?