wfs:wfs-1.1.0-Basic-GetFeature-tc4 -- checks look incorrect (wrong value)

[davidblasby]

I am looking at the wfs-1.1.0-Basic-GetFeature-tc4 tests and getting failures. However, I think my results are correct and the test case is incorrect.

service

request2 makes a "bad" service KVP request:

https://github.com/opengeospatial/ets-wfs11/blob/master/src/main/scripts/ctl/basic/GetFeature/GetFeature-GET.xml#L438

However, the check tests for a bad request

https://github.com/opengeospatial/ets-wfs11/blob/master/src/main/scripts/ctl/basic/GetFeature/GetFeature-GET.xml#L525

This should be "service"

version

Likewise, for request3, version is bad;

https://github.com/opengeospatial/ets-wfs11/blob/master/src/main/scripts/ctl/basic/GetFeature/GetFeature-GET.xml#L461

However, the check tests for a bad request

https://github.com/opengeospatial/ets-wfs11/blob/master/src/main/scripts/ctl/basic/GetFeature/GetFeature-GET.xml#L545

This should be "version".

NOTE: request4 is correct (tests request)

[davidblasby]

The other issue with the "version" test is that its unclear which ExceptionReport should be given.

For example, Geoserver support WFS 1.0.0, 1.1.0, and 2.0

If no version=... is given, it unclear with version of the exceptionreport should be given.

I believe that ets-wfs11 assumes this is a WFS 1.1.0 exception.

However, geoserver returns a WFS 2.0 exception. This looks the same (xml document content), however the namespaces are different. I believe the ets-wfs11 parser cannot handle these types of exceptions.

`Validation error: cvc-elt.1.a: Cannot find the declaration of element 'ows:ExceptionReport'. 1 validation error detected.

<ows:ExceptionReport xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.0" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://10.0.0.219:8080/geoserver/schemas/ows/1.1.0/owsAll.xsd"> <ows:Exception ...> ... </ows:Exception> </ows:ExceptionReport>

You can see the namespace for this is xmlns:ows="http://www.opengis.net/ows/1.1" while the WFS 1.1.0 namespace is xmlns:ows="http://www.opengis.net/ows"

Perhaps this test can be changed so it just checks that a document is returned with "ExceptionReport" in the text.

[jodygarnett]

This is very similar to #66

[davidblasby]

PR is here - #97

[dstenger]

Thank you for providing the pull request.
Your changes seem to be fine. I also agree that making the test suite more lax when no version parameter is provided is OK as this is a general problem we are having with many test suites when either version or service is not clear (only solution would be to setup individual services for each service version which is not very practical).
We will do some functional tests and merge the pull request afterwards.

[davidblasby]

@dstenger @bpross-52n

Just wanted to follow up on this and #97

Thanks a lot!

[dstenger]

@davidblasby
I just reviewed your pull request.
Somehow, when using a reference implementation [1] both versions of the test suite (current master and your branch) pass successfully. This gives me the impression that the checks are not applied at all. When checking the service, the behavior is as provided by your pull request.
Can you provide a test service which fails with current master (or Production) and passes with your fix?

The fix regarding the version is currently also a discussed issue in other test suites. So, @ghobona and @bpross-52n we should find a general solution how to tackle this problem consistently between all test suites.

[1] https://cite.deegree.org/deegree-webservices-3.4.30/services/wfs110?service=WFS&request=GetCapabilities

[davidblasby]

I have been running the test locally (localhost). However, you can try here (GeoServer);

https://how2map.geocat.live/geoserver/ows?service=wfs&version=1.1.0&request=GetCapabilities

(that doesn't have the CITE data in it)

If a server ONLY supports WFS 1.1, I expect that the test would work. However, if it supports multiple version (i.e. 2.0) then I would expect the test to potentially fail (since it will likely be sending a 2.0 error). Geoserver returns a 2.0 error message, which doesn't validate in the test case.

Looking at:

https://cite.deegree.org/deegree-webservices-3.4.30/services/wfs110?service=WFS&request=GetFeature

This returns a 1.0.0 Exception (should it be 1.1?):

<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd" version="1.0.0">
  <ows:Exception exceptionCode="MissingParameterValue" locator="version">
    <ows:ExceptionText>Missing version parameter.</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

However, for GS supports multiple versions (1.0, 1.1, and 2.0). If you don't put in a version, it defaults to the latest version for processing. You will eventually get a "version missing" error.

https://how2map.geocat.live/geoserver/ows?service=wfs&request=GetFeature&typename=naturalearthscaleranks:110m_admin_0_boundary_lines_land_us

This gives a missing typesnames error.
Using typenames:

https://how2map.geocat.live/geoserver/ows?service=wfs&request=GetFeature&typenames=naturalearthscaleranks:110m_admin_0_boundary_lines_land_us

I think the first should be giving a different error message (version missing) - but I can fix that later (if needed).

However, that error message returned is a 2.0 error message (ows 1.1):

<ows:ExceptionReport xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.0" xsi:schemaLocation="http://www.opengis.net/ows/1.1 https://how2map.geocat.live/geoserver/schemas/ows/1.1.0/owsAll.xsd">
<ows:Exception exceptionCode="MissingParameterValue" locator="version">
<ows:ExceptionText>Could not determine version</ows:ExceptionText>
</ows:Exception>
</ows:ExceptionReport>

[dstenger]

Regarding the version of the exception report:

WFS 1.1 specification (OGC 04-094) references "Whiteside, Arliss (ed.), “OWS Common Implementation Specification, V0.3.0”,
Open Geospatial Consortium Inc. document 04-016r3". Unfortunately, I cannot find the document on the web. However, I strongly assume that this is version 1.0.0 of OWS Common (the test suite also expects OWS schemas of version 1.0.0).
Your service also returns version 1.0.0: https://how2map.geocat.live/geoserver/ows?service=wfs&version=1.1.0&reequest=GetFeature&typenames=naturalearthscaleranks:110m_admin_0_boundary_lines_land

Test of fix

Unfortunately, the tests are not executed with your provided endpoint.
I did some further research and found out that tests of request2 and request3 are not executed (at least the if clauses).
A code analysis led to the result that the is an error in the expression of request3:
Must be <xsl:if test="not(contains($request3, 'version'))"> instead of <xsl:if test="not(contains($request3, 'version')"> (closing) is missing).
Not executed request2 was caused by a bug in TEAM Engine. An update to TEAM Engine v5.5.2 solved that problem (see #99).

[davidblasby]

yes - if you ask for "version=1.1.0" you will get a 1.1.0 (ows 1.0.0) error. However, the problematic test doesn't include a 'version' so the error report version isn't clearly defined - i.e. should this return a WFS 1.0.0, 1.1.0, or 2.0 error?

[davidblasby]

I updated the PR - see note there.

Thanks a lot!

[dstenger]

Discussion during CITE team meeting 2022-01-26:
We accept the changes regarding the version. However, I would like to notify you there is an ongoing discussion about this topic in other SWGs as well.