Search plugin not working for QGIS Server

[WCarlsson]

Edit

This became quite big as we realised that what we really want to do is rewrite how search results are handled inside Hajk. Currently the entire search result flow is built around – and expects – GeoJSON objects. A better, and more format independent solution, would be to accept any standard geo format as response and convert that to a common ground (i.e. OpenLayer's Features) before going further down the flow.

So what we'll be doing now is exactly that. Search will be rewritten to work around OpenLayer's features, and it will accept both GML and JSON responses.

As it is a quite substantial rewrite, I'm creating a separate feature branch from here: https://github.com/hajkmap/Hajk/tree/feature/882-search-uses-ol-features.

Original issue

Describe the bug
When using the search plugin with a QGIS Server layer there is no result shown in the search bar. The response from the server is in gml-format but it seems Hajk expects json-format.
We have tried to add outputFormat=application/vnd.geo+json to the request URL but the response is still in gml-format.

To Reproduce
Steps to reproduce the behavior:

1. Set up a search layer from QGIS Server in Hajk admin and activate it in a map
2. Search for a value that should give a hit
3. See that no value populates the search bar
4. If you look at the response you can see a hit in gml-format

Expected behavior
The search bar should be populated with the expected values.

Desktop (please complete the following information):

• OS: Windows 10
• Browser: Any

[jacobwod]

We had a look at this and have managed to get back results from textual search. The spatial search needs more work though. In addition, we've found that there seems to be some issues with how different coordinates are handled: QGIS seems to return Lon/Lat even if we as for e.g. EPSG:3008. Perhaps there are some options that need to be made in the QGS file?

If you're ready to work on this together, let me know. It'd be nice if you have some public QGIS Server that we can access for testing purposes, feel free to PM me with details.

Below is a response for a QGIS WFS GetFeature request:

[jacobwod]

@WCarlsson, I've pushed what I've got so far to develop. Give it a go locally and let me know the results.

[matself]

Hi! If this can wait a week or so, I would be happy to provide access to a public QGIS 3.20 server that I set up yesterday.
I need to populate the server with some open data and services first, which will happen early next week at the earliest. I am out of office over the weekend.

[Hallbergs]

@matself It would be great with some open data from QGIS server so that we could run some more tests!

[matself]

Ok. Jag hör av mig nästa vecka om ni inte hittar någon annan dessförinnan.

[jacobwod]

I've made some progress again. But it's a dead end. It turns out that will always return geometries in WGS84, no matter the srsName value provided in the request. This means that we must rewrite substantial chunks of the search model. More info on the limitation here: qgis/QGIS#25171.

And here are my results. Only the value of outputFormat differs in the requests. Note that while GML output does respect SRS, GeoJSON will always return WGS84:

# Req GML:
<GetFeature xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" outputFormat="GML2" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Query typeName="sitac_angrepp" srsName="EPSG:3008"></Query>
</GetFeature>

# Res GML (relevant part only):
<qgs:geometry>
                <MultiLineString xmlns="http://www.opengis.net/gml" srsName="EPSG:3008">
                    <lineStringMember xmlns="http://www.opengis.net/gml">
                        <LineString xmlns="http://www.opengis.net/gml">
                            <coordinates xmlns="http://www.opengis.net/gml" ts=" " cs=",">131704.75777081,6303573.79089982 130363.42703218,6304008.2594847</coordinates>
                        </LineString>
                    </lineStringMember>
                </MultiLineString>
            </qgs:geometry>


-------------------

# Req GeoJSON:
<GetFeature xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" outputFormat="application/json" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Query typeName="sitac_angrepp" srsName="EPSG:3008"></Query>
</GetFeature>

# Res GeoJSON (relevant part only):
"geometry": {
                "coordinates": [
                    [
                        [
                            13.200135,
                            56.852906
                        ],
                        [
                            13.178117,
                            56.856753
                        ]
                    ]
                ],
                "type": "MultiLineString"
            },

[Hallbergs]

Interesting discussion on the Qgis gh… I guess we’ll port it to gml instead then? That or transforming the returned features to the map-projection.

[jacobwod]

Yeah… so here's an interesting find, see comments inside XML:

<GetFeature xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" outputFormat="GML2" maxFeatures="100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd">
    <Query typeName="sitac_vind" srsName="EPSG:3008">
        <Filter xmlns="http://www.opengis.net/ogc">
            <Intersects>
                <PropertyName>geometry</PropertyName>
                <Polygon xmlns="http://www.opengis.net/gml" srsName="EPSG:3008">
                    <exterior>
                        <LinearRing srsName="EPSG:3008">
                            <!-- The coordinates below is what OpenLayers sends in the WFS request. 
                                 They seem to be in the SRS of View. In my case, it's 3008. 
                                 They won't work with QGIS Server, as it expects the request coordinates to be 
                                 in the same SRS as the layer itself. -->
                            <!-- <posList srsDimension="2">69188 6245797 69188 6321789 152012 6321789 152012 6245797 69188 6245797</posList> -->

                            <!-- So when the coordinates are converted to the SRS of the layer itself, 3006 in this case, they work perfectly. -->
                            <posList srsDimension="2">326485.31 6246069.79 326485.31 6401142.83 6401142.83 6401142.83 6401142.83 6246069.79 326485.31 6246069.79</posList>
                            
                            <!-- Conclusion: QGIS Server ignores the srsName for the request so we must transform the REQUEST'S coordinates client-side. -->
                            
                            <!-- srsName does however affect the response, at least for the GML output.
                                 If outputFormat="application/json", we will always get WGS84 from QGIS Server though. 
                            -->
                        </LinearRing>
                    </exterior>
                </Polygon>
            </Intersects>
        </Filter>
    </Query>
</GetFeature>

And just for fun: a screenshot with the response that QGIS Server send, in WGS84…

[jacobwod]

It turns out that QGIS Server does understand WFS GetFeature requests in other SRS than the layer's, but you must explicitly enable those SRSes in the project. In addition, it's super counterintuitive because a setting grouped under the WMS settings does affect a WFS response!

Before

The project's SRS and the "default" world projections available.

After

Available projections are exposed in the response, the "default" projections are gone.

[jacobwod]

@WCarlsson Oh, I almost forgot: I did some major changes to how search works yesterday and you should be able to check out this branch and get the textual search going. Let me know if you want some guidance on how to checkout another branch and build locally (I think you've already done it though, but I'll help you out in case you need!)

[jacobwod]

Just some more work to do in order to get the spatial search going. QGIS Server seems to be very formal in which request bodies are allowed: if the polygon sent in the request has duplicate points, QGIS won't give any response at all. See below, in the first screen shot the first and last points are duplicates and there is no response. As soon as I remove the duplicate, I get the correct response.

Duplicate

No duplicate

[jacobwod]

I'm ready to merge it to develop but would like to see some test before. Can some of you please checkout feature/882-search-uses-ol-features branch and test locally? @WCarlsson

Make sure to select the appropriate settings, I've tested both GML2/3 and GeoJSON with QGIS Server and GeoServer.

[WCarlsson]

I can´t seem to get it to work properly.
The textual search works fine and if I search for an object with only a few hits, 2 in my case, they get marked in the map and a broad bounding box is zoomed to.
But if I select one of the objects the client crashes and i get errors.
I have tried with the different respone types provided and get the same result with both GML and GEOJSON with our QGIS Server.

Object is select in the list of results:

Object is specifically searched for:

[jacobwod]

The first error is interesting. What happens if you disable the StreetView plugin in Admin UI for the current map and try again?

[WCarlsson]

Different kind of error, looks like react errors. I have tried reinstalling npm-packages after i switched branch but it hasn´t helped.

[jacobwod]

This is totally unexpected. Let's take a look together!

[jacobwod]

Turns out the previous errors where due to typos in config files. I've merged this feature into develop in d6bf36c and I'm closing this issue now.

[Hallbergs]

Fantastic work getting the search working for QGIS-server-data! @jacobwod