ST_Intersects with MultiPolygons returns 42000 Error GEO

[rapw3k]

Hi,
Some months ago, I had an issue with ST_Intersects with complex polygons, I got error:

Virtuoso 42000 Error GEO..: for after check of geo intersects, some shape types (e.g., polygon rings and curves) are not yet supported; see https://www.mail-archive.com/virtuoso-users@lists.sourceforge.net/msg08182.html

@HughWilliams recommended to upgrade virtuoso. After I did it, this issue got solved with that query. However, now I am now having the same issue but with other query....
I am using Virtuoso version 07.20.3217
And you can test my endpoint at https://www.foodie-cloud.org/sparql

Query (with error triggering POLYGON) is below.

• If I use Geometry: POLYGON((13.463186854291394 50.22782104565781,13.52405965702576 50.22782104571184,13.523373011517947 50.19282422040329,13.464560145307019 50.19194502376654,13.463186854291394 50.22782104565781)), it works
• If I use Geometry: POLYGON((13.463186854291394 50.22782104565781,13.52405965702576 50.22782104571184,13.523373011517947 50.18359184367178,13.464560145307019 50.18359184367178,13.463186854291394 50.22782104565781)) it throws error
• If I use Geometry (that is the difference): POLYGON((13.463186854291394 50.18359184367177761, 13.52405965702576 50.18359184367177761, 13.523373011517947 50.19194502376654, 13.464560145307019 50.19194502376654, 13.463186854291394 50.18359184367177761)), it works

So, I cannot understand what is going on here, and is not just about upgrading.
Could you give me some help?
thanks,
Raul

PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> 
PREFIX poi: <http://www.openvoc.eu/poi#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foodie-cz: <http://foodie-cloud.com/model/foodie-cz#>
PREFIX foodie: <http://foodie-cloud.com/model/foodie#>
PREFIX olu: <http://w3id.org/foodie/olu#>


SELECT DISTINCT ?plot ?code ?shortId ?landUse ?coordPlotFinal
WHERE {
   ?plot geo:hasGeometry ?geoPlotFinal .
   ?geoPlotFinal ogcgs:asWKT  ?coordPlotFinal .
   FILTER(bif:st_intersects(?coordPlotFinal, ?coordWBody, 0.00025)) .
   
   GRAPH ?graph1 {
      SELECT ?plot ?code ?shortId ?landUse
      FROM <http://w3id.org/foodie/open/cz/180308_pLPIS_WGS#>
      WHERE{ 
         ?plot a foodie:Plot ;
            foodie:code ?code ;
            foodie-cz:shortId ?shortId ;
            olu:specificLandUse ?landUse ;
            geo:hasGeometry ?geoPlot .
         ?geoPlot ogcgs:asWKT  ?coordPlot .
         FILTER(bif:st_intersects (?coordPlot, bif:st_geomFromText("POLYGON((13.463186854291394 50.22782104565781,13.52405965702576 50.22782104571184,13.523373011517947 50.18359184367178,13.464560145307019 50.18359184367178,13.463186854291394 50.22782104565781))"))) .   
     }
   }
   GRAPH ?graph2 {
      SELECT ?waterBody ?label ?coordWBody
      FROM <http://w3id.org/foodie/open/cz/Water_bodies_buff25m_WGS#>
      WHERE {
          ?waterBody a foodie-cz:WaterBody ;
                 rdfs:label ?label ;
                 geo:hasGeometry ?geoWBody .
                 ?geoWBody ogcgs:asWKT ?coordWBody .
     FILTER(bif:st_intersects (?coordWBody, bif:st_geomFromText("POLYGON((13.463186854291394 50.22782104565781,13.52405965702576 50.22782104571184,13.523373011517947 50.18359184367178,13.464560145307019 50.18359184367178,13.463186854291394 50.22782104565781))"))) .
      }
   }
}

[rapw3k]

I have to clarify that the outer intersect in the query: FILTER(bif:st_intersects(?coordPlotFinal, ?coordWBody, 0.00025)), works over MULTIPOLYGON shapes (both ?coordPlotFinal and ?coordWBody return MULTIPOLYGONs).

Each of the inner queries (?graph1 and ?graph2) work fine individually. So the problem is in the outer intersect

[TallTed]

Please note that 7.20.3229 is current, and this does include a number of GeoSPARQL-related fixes, relative to your 7.20.3217. I would therefore encourage testing with the current stable/7, if not develop/7.

[kidehen]

Do we have a Turtle doc containing the data in question so we can load and test so our URIBurner instance?

[TallTed]

@rapw3k - We'd like to work on this further with you. To do so, we'll need a data dump that we can load locally, for best analysis. If you don't want to make such a dump public, you can email a private download link to us.

[rapw3k]

Thank you @TallTed @kidehen , I found an issue in two geometries of the original dataset, so, I will first try reloading the dataset, to discard any geometry related issue, and then if the problem persist I will come back to you. I also plan to upgrade our virtuoso in the next few days.

[rapw3k]

Hi @TallTed @kidehen, I loaded new datasets and made sure no geometry related issues were in other ones. Then I upgraded virtuoso to latest version 07.20.3229 .

Now it is worse. I cannot even query previous datasets that were working fine.
I am about to revert to previous version of virtuoso.

See sample file (nt) that you can download and use it for testing.
https://box.psnc.pl/f/78a4d75f37/?raw=1

I tried simple query below, and I get error: Virtuoso 42000 Error GEO..: for after check of geo intersects, some shape types (e.g., polygon rings and curves) are not yet supported.
You can also check in our endpoint: https://www.foodie-cloud.org/sparql

Any ideas?

PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> 
PREFIX poi: <http://www.openvoc.eu/poi#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foodie-cz: <http://foodie-cloud.com/model/foodie-cz#>
PREFIX foodie: <http://foodie-cloud.com/model/foodie#>
PREFIX olu: <http://w3id.org/foodie/olu#>


SELECT DISTINCT ?plot ?code ?shortId ?landUse ?coordPlotFinal
FROM <http://w3id.org/foodie/open/cz/180308_pLPIS_WGS_sample#>
WHERE{ 
  ?plot a foodie:Plot ;
  foodie:code ?code ;
  foodie-cz:shortId ?shortId ;
  olu:specificLandUse ?landUse ;
  geo:hasGeometry ?geoPlot .
  ?geoPlot ogcgs:asWKT  ?coordPlotFinal .
  FILTER(bif:st_intersects (?coordPlotFinal, bif:st_geomFromText("POLYGON((15.921601771283576 49.78152920564588,15.877656458783576 48.6876051971939,18.333100794721076 48.65858571922563,18.327607630658576 49.79571495785491,15.921601771283576 49.78152920564588))"))) .    
}

[HughWilliams]

@rapw3k The current Virtuoso 7 open source versions only supports points and does not support other geometry shapes/types.

Support for additional geometry types like polygon, multi-line strings, multi-polygons, geometry collections and GeoSPARQL has been added to the latest Virtuoso 8.1+ commercial product, thus loading your dataset into our uriburner.com instance, the query you provide above does run, see:

http://linkeddata.uriburner.com/sparql?default-graph-uri=&qtxt=PREFIX%20geo%3A%20%3Chttp%3A%2F%2Fwww.opengis.net%2Font%2Fgeosparql%23%3E%0APREFIX%20geof%3A%20%3Chttp%3A%2F%2Fwww.opengis.net%2Fdef%2Ffunction%2Fgeosparql%2F%3E%0APREFIX%20virtrdf%3A%09%3Chttp%3A%2F%2Fwww.openlinksw.com%2Fschemas%2Fvirtrdf%23%3E%20%0APREFIX%20poi%3A%20%3Chttp%3A%2F%2Fwww.openvoc.eu%2Fpoi%23%3E%20%0APREFIX%20rdfs%3A%20%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E%0APREFIX%20foodie-cz%3A%20%3Chttp%3A%2F%2Ffoodie-cloud.com%2Fmodel%2Ffoodie-cz%23%3E%0APREFIX%20foodie%3A%20%3Chttp%3A%2F%2Ffoodie-cloud.com%2Fmodel%2Ffoodie%23%3E%0APREFIX%20olu%3A%20%3Chttp%3A%2F%2Fw3id.org%2Ffoodie%2Folu%23%3E%0A%0A%0ASELECT%20DISTINCT%20%3Fplot%20%3Fcode%20%3FshortId%20%3FlandUse%20%3FcoordPlotFinal%0AFROM%20%3Chttp%3A%2F%2Fw3id.org%2Ffoodie%2Fopen%2Fcz%2F180308_pLPIS_WGS_sample%23%3E%0AWHERE%7B%20%0A%20%20%3Fplot%20a%20foodie%3APlot%20%3B%0A%20%20foodie%3Acode%20%3Fcode%20%3B%0A%20%20foodie-cz%3AshortId%20%3FshortId%20%3B%0A%20%20olu%3AspecificLandUse%20%3FlandUse%20%3B%0A%20%20geo%3AhasGeometry%20%3FgeoPlot%20.%0A%20%20%3FgeoPlot%20ogcgs%3AasWKT%20%20%3FcoordPlotFinal%20.%0A%20%20FILTER(bif%3Ast_intersects%20(%3FcoordPlotFinal%2C%20bif%3Ast_geomFromText(%22POLYGON((15.921601771283576%2049.78152920564588%2C15.877656458783576%2048.6876051971939%2C18.333100794721076%2048.65858571922563%2C18.327607630658576%2049.79571495785491%2C15.921601771283576%2049.78152920564588))%22)))%20.%20%20%20%20%0A%7D&should-sponge=&format=text%2Fhtml&CXML_redir_for_subjs=121&CXML_redir_for_hrefs=&timeout=30000000&action=logout

Adding similar support to Virtuoso open source is scheduled to be done but not available yet ...

[rapw3k]

Hi @HughWilliams , I am pretty sure, support for other geometry types including polygon and multi polygon were supported. This was working (in most cases) before upgrading to latest version.

I made several demoes using points, polygons and multi-polygons.

Also documentation supports this claim:
http://docs.openlinksw.com/virtuoso/sqlrefgeospatial7enchance/#sqlrefgeospatial7enchancedtsupported

So, it is not a commercial product feature, or at least it was not. I would very disappointed if that would be the case.

[HughWilliams]

@rapw3k: The only way this could have previously work is if you had built form the old "v7fasttrack" R&D branch:

 https://github.com/v7fasttrack/virtuoso-opensource

where some of this functionality had been made available sometime ago as detailed at:

  http://docs.openlinksw.com/virtuoso/sqlrefgeospatial7enchance/#sqlrefgeospatial7enchanceplgcomp

and the productisation has not been completed until now for addition to the main archives ...

[rapw3k]

@HughWilliams
I am now getting very confused.
How is then possible that the query below works in my current installation 07.20.3229 ?
This is only working over polygons and multi-polygons.

Besides, the documentation says that the v7fasttrack is only needed for the Virtuoso v7proj4 plugin module that is required for performing transformation between different co-ordinates systems using the ST_Transform() function.

PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> 
PREFIX poi: <http://www.openvoc.eu/poi#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foodie-cz: <http://foodie-cloud.com/model/foodie-cz#>
PREFIX foodie: <http://foodie-cloud.com/model/foodie#>
PREFIX olu: <http://w3id.org/foodie/olu#>

SELECT ?waterBody ?label (bif:GeometryType (?coordWBody) as ?shapeType)
      FROM <http://w3id.org/foodie/open/cz/Water_bodies_buff25m_WGS#>
      WHERE {
          ?waterBody a foodie-cz:WaterBody ;
                 rdfs:label ?label ;
                 geo:hasGeometry ?geoWBody .
                 ?geoWBody ogcgs:asWKT ?coordWBody .
     FILTER(bif:st_intersects (?coordWBody, bif:st_geomFromText("POLYGON((13.463186854291394 50.22782104565781,13.52405965702576 50.22782104571184,13.523373011517947 50.18359184367178,13.464560145307019 50.18359184367178,13.463186854291394 50.22782104565781))"))) .
      }
order by ?shapeType

[HughWilliams]

@rapw3k: Correction, my mistake ... the work on polygons and other shapes was added to open source and commercial when the first phase was done a few years ago as I have used it in both ... but the necessary plugins modules for GeoSPARQL support/compliance and performing CRS transformations in the v7fasttrack branch were not completed ... they are now completed and currently available in the latest in-house Virtuoso 8.1 commercial archive (as running on uriburner.com) and to be scheduled for addition the to open source archive ... so the original comment still remains true in that this issue will be fixed when the work from commercial archive is made available in the open source archive ...

[rapw3k]

Hi @HughWilliams , sorry but your answer is still not clear to me...

1. GeoSPARQL support in (latest) open source v7 is available, correct? I have used it many times and with many different datasets...

2. performing CRS transformations in latest open source v7 is NOT available, correct? I am not interested at the moment in the CRS transformations...And my initial query is not performing any transformation.

First key question: Why the same simple query for practical effects (intersects between polygon and multipolygons) works for one graph [2] and not for another [1] ?
Second key question: Why this simple query [1] was working in 07.20.3217 and not in 07.20.3229 ? I made the upgrade because @TallTed suggested it. But before the upgrade the only query not working was the original query in the thread.

[1] #775 (comment)
[2] #775 (comment)

[rapw3k]

You can also see the release notes from https://github.com/openlink/virtuoso-opensource/blob/stable/7/NEWS.md#february-17-2015-v720

GEO functions

Added support for SPARQL INSERT with GEO literals
Added support for upcoming proj4 plugin
Fixed issue with rdf box with a geometry rdf type and a non geometry content
Fixed calculation of serialization lengths for geo boxes
Fixed compilation of a query with bif:st_intersects inside service {}
Fixed serialization of geo boxes
Fixed intersect to working with other geo shapes

[HughWilliams]

@rapw3k

2015 is when the support for the additional geometry shapes/functions was added to open source and commercial to improve our Geo Spatial support in Virtuoso (SPARQL & SQL) ...

Note the release note says "Added support for upcoming proj4 plugin" , which is what has been added to Virtuoso 8 commercial now and then needs to also be added to open source ...

What do you mean by GeoSPARQL (https://en.wikipedia.org/wiki/GeoSPARQL) as it is a standard using geo: and geof: prefixes that Virtuoso 7 does not support ... so although we support polygons and other Geometry types in open source we are not GeoSPARL compliant, until the work from commercial is made available in open source ...

With regards to your issue with Geo query that worked in 3217 but not in 3229, it could be due to a bug or regression, but as we have tested with the latest GeoSpatial work (to be GeoSPARQL compliant) done in commercial, when this is made available in open source it will also work ...

[rapw3k]

For my purposes it was enough for now to use the bif: geo functions to manipulate geospatial data....
I will try reversing to previous version, which was slightly better.....
In any case this is rather sad and disappointing after all demoes I have made and published using virtuoso see e.g., [1] [2].

To conclude, I understand you cannot support me with these issues because I am using v7 open source, and you have solved all issues in v8 which is commercial, and you plan someday to push these fixes to v7 (or v8 open source?) but you dont know when (date) that will happen, I am right?

[1] https://www.slideshare.net/rapw3k/publication-of-inspirebased-agricultural-linked-data
[2] https://www.slideshare.net/rapw3k/inspire-hack-2017linkeddata

[HughWilliams]

@rapw3k: It is planned than these updates will be made to the existing Virtuoso 7 open source code base and this will be soon, although no date has been set for it yet ...

[rapw3k]

Yes that was my guess...

I was hoping you had also a v7 uriburner for support....

Nevertheless, the question: Why the same simple query for practical effects (intersects between polygon and multipolygons) works for one graph [2] and not for another [1] is unclear

I also point out that the less accurate bif:st_may_intersect works even with the initial (more complex) query.

[1] #775 (comment)
[2] #775 (comment)

[rapw3k]

I reverted to 07.20.3217, and is much better, all simple queries above work. I just come back to my original issue, that the complex query sometimes works but sometimes it throws error, depending on the input polygon in the intersect...

As a temporal workaround (until either v8 updates are pushed to virtuoso OS, or other solutions like Straton are tested) -> when there is an error, we can use the less accurate bif:st_may_intersect