Gravsearch search for list value using complex schema #899

benjamingeer · 2018-06-19T10:13:07Z

Resolves #813.

…inspectors.

…(doesn't work yet)

- Add lots of tests.

- Add pathological type inference test.

- Fix test data.

…ravsearch query.

- Add sanity check for #904. - Remove unused SPARQL query template.

…alues. - Add a test that searches for a list node. - Add missing properties to knora-api v2 complex. - Don't allow the use of knora-api properties that aren't in the specified schema. - Forbid more knora-api complex properties in Gravsearch.

…ction.

benjamingeer · 2018-06-22T13:04:45Z

@tobiasschweizer I disallowed rdf:type in Gravsearch CONSTRUCT clauses, because it actually serves no purpose there. Will that require a change in SALSAH? Also, could you maybe add a more realistic test with a list value from BEOL?

tobiasschweizer · 2018-06-22T13:13:31Z

@benjamingeer I do not think so, but I will check on Monday!

tobiasschweizer · 2018-06-25T09:07:04Z

I disallowed rdf:type in Gravsearch CONSTRUCT clauses, because it actually serves no purpose there. Will that require a change in SALSAH?

I checked the Gravsearch query templates in SALSAH and they work with this branch.

Also, could you maybe add a more realistic test with a list value from BEOL?

You mean a v2 search route test for a beol list value? Sure, I will do that now.

benjamingeer · 2018-06-25T09:41:11Z

You mean a v2 search route test for a beol list value? Sure, I will do that now.

Yes, thanks.

…ific list value

tobiasschweizer · 2018-06-25T09:46:13Z

@benjamingeer and here you go: 81643cd

benjamingeer · 2018-06-25T09:52:30Z

@tobiasschweizer Great, thank you! Would you have time to finish reviewing this today?

tobiasschweizer · 2018-06-25T09:53:47Z

docs/src/paradox/03-apis/api-v2/query-language.md


-#### Functions in FILTER Expressions
+A Knora value may not be represented as the literal object of a predicate;


I think we should see how hard it is to allow literals in the simple schema. The main reason why we forbade this was because it makes the processing harder in SearchResponderV2.

In simple schema, we have to add an extra level by generating a statement for valueHasString. However, from the user's perspective it is rather hard to understand why Gravsearch supports literals in complex schema while it does not in simple schema.

If you used a literal instead of a variable, that would mean you couldn't mention that value in the CONSTRUCT clause, so wouldn't it be excluded from the results? I think this is rarely what the user would want.

By the way, the rule is currently the same for both schemas: you can't represent a Knora value as the literal object of a predicate. In the complex schema, you can use literals for things below the value level, but not for the value itself.

If you used a literal instead of a variable, that would mean you couldn't mention that value in the CONSTRUCT clause, so wouldn't it be excluded from the results?

I think in that case you would just use the same statement:

PREFIX beol: <http://0.0.0.0:3333/ontology/0801/beol/simple/v2#> PREFIX knora-api: <http://api.knora.org/ontology/knora-api/simple/v2#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> CONSTRUCT { ?letter knora-api:isMainResource true . ?letter beol:creationDate ?date . ?letter ?linkingProp1 ?person1 . ?person1 beol:hasFamilyName "Meier" . } WHERE { ?letter a beol:letter . ?letter beol:creationDate ?date . ?letter ?linkingProp1 ?person1 . FILTER(?linkingProp1 = beol:hasAuthor || ?linkingProp1 = beol:hasRecipient ) ?person1 beol:hasFamilyName "Meier". } ORDER BY ?date

I think in a first version of Gravsearch, we allowed for literals. But when I refactored it, I did not support them anymore because the processing logic would have been more complex.

In webapi/src/test/scala/org/knora/webapi/e2e/v2/SearchRouteV2R2RSpec.scala there are still some old tests involving literals that are ignored.

OK, let's look at this after standoff search is done.

tobiasschweizer · 2018-06-25T09:54:15Z

Would you have time to finish reviewing this today?

I am doing it now :-)

tobiasschweizer · 2018-06-25T11:49:59Z

OK, let's look at this after standoff search is done.

Yes, absolutely! I do not think it is urgent since you can do everything with a Filter that you can do with a literal.

tobiasschweizer · 2018-06-25T11:37:36Z

webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala

-                case _ =>
-                    // value property
+            } else {
+                // Is the subject a resource?


If we get here, it is not a linking property, right?

Yes. Clarified the comments there.

tobiasschweizer · 2018-06-25T12:03:24Z

webapi/src/main/scala/org/knora/webapi/responders/v2/SearchResponderV2.scala

@@ -2216,22 +2178,25 @@ class SearchResponderV2 extends ResponderWithStandoffV2 {
            typeInspectionResult: GravsearchTypeInspectionResult <- gravsearchTypeInspectionRunner.inspectTypes(inputQuery.whereClause, requestingUser)
            whereClauseWithoutAnnotations: WhereClause = GravsearchTypeInspectionUtil.removeTypeAnnotations(inputQuery.whereClause)

-            // Preprocess the query to convert API IRIs to internal IRIs and to set inference per statement.
-
-            preprocessedQuery: ConstructQuery = QueryTraverser.transformConstructToConstruct(


Did you get rid of this because in the old way, we just converted everything to internal assuming that we got it in simple schema. Now, we have to check if we get it in simple or complex.

tobiasschweizer · 2018-06-25T12:11:19Z

webapi/src/test/resources/test-data/ontologyR2RV2/standoffOntologySimple.rdf

-	<rdfs:subClassOf rdf:nodeID="genid-55a441049d76415ba3999f7d47ee542f-b13"/>
-	<rdfs:subClassOf rdf:nodeID="genid-55a441049d76415ba3999f7d47ee542f-b14"/>
-	<rdfs:subClassOf rdf:nodeID="genid-55a441049d76415ba3999f7d47ee542f-b15"/>
+	<rdfs:subClassOf rdf:nodeID="genid-0e6b49fe8a374047959ae8a349b1a4de-b8"/>


Are these blank node Iris?

Yes, for cardinalities.

But aren't those different for every request you make?

Yes. Every time you convert a JSON-LD document containing blank nodes to RDF/XML, you get different randomly generated blank node IDs. Semantically it makes no difference.

I just wondered how the tests work. I assume the blank node Iris are just ignored, right?

The tests use RDF4J Rio to parse the RDF/XML into an org.eclipse.rdf4j.model.Model. If you compare two Model objects that contain blank nodes using the == operator, the Model class understands that the blank node IDs aren't significant. It just compares their contents, not their IDs.

Ok, got it. Thanks!

feature (webapi): Allow complex schema in Gravsearch parser and type …

235eb43

…inspectors.

benjamingeer added enhancement improve existing code or new feature API/V2 Gravsearch labels Jun 19, 2018

benjamingeer self-assigned this Jun 19, 2018

Benjamin Geer added 15 commits June 19, 2018 18:10

feature (webapi): Refactor search responder to accept complex schema …

4df4e35

…(doesn't work yet)

Merge branch 'develop' into wip/gravsearch-complex-schema

bb9577c

feature (webapi): Allow complex schema in Gravsearch (ongoing).

0838e14

feature (gravsearch): Support complex schema (ongoing).

62cc9b6

feature (gravsearch): Support complex schema (ongoing).

ee73fa9

Merge branch 'develop' into wip/gravsearch-complex-schema

e1b2b54

feature (gravsearch): Support queries in the complex schema (ongoing).

ed31ce0

- Add lots of tests.

feature (gravsearch): Check for consistent schema usage.

84b8c5b

test (gravsearch): Fix parser tests.

9583c77

- Add pathological type inference test.

feature (gravsearch): Infer types of function arguments.

27c6353

- Fix test data.

feature (gravsearch): Validate schemas in the CONSTRUCT clause of a G…

559227f

…ravsearch query.

test (gravsearch): Fix test data.

add68be

- Add sanity check for #904. - Remove unused SPARQL query template.

Merge branch 'develop' into wip/gravsearch-complex-schema

6b9cdac

fix (gravsearch): Improve identification of valid types in type inspe…

aca694a

…ction.

benjamingeer mentioned this pull request Jun 21, 2018

Implement Standoff Search #630

Closed

feature (gravsearch): Check Gravsearch queries more strictly.

4881bfd

benjamingeer requested a review from tobiasschweizer June 22, 2018 12:59

benjamingeer assigned tobiasschweizer Jun 22, 2018

benjamingeer added the ready-for-review label Jun 22, 2018

style (gravsearch): Remove redundant parentheses.

40a97bc

Merge branch 'develop' into wip/gravsearch-complex-schema

e9a6621

tests (list value): add another Gravsearch query to search for a spec…

81643cd

…ific list value

tobiasschweizer reviewed Jun 25, 2018

View reviewed changes

style (gravsearch): Add clearer comments.

9152d82

tobiasschweizer approved these changes Jun 25, 2018

View reviewed changes

benjamingeer merged commit d2140a8 into develop Jun 25, 2018

benjamingeer deleted the wip/gravsearch-complex-schema branch June 25, 2018 13:06

benjamingeer mentioned this pull request Sep 13, 2018

Ben's PR history #571

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gravsearch search for list value using complex schema #899

Gravsearch search for list value using complex schema #899

benjamingeer commented Jun 19, 2018 •

edited

benjamingeer commented Jun 22, 2018

tobiasschweizer commented Jun 22, 2018

tobiasschweizer commented Jun 25, 2018

benjamingeer commented Jun 25, 2018

tobiasschweizer commented Jun 25, 2018

benjamingeer commented Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer commented Jun 25, 2018

tobiasschweizer commented Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018

benjamingeer Jun 25, 2018

tobiasschweizer Jun 25, 2018


		#### Functions in FILTER Expressions
		A Knora value may not be represented as the literal object of a predicate;

Gravsearch search for list value using complex schema #899

Gravsearch search for list value using complex schema #899

Conversation

benjamingeer commented Jun 19, 2018 • edited

benjamingeer commented Jun 22, 2018

tobiasschweizer commented Jun 22, 2018

tobiasschweizer commented Jun 25, 2018

benjamingeer commented Jun 25, 2018

tobiasschweizer commented Jun 25, 2018

benjamingeer commented Jun 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tobiasschweizer commented Jun 25, 2018

tobiasschweizer commented Jun 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Jun 19, 2018 •

edited