F.temp optimizer and sort order merged (DONT MERGE, but Hannah may test it) #326
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- The previous version put all triples and filters that appeared directly in a WHERE clause together.
- this is not correct, since always when an OPTIONAL, UNION, SUBQUERY etc. appears a new scope
is started. For example in
SELECT ?x ?y WHERE {
?x <is-a> <Scientist>
OPTIONAL { ?x <Spouse> ?y}
FILTER ?x < <Ada_Lovelace>
}
the filter should have no effect, because it is not in the same scope as the x is a scientist triple.
the probably more useful query would be
SELECT ?x ?y WHERE {
?x <is-a> <Scientist> .
FILTER ?x < <Ada_Lovelace>
OPTIONAL { ?x <Spouse> ?y}
}
where the filter is actually applied on the entities that are scientists.
However, QLever previously handled these queries both in the second way.
- To correct this, implemented a BasicGraphPattern (triples, filters and values that are "directly" in the Body of another graph pattern or a WHERE clause) class.
BasicGraphPatterns are also children of their parent body.
- In addition, Optionals were also treated wrong as the GraphPatterns must be treated in order which makes a difference for Optionals.
- This version of the QueryPlanner does some degree of optimization, however one could optimize multiple adjacent GraphPatterns that are not Optional, if their
scopes are respected.
- TODO: There still are bugs in the handling of optionals as the SPARQL standard knows "unbound values" which can be rebound, e.g.
SELECT ?x ?name WHERE {
?x <is-a> <human>
OPTIONAL {?x <TwitterAccount> ?name}
OPTIONAL {?x rdfs:label ?name }
}
which should only add the names only for humans that don't have twitter accounts and the handling of the empty query
SELECT ?x WHERE {} (1 result where ?x is unbound/NULL)
- TODO: Some candidate plans found by optimizing a TransitivePath are currently unable to be joined with adjacent patterns.
It seems that we can always find a suitable candidate, so that this works, but we still should inspect, whether this is the correct behavior.
…type of GraphPatternOperation. The parser already compiles.
TODO: clean up, self-review, commit messages and squash/rewrite history.
…identical again. Next step: make merge call join.
TODO<Hannah> checkout how this performs on the Wikidata evaluation.
TODO: squash, git history and then self-review (but first let them do the ParsedQuery stuff.)
# Conflicts: # test/QueryPlannerTest.cpp
…. Not yet working at all.
…ern trick in cases where it was actually legal.
…rack down the strange error of prefix filters.
All other types of filter disjunctions will currently lead to parse errors since they are harder to properly evaluate and to estimate during planning.
# Conflicts: # src/engine/Filter.cpp # src/parser/SparqlParser.cpp
All other types of filter disjunctions will currently lead to parse errors since they are harder to properly evaluate and to estimate during planning.
# Conflicts: # src/engine/QueryPlanner.cpp
…OrderMerged # Conflicts: # e2e/scientists_queries.yaml
TODO: merge with the new optimizer to properly include it, and make this all work and test it.
…put. TODO: UnitTest this.
# Conflicts: # src/parser/ParsedQuery.h
f compile. Not yet done at all.
TODO: cleanUp + verification + Optimization for Max.
TODO: limit the number of threads and make it configurable.
…ing the vocabulary from disk.
# Conflicts: # src/engine/QueryPlanner.cpp # src/parser/ParsedQuery.cpp # src/parser/ParsedQuery.h # test/QueryPlannerTest.cpp
- The syntax is BIND ("Some String" AS ?new_variable)
# Conflicts: # src/engine/GroupBy.cpp # src/engine/QueryPlanner.cpp # src/parser/ParsedQuery.h # test/QueryPlannerTest.cpp
|
There is a newer version of this "One for everything TEMP pr, close this one. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.