Subqueries keep their orderBy clauses and the query optimizer is ..

[joka921]

aware of cheap filters

• Previously all OrderBy clauses that occured in subqueries were deleted. This gained speed when their were superfluous orderBys that were overwritten by ancestor's orderBy clauses
  but plain wrong if there is an orderBy clause in a subquery but not in the main query

• Also makes the query planner aware that filters are even cheapier if their input is sorted.

Fixes #304

[joka921]

I have build a Docker container called qlever-ordered-subqueries on galera, so if
@hannahbast or @niklas88 could (at least temporarily) replace the running wikidata-full with this
image we could immediately verify if this

• fixes the problem
• enables us to speed up the simply autocompletion queries that currently take ~150ms to basically 0.

[niklas88]

My account seems to have been deactivated, I'll clear that with Hannah and Frank

[niklas88]

Do you have a test query on scientists, this would be a good candidate for the end to end tests.

[hannahbast]

I started it and first got an Exception: ParseException, cause: The literal: "^"Xylo" was not terminated properly. I then replaced the \" in the regular expression by \\\" and then it worked. It seems you are not working with the master but with some more advanced branch. Anyway, the good news is:

1. The query above now produces results in sorted order.

2. The time needed for the filter operation is now 0ms.

Great!

[joka921]

@hannahbast This is indeed the master branch, we currently fixed this Escaping in literals because it is actually required.
There is also the problem, that it should be FILTER regex(?x, "^something") that should match literals,
but this is something I will tackle after merging the unicode branch to not get too many conflicts

[joka921]

@hannahbast Thanks for the feedback, Indeed you found a bug in the current master branch for which I partially take responsibility. I just added a commit that should fix this issue to this PR.

It is a symptom of a much bigger problem: Currently everything is a std::string inside QLever.
This makes it hard to reason about "Is this a Literal, an entity, A Prefix regex that was escaped on a certain level, a floating point value" etc. which has led to several bugs in the past and makes maintaining the code hard.
I will work on encoding this information into the Type system soon by making all those distinct types without implicit conversion.

[joka921]

Ok, so now we are back to:

FILTER regex(?x, "^\"xylo) works. (previous status of the code, fixing this is a separate issue.
FILTER regex(?x, "^\\\"xylo) throws an understandable Error that the user overescaped the " which is not necessary and not supported in QLever.

[niklas88]

Why do you have the std::literals::string_literals::operator""s only on the first literal?

[niklas88]

LGTM with just a small clariffication question, feel free to merge