Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Completions for property paths with * get error response #311

Closed
jbuerklin opened this issue Jan 19, 2020 · 2 comments
Closed

Completions for property paths with * get error response #311

jbuerklin opened this issue Jan 19, 2020 · 2 comments

Comments

@jbuerklin
Copy link
Contributor

jbuerklin commented Jan 19, 2020

On http://qlever.informatik.uni-freiburg.de/Wikidata_Full, when retreiving completions for

PREFIX wdt: <http://www.wikidata.org/prop/direct/>
SELECT WHERE {
  ?x wdt:P31/wdt:P279* [cursor]
}

QLeverUI sends following query:
(The currently available version of QLeverUI on http://qlever.informatik.uni-freiburg.de/ does not build the same query as I'm describing below, but I already pushed an update that will result in queries that look exactly like the following one.)

SELECT ?qleverui_entity (SAMPLE(?qleverui_name) as ?qleverui_name) (SAMPLE(?qleverui_count) as ?qleverui_count) WHERE {
    {
      SELECT ?qleverui_entity (COUNT(?qleverui_entity) AS ?qleverui_count) WHERE {
        ?x <http://www.wikidata.org/prop/direct/P31>/<http://www.wikidata.org/prop/direct/P279>* ?qleverui_entity .
      }
      GROUP BY ?qleverui_entity
    }
    OPTIONAL {
      ?qleverui_entity @en@<http://www.w3.org/2000/01/rdf-schema#label> ?qleverui_name .
    }
    OPTIONAL {
      ?qleverui_entity @en@<http://www.w3.org/2004/02/skos/core#altLabel> ?qleverui_altname .
    }
}
GROUP BY ?qleverui_entity
ORDER BY DESC(?qleverui_count)
LIMIT 40
OFFSET 0

and receives following response:

{
"query": "PREFIX wdt: <http://www.wikidata.org/prop/direct/>\nSELECT ?qleverui_entity (SAMPLE(?qleverui_name) as ?qleverui_name) (SAMPLE(?qleverui_count) as ?qleverui_count) WHERE {\n    {\n      SELECT ?qleverui_entity (COUNT(?qleverui_entity) AS ?qleverui_count) WHERE {\n        ?x <http://www.wikidata.org/prop/direct/P31>/<http://www.wikidata.org/prop/direct/P279>* ?qleverui_entity .\n      }\n      GROUP BY ?qleverui_entity\n    }\n    OPTIONAL {\n      ?qleverui_entity @en@<http://www.w3.org/2000/01/rdf-schema#label> ?qleverui_name .\n    }\n    OPTIONAL {\n      ?qleverui_entity @en@<http://www.w3.org/2004/02/skos/core#altLabel> ?qleverui_altname .\n    }\n}\nGROUP BY ?qleverui_entity\nORDER BY DESC(?qleverui_count)\nLIMIT 40\nOFFSET 0",
"status": "ERROR",
"resultsize": "0",
"time": {
"total": "3801ms",
"computeResult": "3801ms"
},
"exception": "std::bad_alloc"
}
@hannahbast
Copy link
Member

I can confirm that the error already occurs for the following simpler query (= the query above without the optional joins and without the final ordering) and even with over 100 GB of RAM still free:

PREFIX wdt: <http://www.wikidata.org/prop/direct/>
SELECT ?o (COUNT(?o) AS ?count) WHERE {
  ?s wdt:P31/wdt:P279* ?o
}
GROUP BY ?o

The last lines written to the log before the crash are as follows:

Sun Jan 19 19:21:09.273	- DEBUG: TransitivePath subresult computation done.
Sun Jan 19 19:21:18.748	- DEBUG: TransitivePath result computation done.
Sun Jan 19 19:21:18.892	- DEBUG: Computing Join result...
Sun Jan 19 19:21:18.895	- DEBUG: Performing join between two tables.
Sun Jan 19 19:21:18.895	- DEBUG: A: width = 2, size = 72,175,797
Sun Jan 19 19:21:18.895	- DEBUG: B: width = 2, size = 62,050,972
Sun Jan 19 19:21:34.155	- DEBUG: Join done.
Sun Jan 19 19:21:34.155	- DEBUG: Result: width = 3, size = 1,419,740,509
Sun Jan 19 19:21:34.155	- DEBUG: Join result computation done.
Sun Jan 19 19:21:34.155	- DEBUG: Union subresult computation done.
Sun Jan 19 19:21:51.470	- DEBUG: Union result computation done.
Sun Jan 19 19:21:51.470	- DEBUG: OrderBy result computation...

The query above without the GROUP BY and the COUNT works and has a result size of 1,491,916,306 (similar, but not the same as in the log above, why?) and uses 60 - 70 GB. With 40 GB of RAM left, the sorting then fails.

@hannahbast
Copy link
Member

This is a very hard query with a huge result. QLever has a memory limit for quite some time now and the query above runs into it, now producing a proper error message of the form Tried to allocate XXX MB, but only YYY MB were available. Clear the cache or allow more memory for QLever during startup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants