Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely low perfomance of query with many relations #6409

Closed
DmitriyMgar opened this issue Jul 23, 2021 · 7 comments
Closed

Extremely low perfomance of query with many relations #6409

DmitriyMgar opened this issue Jul 23, 2021 · 7 comments

Comments

@DmitriyMgar
Copy link

Description

Query to TypeDB involving many relations takes extremly long time.

This is theschema with approximate number of instances:
2021-05-18_18-51-25 (2)

Environment

  1. OS (where TypeDB server runs): Ubuntu 20.04.2 LTS
  2. TypeDB version (and platform): TypeDB 2.2.0
  3. TypeDB client: console
  4. Other environment details: RAM 15.5GB, Processor Inter Core i5-7300HQ CPU 2.50GHz*4

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Run TypeDB server
  2. Write schema, load data
  3. Query -
    match (employee:$person,employer:$company,offered_position:$position) isa employment,has employment_number $employment_number,has employment_start_date $employment_start_date,has employment_end_date $employment_end_date,has employment_status $employment_status; (member:$person,union:$department) isa membership; (superior_company:$company,subordinate_department:$department) isa company_hierarchy; $person isa person ,has person_id $person_id,has person_gender $person_gender,has person_age $person_age,has person_birthyear $person_birthyear; $company isa company ,has company_id $company_id,has company_name $company_name; $position isa position ,has position_id $position_id,has position_name $position_name,has position_function $position_function,has position_upc $position_upc; $department isa department ,has department_id $department_id,has department_name $department_name; {$person_gender='male';};{$company_name='facebook';}; get $employment_number, $employment_start_date, $employment_end_date, $employment_status, $person_id, $person_gender, $person_age, $person_birthyear, $company_id, $company_name, $position_id, $position_name, $position_function, $position_upc, $department_id, $department_name;
  4. Wait for the response for 40 minutes

Expected Output

We have been expecting to get response mutch more faster.

Actual Output

We got response in about 40 minutes

Additional information

We've got something like this during the process of query
image_2021_07_21T07_06_13_392Z

@haikalpribadi
Copy link
Member

This looks like a familiar issue that we've uncovered, @DmitriyMgar: when the query is long, the "type resolver" performs poorly in verifying the query is valid. We already have a solution to solve and it's documented here: #6194 . @flyingsilverfin is currently working on this issue, so you can expect this to be resolved soon.

@flyingsilverfin
Copy link
Member

@haikalpribadi i think this is actually not type resolver - if you look at the above procedure it's not over the schema, it's over the data -- the type resolver procedure that is printed should have mapped the has to owns for instance

@flyingsilverfin
Copy link
Member

@DmitriyMgar do you have the schema you can share with us to reproduce this issue as we test fixes for the issue?

@DmitriyMgar
Copy link
Author

@flyingsilverfin this is our schema - schema.txt

By the way, we have noticed a great enhancment in response time after reducing the number of attributes in the get clause.
For example, query like this takes about 95 seconds:
match (employee:$person,employer:$company,offered_position:$position) isa employment;(member:$person,union:$department) isa membership;(superior_company:$company,subordinate_department:$department) isa company_hierarchy; $person isa person ,has person_gender $person_gender; $company isa company ,has company_name $company_name; $position isa position; $department isa department ,has department_name $department_name;{$person_gender='male';};{$company_name='facebook';};get $department_name, $person ,$company ,$position ,$department;
But if we add another three attributes: "$person_gender, $company_name, $position_function" to the get clause, it will takes about 320 seconds.

@flyingsilverfin
Copy link
Member

Okay cool! Thanks for that. We're working on improving our static type checker that runs on every query, which is what causes your slowdowns (even 95 seconds sounds slow). Should be out in the next release!

@DmitriyMgar
Copy link
Author

Thanks a lot! Looking forward to the next release :)

@flyingsilverfin
Copy link
Member

Solved with #6431 !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants