Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(interactive): Support LDBC SNB IC1,4,7,10 for HQPS Engine #3147

Merged
merged 6 commits into from
Sep 19, 2023

Conversation

zhanglei1949
Copy link
Collaborator

What do these changes do?

Support for IC1,4,7,10.

Related issue number

Fixes

@zhanglei1949 zhanglei1949 marked this pull request as draft August 28, 2023 02:05
@zhanglei1949 zhanglei1949 force-pushed the support-more-ic branch 2 times, most recently from f22a8c1 to c637736 Compare September 1, 2023 09:53
@zhanglei1949 zhanglei1949 force-pushed the support-more-ic branch 2 times, most recently from fe7509b to 127acc8 Compare September 15, 2023 03:58
@zhanglei1949 zhanglei1949 changed the title Support more ic feat(interactive): Support LDBC SNB IC1,4,7,10 for HQPS Engine Sep 15, 2023
@codecov-commenter
Copy link

codecov-commenter commented Sep 16, 2023

Codecov Report

Merging #3147 (3fd32a4) into main (11d6ae6) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3147   +/-   ##
=======================================
  Coverage   42.05%   42.05%           
=======================================
  Files         101      101           
  Lines       10985    10985           
=======================================
  Hits         4620     4620           
  Misses       6365     6365           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 11d6ae6...3fd32a4. Read the comment docs.

<!--
Thanks for your contribution! please review
https://github.com/alibaba/GraphScope/blob/main/CONTRIBUTING.md before
opening an issue.
-->

<!-- Please give a short brief about these changes. -->

As titled. We support to project property of a `path`, i.e., we project
the property of each element in `path`.

e.g., on modern graph:
```
gremlin> g.V().out("1..3", "knows").with('RESULT_OPT', 'ALL_V').values("name")
==>[marko, vadas]
==>[marko, josh]
```

<!-- Are there any issues opened that will be resolved by merging this
change? -->

Fixes alibaba#3199

test for multi-match logical plan and physical plan

Committed-by: bingqing.lbq from Dev container

[Fix]
1. Fix the bug of multiple pattern matching
2. Add more test cases for pattern matching

[Test]
1. Edit testcases
2. Fix some bugs
3. Now the new multiple match implementation with dummy source can pass all the tests in the rust side

minor: test for multi-source-multi-match logical plan and physical plan

[CI Tests] more multi source related ci tests

[Bug Fix]
1. Edit methods of get subplans and get branch plans, now it can support nested branch plans
2. Introduce a new Branch Logical Operator for helping handle branches

[Bug Fix and Testcases]
1. Fix bugs in `get_merge_node` and `get_branch_node` method when facing more comlicated logical plan, now it will uses more strict condition for final merge/first branch checking
2. Fix a bug in `append_branch_plan`, when branch plan having a node with id already exist in the original plan, now it will not cover the existing node, but it will now merge the children of the two nodes. It solves the problem in finding subplans that two branch nodes overlap
3. Add more test cases to validate the new `subplan` and `get_branch_plans` method

[Bug Fix and Testcases]

1. Add 6 types logical plans for testing
1.1 Verify the correctness `get_merge_node`, `get_branch_node`, `subplan`, `get_branch_plans` with these logical plans
1.2 Verify whether these logical plan can converted to expected physical plan
1.3 Verify whether the query based on these logical plans can generate expected results

2. Use flow algorithm in `get_merge_node` and `get_branch_node`, greatly improve the efficiency
2.1 Implement a Fraction abstraction for the precision during flow comparison

3. Solve a problem in `append_branch_plans`, now it supports to append branch plans which having nodes alreading existing in the original plan

4. Completely rewrite the logic of `get_merge_node`, `get_branch_node`, `subplan`, `get_branch_plans
4.1 Now it in theory can support arbitrary DAG logical plan as long as it is reasonable

5. Solve a problem in `extract_subplans`, now the last node of a subplan can be a merge node

6. Make the code much safer that now it will check the existence of merge node subplans much more strictly
6.1 Remove `expect`, instead throws None or Error

7. Add more comments which suggests the logic for handling nested branches

import a third-party fraction lib to replace the self-written one

[GIE Compiler] support optional match in compiler

add FilterIntoJoin Rule

[GIE Compiler] support option match to left outer join

[GIE Compiler] add 'NotExistToAntiJoinRule' to convert not exist subquery to anti join

[GIE Compiler] fix bugs in ic queries

[GIE Compiler] fix unit tests

[GIE Compiler] support IS_NULL and IS_NOT_NULL in cypher queries

[GIE Compiler] minor fix

[GIE Compiler] minor fix

[GIE Compiler] remove IS_NULL

[GIE Compiler] remove ffi build unit test from compiler

[GIE Compiler] add doc

[GIE Compiler] support IS_NULL and IS_NOT_NULL in cypher queries

[GIE Compiler] minor fix

[GIE Compiler] support ListLiteral in cypher queries

[IR Core] add First in FfiAggOpt

[IR Runtime] support VarMap in Project

[GIE Compiler] support types : 'GraphPathType' and 'List<Any>'

[GIE Compiler] minor fix

[GIE Compiler] support extract operator in compiler

[GIE Compiler] fix bugs

[GIE Compiler] minor fix

add support for ic1

todo ic10

fix ic10 support

modify ic1 cypher

stash

todo: fix correctness

fix ic10

fix

rebase main

fixing rebase

todo: add e2d ci

adding 4,7,10 tet

check ci

format and refactor

format

todo: reset gstest

some refactor and format

disable ic8 test

fixup

fixup
@zhanglei1949 zhanglei1949 marked this pull request as ready for review September 18, 2023 05:10
luoxiaojian
luoxiaojian previously approved these changes Sep 18, 2023
@zhanglei1949 zhanglei1949 merged commit 83d5a26 into alibaba:main Sep 19, 2023
47 of 48 checks passed
@zhanglei1949 zhanglei1949 deleted the support-more-ic branch March 14, 2024 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants