Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug fix - Gaia-IR] The plan of pattern matching faces a multi-join error #1704

Merged
merged 68 commits into from
Jun 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
4ff2734
[dyn_type] Change the dependency from pegasus to pegasus_common to av…
longbinlai May 26, 2021
111a401
Merge branch 'alibaba:main' into main
longbinlai May 26, 2021
f69a6e4
Merge branch 'alibaba:main' into main
longbinlai May 27, 2021
c6afa75
Merge branch 'alibaba:main' into main
longbinlai May 27, 2021
6d53bd1
Merge branch 'alibaba:main' into main
longbinlai Jun 3, 2021
8d57c28
Merge branch 'alibaba:main' into main
longbinlai Jul 23, 2021
0befa2c
Merge branch 'alibaba:main' into main
longbinlai Aug 5, 2021
246962b
Merge branch 'alibaba:main' into main
longbinlai Sep 8, 2021
a457c85
Merge branch 'alibaba:main' into main
longbinlai Sep 15, 2021
95ba900
Merge branch 'alibaba:main' into main
longbinlai Sep 29, 2021
0b26524
Merge branch 'alibaba:main' into main
longbinlai Oct 12, 2021
ec8ddd9
Merge branch 'alibaba:main' into main
longbinlai Oct 14, 2021
15d11eb
Merge branch 'alibaba:main' into main
longbinlai Oct 20, 2021
3d45182
Merge branch 'alibaba:main' into main
longbinlai Nov 2, 2021
38a4d7b
Merge branch 'alibaba:main' into main
longbinlai Dec 3, 2021
bd073b1
Merge branch 'alibaba:main' into main
longbinlai Dec 3, 2021
d508f00
Merge branch 'alibaba:main' into main
longbinlai Dec 8, 2021
f913a4e
Merge branch 'alibaba:main' into main
longbinlai Dec 10, 2021
cc9893b
Merge branch 'alibaba:main' into main
longbinlai Dec 28, 2021
040db0c
Merge branch 'alibaba:main' into main
longbinlai Jan 25, 2022
efde3f1
Merge branch 'alibaba:main' into main
longbinlai Mar 9, 2022
61e94bf
Merge remote-tracking branch 'upstream/main' into main
BingqingLyu Mar 23, 2022
191cd21
Merge branch 'alibaba:main' into main
longbinlai Mar 31, 2022
7a8b879
Merge branch 'alibaba:main' into main
longbinlai Apr 7, 2022
7673fba
[IR Runtime] use KeyId for all tags in Runtime, and map tag_id back t…
BingqingLyu Apr 18, 2022
51952f1
[IR Core] pass tag id_name_mappings to Runtime in SinkOp
BingqingLyu Apr 18, 2022
c3e87f9
[CI Test] fix all tag related ci tests
BingqingLyu Apr 18, 2022
e106472
Merge pull request #134 from longbinlai/ir_tag
BingqingLyu Apr 18, 2022
1f24093
[CI Test] enable ci tests of gaia running on ubuntu20.04
BingqingLyu Apr 19, 2022
3a539b4
[CI Test] enable ci tests of gaia running on ubuntu-20.04
BingqingLyu Apr 19, 2022
8a9d1dc
[IR Runtime] support EdgeExpandIntersectionOperator in Runtime
BingqingLyu Apr 19, 2022
6bf0147
[IR Runtime] support UnfoldOperator in Runtime
BingqingLyu Apr 20, 2022
9193420
[CI Test] add ut for edge expand with intersection
BingqingLyu Apr 20, 2022
1553146
[IR Proto] update proto of Union
BingqingLyu Apr 20, 2022
d582661
[IR Core] translate union_opr of intersection into physical plan
BingqingLyu Apr 21, 2022
e22da16
[IR Core] consider repartition and auxilia in union_opr of intersection
BingqingLyu Apr 21, 2022
7718d16
refine the codes
BingqingLyu May 5, 2022
3ec2206
[IR Runtime] start_v in edge_expand of intersection cannot be None
BingqingLyu May 5, 2022
da0a77d
minor refine
BingqingLyu May 5, 2022
6081f2e
refine the codes
BingqingLyu May 6, 2022
c2872c1
[IR Core] Define an Intersect Operator individually as an algebra op
BingqingLyu May 6, 2022
9037adf
refine codes
BingqingLyu May 7, 2022
03763c4
refine expand_intersect
BingqingLyu May 7, 2022
8433ed3
refine expand_intersect
BingqingLyu May 9, 2022
72db2fb
[IR Runtime] refine errors
BingqingLyu May 9, 2022
91885d1
[IR runtime] Refine in unfold operator
BingqingLyu May 9, 2022
93a9d03
refine codes
BingqingLyu May 9, 2022
ff7a6d6
[IR Runtime] refine in expand_intersect and more ci tests
BingqingLyu May 9, 2022
0a7bd06
[IR Runtime] refine in unfold
BingqingLyu May 9, 2022
e073b3e
minor refine
BingqingLyu May 10, 2022
02e4cdf
Merge pull request #135 from longbinlai/ir_core_intersection
longbinlai May 11, 2022
3e6b380
[Gaia/IR] Reimplement the function of doing intersection.
longbinlai May 11, 2022
36910d4
[Gaia/IR] Allow duplication in `probing` array.
longbinlai May 11, 2022
841b2fd
Merge pull request #136 from longbinlai/ir_runtime_intersection
longbinlai May 11, 2022
6edf7fb
Revert "[IR Runtime] Support EdgeExpandWithIntersection and Unfold Op…
BingqingLyu May 19, 2022
642346d
Revert "[IR Core] Support Union of Intersection in IR-Core"
BingqingLyu May 19, 2022
9eded2c
Merge pull request #153 from longbinlai/revert-135-ir_core_intersection
BingqingLyu May 19, 2022
277c559
Merge pull request #152 from longbinlai/revert-136-ir_runtime_interse…
BingqingLyu May 19, 2022
89a17b6
Merge remote-tracking branch 'upstream/main' into main
BingqingLyu May 19, 2022
7107b34
[Data Loading Test] update endpoint of MaxNode
shirly121 May 20, 2022
3688d8e
Merge branch 'alibaba:main' into main
longbinlai May 20, 2022
8a74724
Merge branch 'alibaba:main' into main
longbinlai May 24, 2022
913a317
Merge branch 'alibaba:main' into main
longbinlai May 25, 2022
aec6592
Merge branch 'alibaba:main' into main
longbinlai Jun 2, 2022
6fb2a5c
Merge branch 'alibaba:main' into main
longbinlai Jun 9, 2022
8b3806d
[Gaia/IR] Fix a bug of processing multi-branch join in pattern matching.
longbinlai Jun 9, 2022
4c4e6c4
[IR Compiler] update expected results of unit tests with changes of i…
shirly121 Jun 10, 2022
9c9bd11
[IR Compiler] minor fix
shirly121 Jun 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions research/query_service/ir/common/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ fn codegen_inplace() -> Result<(), Box<dyn std::error::Error>> {

#[cfg(not(feature = "proto_inplace"))]
fn codegen_inplace() -> Result<(), Box<dyn std::error::Error>> {
println!("cargo:rerun-if-changed=../proto/common.proto");
println!("cargo:rerun-if-changed=../proto/expr.proto");
println!("cargo:rerun-if-changed=../proto/algebra.proto");
println!("cargo:rerun-if-changed=../proto/schema.proto");
println!("cargo:rerun-if-changed=../proto/results.proto");
prost_build::Config::new()
.type_attribute(".", "#[derive(Serialize,Deserialize)]")
.compile_protos(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,8 @@
},
"children": []
}
],
"roots": [
0
]
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,8 @@
},
"children": []
}
],
"roots": [
0
]
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -35,5 +35,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -69,5 +69,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -89,5 +89,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -53,5 +53,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,8 @@
},
"children": []
}
],
"roots": [
0
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -78,5 +78,8 @@
},
"children": []
}
],
"roots": [
0
]
}
4 changes: 1 addition & 3 deletions research/query_service/ir/core/src/plan/ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1012,9 +1012,7 @@ mod params {
if val.is_err() {
return val.err().unwrap();
}
params
.extra
.insert(key.unwrap(), val.unwrap());
params.extra.insert(key.unwrap(), val.unwrap());
std::mem::forget(params);

FfiError::success()
Expand Down
12 changes: 8 additions & 4 deletions research/query_service/ir/core/src/plan/logical.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,15 @@ impl TryFrom<pb::LogicalPlan> for LogicalPlan {
impl From<LogicalPlan> for pb::LogicalPlan {
fn from(plan: LogicalPlan) -> Self {
let mut id_map: HashMap<u32, i32> = HashMap::with_capacity(plan.len());
let mut roots = vec![];
// As there might be some nodes being removed, we gonna remap the nodes' ids
for (id, node) in plan.nodes.iter().enumerate() {
id_map.insert(node.0 as u32, id as i32);
for (new_id, (old_id, node)) in plan.nodes.iter().enumerate() {
id_map.insert(old_id as u32, new_id as i32);
if node.borrow().parents.is_empty() {
roots.push(new_id as i32);
}
}
let mut plan_pb = pb::LogicalPlan { nodes: vec![] };
let mut plan_pb = pb::LogicalPlan { nodes: vec![], roots };
for (_, node) in &plan.nodes {
let mut node_pb = pb::logical_plan::Node { opr: None, children: vec![] };
let mut operator = node.borrow().opr.clone();
Expand Down Expand Up @@ -1432,7 +1436,7 @@ mod test {
let root_pb = pb::logical_plan::Node { opr: Some(opr.clone()), children: vec![1, 2] };
let node1_pb = pb::logical_plan::Node { opr: Some(opr.clone()), children: vec![2] };
let node2_pb = pb::logical_plan::Node { opr: Some(opr.clone()), children: vec![] };
let plan_pb = pb::LogicalPlan { nodes: vec![root_pb, node1_pb, node2_pb] };
let plan_pb = pb::LogicalPlan { nodes: vec![root_pb, node1_pb, node2_pb], roots: vec![0] };

let plan = LogicalPlan::try_from(plan_pb).unwrap();
assert_eq!(plan.len(), 3);
Expand Down
Loading