Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: support hint for IndexHashJoin and IndexMergeJoin #13238

Merged
merged 19 commits into from Nov 13, 2019
Merged

planner: support hint for IndexHashJoin and IndexMergeJoin #13238

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
efa1b90
planner: support hint for IndexHashJoin and IndexMergeJoin
XuHuaiyu Nov 7, 2019
6ad9281
add test
XuHuaiyu Nov 7, 2019
c785fee
update go.mod
XuHuaiyu Nov 7, 2019
97fe931
Merge branch 'master' of https://github.com/pingcap/tidb into indexjo…
XuHuaiyu Nov 7, 2019
4adf5c5
go mod tidy
XuHuaiyu Nov 7, 2019
cedd970
test
XuHuaiyu Nov 7, 2019
089ca0f
fix ci
XuHuaiyu Nov 8, 2019
a947e9b
fix ci
XuHuaiyu Nov 8, 2019
4722902
Merge branch 'master' of https://github.com/pingcap/tidb into indexjo…
XuHuaiyu Nov 8, 2019
a027979
fix ci
XuHuaiyu Nov 8, 2019
2032605
fix ci
XuHuaiyu Nov 8, 2019
918098d
address comment
XuHuaiyu Nov 12, 2019
8665011
Merge branch 'master' of https://github.com/pingcap/tidb into indexjo…
XuHuaiyu Nov 12, 2019
33a738c
fix conflicts
XuHuaiyu Nov 12, 2019
2ae0b6a
address comment
XuHuaiyu Nov 12, 2019
283632f
address comment
XuHuaiyu Nov 12, 2019
06cc863
Merge branch 'master' of https://github.com/pingcap/tidb into indexjo…
XuHuaiyu Nov 13, 2019
8a800df
address comment
XuHuaiyu Nov 13, 2019
0c6b02f
Merge branch 'master' into indexjoin_hint
ngaut Nov 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 9 additions & 8 deletions cmd/explaintest/r/explain_complex.result
View file
Open in desktop
Expand Up @@ -152,14 +152,15 @@ explain select dt.id as id, dt.aid as aid, dt.pt as pt, dt.dic as dic, dt.cm as
id count task operator info
Projection_10 0.00 root Column#1, Column#2, Column#4, Column#5, Column#3, Column#24, Column#25, Column#26, Column#11, Column#12, Column#13, Column#14, Column#15, Column#16, Column#17
└─Limit_13 0.00 root offset:0, count:2000
└─IndexJoin_19 0.00 root inner join, inner:IndexLookUp_18, outer key:Column#2, Column#5, inner key:Column#21, Column#23
├─TableReader_43 0.00 root data:Selection_42
│ └─Selection_42 0.00 cop[tikv] eq(Column#18, 0), eq(Column#4, "ios"), gt(Column#9, 1478185592), not(isnull(Column#5))
│ └─TableScan_41 10000.00 cop[tikv] table:dt, range:[0,+inf], keep order:false, stats:pseudo
└─IndexLookUp_18 0.00 root
├─IndexScan_15 1.25 cop[tikv] table:rr, index:aid, dic, range: decided by [eq(Column#21, Column#2) eq(Column#23, Column#5)], keep order:false, stats:pseudo
└─Selection_17 0.00 cop[tikv] eq(Column#22, "ios"), gt(Column#26, 1478185592)
└─TableScan_16 1.25 cop[tikv] table:rr, keep order:false, stats:pseudo
└─IndexMergeJoin_41 0.00 root inner join, inner:IndexLookUp_39, outer key:Column#21, Column#23, inner key:Column#2, Column#5
lzmhhh123 marked this conversation as resolved.
Show resolved Hide resolved
├─IndexLookUp_39 0.00 root
│ ├─Selection_37 0.00 cop[tikv] not(isnull(Column#5))
│ │ └─IndexScan_35 0.00 cop[tikv] table:dt, index:aid, dic, range: decided by [eq(Column#2, Column#21) eq(Column#5, Column#23)], keep order:true, stats:pseudo
│ └─Selection_38 0.00 cop[tikv] eq(Column#18, 0), eq(Column#4, "ios"), gt(Column#9, 1478185592)
│ └─TableScan_36 0.00 cop[tikv] table:dt, keep order:false, stats:pseudo
└─TableReader_61 3.33 root data:Selection_60
└─Selection_60 3.33 cop[tikv] eq(Column#22, "ios"), gt(Column#26, 1478185592)
└─TableScan_59 10000.00 cop[tikv] table:rr, range:[-inf,+inf], keep order:false, stats:pseudo
explain select pc,cr,count(DISTINCT uid) as pay_users,count(oid) as pay_times,sum(am) as am from pp where ps=2 and ppt>=1478188800 and ppt<1478275200 and pi in ('510017','520017') and uid in ('18089709','18090780') group by pc,cr;
id count task operator info
Projection_5 1.00 root Column#14, Column#16, Column#22, Column#23, Column#24
Expand Down
17 changes: 9 additions & 8 deletions cmd/explaintest/r/explain_complex_stats.result
View file
Open in desktop
Expand Up @@ -160,14 +160,15 @@ explain select dt.id as id, dt.aid as aid, dt.pt as pt, dt.dic as dic, dt.cm as
id count task operator info
Projection_10 428.32 root Column#1, Column#2, Column#4, Column#5, Column#3, Column#24, Column#25, Column#26, Column#11, Column#12, Column#13, Column#14, Column#15, Column#16, Column#17
└─Limit_13 428.32 root offset:0, count:2000
└─IndexJoin_19 428.32 root inner join, inner:IndexLookUp_18, outer key:Column#2, Column#5, inner key:Column#21, Column#23
├─TableReader_43 428.32 root data:Selection_42
│ └─Selection_42 428.32 cop[tikv] eq(Column#18, 0), eq(Column#4, "ios"), gt(Column#9, 1478185592), not(isnull(Column#5))
│ └─TableScan_41 2000.00 cop[tikv] table:dt, range:[0,+inf], keep order:false
└─IndexLookUp_18 0.48 root
├─IndexScan_15 1.00 cop[tikv] table:rr, index:aid, dic, range: decided by [eq(Column#21, Column#2) eq(Column#23, Column#5)], keep order:false
└─Selection_17 0.48 cop[tikv] eq(Column#22, "ios"), gt(Column#26, 1478185592)
└─TableScan_16 1.00 cop[tikv] table:rr, keep order:false
└─IndexMergeJoin_41 428.32 root inner join, inner:IndexLookUp_39, outer key:Column#21, Column#23, inner key:Column#2, Column#5
├─IndexLookUp_39 0.09 root
│ ├─Selection_37 0.44 cop[tikv] not(isnull(Column#5))
│ │ └─IndexScan_35 0.44 cop[tikv] table:dt, index:aid, dic, range: decided by [eq(Column#2, Column#21) eq(Column#5, Column#23)], keep order:true
│ └─Selection_38 0.09 cop[tikv] eq(Column#18, 0), eq(Column#4, "ios"), gt(Column#9, 1478185592)
│ └─TableScan_36 0.44 cop[tikv] table:dt, keep order:false
└─TableReader_61 970.00 root data:Selection_60
└─Selection_60 970.00 cop[tikv] eq(Column#22, "ios"), gt(Column#26, 1478185592)
└─TableScan_59 2000.00 cop[tikv] table:rr, range:[-inf,+inf], keep order:false
explain select pc,cr,count(DISTINCT uid) as pay_users,count(oid) as pay_times,sum(am) as am from pp where ps=2 and ppt>=1478188800 and ppt<1478275200 and pi in ('510017','520017') and uid in ('18089709','18090780') group by pc,cr;
id count task operator info
Projection_5 207.86 root Column#14, Column#16, Column#22, Column#23, Column#24
Expand Down
14 changes: 7 additions & 7 deletions cmd/explaintest/r/explain_easy.result
View file
Open in desktop
Expand Up @@ -350,13 +350,13 @@ Projection_11 5.00 root Column#12
├─TableReader_15 5.00 root data:TableScan_14
│ └─TableScan_14 5.00 cop[tikv] table:t, range:[-inf,+inf], keep order:false
└─StreamAgg_20 1.00 root funcs:count(1)
└─MergeJoin_51 2.40 root inner join, left key:Column#4, right key:Column#7
├─IndexReader_38 2.40 root index:Selection_37
│ └─Selection_37 2.40 cop[tikv] eq(3, Column#1)
│ └─IndexScan_36 3.00 cop[tikv] table:s, index:b, range:[3,3], keep order:true
└─TableReader_41 4.00 root data:Selection_40
└─Selection_40 4.00 cop[tikv] eq(3, Column#1)
└─TableScan_39 5.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:true
└─MergeJoin_62 2.40 root inner join, left key:Column#4, right key:Column#7
├─IndexReader_49 2.40 root index:Selection_48
│ └─Selection_48 2.40 cop[tikv] eq(3, Column#1)
│ └─IndexScan_47 3.00 cop[tikv] table:s, index:b, range:[3,3], keep order:true
└─TableReader_52 4.00 root data:Selection_51
└─Selection_51 4.00 cop[tikv] eq(3, Column#1)
└─TableScan_50 5.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:true
explain select t.c in (select count(*) from t s left join t t1 on s.a = t1.a where 3 = t.a and s.b = 3) from t;
id count task operator info
Projection_10 5.00 root Column#12
Expand Down
22 changes: 11 additions & 11 deletions cmd/explaintest/r/generated_columns.result
View file
Open in desktop
Expand Up @@ -71,24 +71,24 @@ VALUES ('{"a": 1}', '{"1": "1"}');
ANALYZE TABLE sgc1, sgc2;
EXPLAIN SELECT /*+ TIDB_INLJ(sgc1, sgc2) */ * from sgc1 join sgc2 on sgc1.a=sgc2.a;
id count task operator info
IndexHashJoin_35 5.00 root inner join, inner:IndexLookUp_25, outer key:Column#8, inner key:Column#3
IndexJoin_26 5.00 root inner join, inner:IndexLookUp_25, outer key:Column#8, inner key:Column#3
├─IndexLookUp_25 5.00 root
│ ├─Selection_24 5.00 cop[tikv] not(isnull(Column#3))
│ │ └─IndexScan_22 5.00 cop[tikv] table:sgc1, index:a, range: decided by [eq(Column#3, Column#8)], keep order:false
│ └─TableScan_23 5.00 cop[tikv] table:sgc1, keep order:false
└─TableReader_38 1.00 root data:Selection_37
└─Selection_37 1.00 cop[tikv] not(isnull(Column#8))
└─TableScan_36 1.00 cop[tikv] table:sgc2, range:[-inf,+inf], keep order:false
└─TableReader_47 1.00 root data:Selection_46
└─Selection_46 1.00 cop[tikv] not(isnull(Column#8))
└─TableScan_45 1.00 cop[tikv] table:sgc2, range:[-inf,+inf], keep order:false
EXPLAIN SELECT * from sgc1 join sgc2 on sgc1.a=sgc2.a;
id count task operator info
Projection_6 5.00 root Column#1, Column#2, Column#3, Column#4, Column#6, Column#7, Column#8, Column#9
└─HashRightJoin_24 5.00 root inner join, inner:TableReader_43, equal:[eq(Column#8, Column#3)]
├─TableReader_43 1.00 root data:Selection_42
│ └─Selection_42 1.00 cop[tikv] not(isnull(Column#8))
│ └─TableScan_41 1.00 cop[tikv] table:sgc2, range:[-inf,+inf], keep order:false
└─TableReader_52 5.00 root data:Selection_51
└─Selection_51 5.00 cop[tikv] not(isnull(Column#3))
└─TableScan_50 5.00 cop[tikv] table:sgc1, range:[-inf,+inf], keep order:false
└─HashRightJoin_38 5.00 root inner join, inner:TableReader_57, equal:[eq(Column#8, Column#3)]
├─TableReader_57 1.00 root data:Selection_56
│ └─Selection_56 1.00 cop[tikv] not(isnull(Column#8))
│ └─TableScan_55 1.00 cop[tikv] table:sgc2, range:[-inf,+inf], keep order:false
└─TableReader_66 5.00 root data:Selection_65
└─Selection_65 5.00 cop[tikv] not(isnull(Column#3))
└─TableScan_64 5.00 cop[tikv] table:sgc1, range:[-inf,+inf], keep order:false
DROP TABLE IF EXISTS sgc3;
CREATE TABLE sgc3 (
j JSON,
Expand Down
28 changes: 14 additions & 14 deletions cmd/explaintest/r/index_join.result
View file
Open in desktop
Expand Up @@ -8,35 +8,35 @@ set session tidb_hashagg_partial_concurrency = 1;
set session tidb_hashagg_final_concurrency = 1;
explain select /*+ TIDB_INLJ(t1, t2) */ * from t1 join t2 on t1.a=t2.a;
id count task operator info
IndexHashJoin_34 5.00 root inner join, inner:IndexLookUp_24, outer key:Column#4, inner key:Column#1
IndexJoin_25 5.00 root inner join, inner:IndexLookUp_24, outer key:Column#4, inner key:Column#1
├─IndexLookUp_24 5.00 root
│ ├─Selection_23 5.00 cop[tikv] not(isnull(Column#1))
│ │ └─IndexScan_21 5.00 cop[tikv] table:t1, index:a, range: decided by [eq(Column#1, Column#4)], keep order:false
│ └─TableScan_22 5.00 cop[tikv] table:t1, keep order:false
└─TableReader_37 1.00 root data:Selection_36
└─Selection_36 1.00 cop[tikv] not(isnull(Column#4))
└─TableScan_35 1.00 cop[tikv] table:t2, range:[-inf,+inf], keep order:false
└─TableReader_43 1.00 root data:Selection_42
└─Selection_42 1.00 cop[tikv] not(isnull(Column#4))
└─TableScan_41 1.00 cop[tikv] table:t2, range:[-inf,+inf], keep order:false
explain select * from t1 join t2 on t1.a=t2.a;
id count task operator info
Projection_6 5.00 root Column#1, Column#2, Column#4, Column#5
└─HashRightJoin_23 5.00 root inner join, inner:TableReader_34, equal:[eq(Column#4, Column#1)]
├─TableReader_34 1.00 root data:Selection_33
│ └─Selection_33 1.00 cop[tikv] not(isnull(Column#4))
│ └─TableScan_32 1.00 cop[tikv] table:t2, range:[-inf,+inf], keep order:false
└─TableReader_40 5.00 root data:Selection_39
└─Selection_39 5.00 cop[tikv] not(isnull(Column#1))
└─TableScan_38 5.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:false
└─HashRightJoin_37 5.00 root inner join, inner:TableReader_48, equal:[eq(Column#4, Column#1)]
├─TableReader_48 1.00 root data:Selection_47
│ └─Selection_47 1.00 cop[tikv] not(isnull(Column#4))
│ └─TableScan_46 1.00 cop[tikv] table:t2, range:[-inf,+inf], keep order:false
└─TableReader_54 5.00 root data:Selection_53
└─Selection_53 5.00 cop[tikv] not(isnull(Column#1))
└─TableScan_52 5.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:false
drop table if exists t1, t2;
create table t1(a int not null, b int not null);
create table t2(a int not null, b int not null, key a(a));
set @@tidb_opt_insubq_to_join_and_agg=0;
explain select /*+ TIDB_INLJ(t2@sel_2) */ * from t1 where t1.a in (select t2.a from t2);
id count task operator info
IndexMergeJoin_14 8000.00 root semi join, inner:IndexReader_12, outer key:Column#1, inner key:Column#4
IndexJoin_10 8000.00 root semi join, inner:IndexReader_9, outer key:Column#1, inner key:Column#4
├─TableReader_18 10000.00 root data:TableScan_17
│ └─TableScan_17 10000.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─IndexReader_12 1.25 root index:IndexScan_11
└─IndexScan_11 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#4, Column#1)], keep order:true, stats:pseudo
└─IndexReader_9 1.25 root index:IndexScan_8
└─IndexScan_8 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#4, Column#1)], keep order:false, stats:pseudo
show warnings;
Level Code Message
set @@tidb_opt_insubq_to_join_and_agg=1;
Expand Down
22 changes: 11 additions & 11 deletions cmd/explaintest/r/topn_push_down.result
View file
Open in desktop
Expand Up @@ -170,11 +170,11 @@ id count task operator info
Projection_12 0.00 root Column#47
└─TopN_15 0.00 root Column#47:asc, offset:0, count:5
└─IndexJoin_24 0.00 root inner join, inner:IndexLookUp_23, outer key:Column#1, inner key:Column#41
├─IndexLookUp_67 0.00 root
│ ├─Selection_65 0.00 cop[tikv] eq(Column#4, 18), eq(Column#5, 1)
│ │ └─IndexScan_63 10.00 cop[tikv] table:tr, index:shop_identy, trade_status, business_type, trade_pay_status, trade_type, delivery_type, source, biz_date, range:[810094178,810094178], keep order:false, stats:pseudo
│ └─Selection_66 0.00 cop[tikv] eq(Column#21, 32314), eq(Column#3, 2)
│ └─TableScan_64 0.00 cop[tikv] table:tr, keep order:false, stats:pseudo
├─IndexLookUp_78 0.00 root
│ ├─Selection_76 0.00 cop[tikv] eq(Column#4, 18), eq(Column#5, 1)
│ │ └─IndexScan_74 10.00 cop[tikv] table:tr, index:shop_identy, trade_status, business_type, trade_pay_status, trade_type, delivery_type, source, biz_date, range:[810094178,810094178], keep order:false, stats:pseudo
│ └─Selection_77 0.00 cop[tikv] eq(Column#21, 32314), eq(Column#3, 2)
│ └─TableScan_75 0.00 cop[tikv] table:tr, keep order:false, stats:pseudo
└─IndexLookUp_23 0.03 root
├─IndexScan_20 1.25 cop[tikv] table:te, index:trade_id, range: decided by [eq(Column#41, Column#1)], keep order:false, stats:pseudo
└─Selection_22 0.03 cop[tikv] ge(Column#47, 2018-04-23 00:00:00.000000), le(Column#47, 2018-04-23 23:59:59.000000)
Expand Down Expand Up @@ -217,20 +217,20 @@ create table t(a int not null, index idx(a));
explain select /*+ TIDB_INLJ(t2) */ * from t t1 join t t2 on t1.a = t2.a limit 5;
id count task operator info
Limit_11 5.00 root offset:0, count:5
└─IndexMergeJoin_19 5.00 root inner join, inner:IndexReader_17, outer key:Column#1, inner key:Column#3
└─IndexJoin_15 5.00 root inner join, inner:IndexReader_14, outer key:Column#1, inner key:Column#3
├─TableReader_23 4.00 root data:TableScan_22
│ └─TableScan_22 4.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─IndexReader_17 1.25 root index:IndexScan_16
└─IndexScan_16 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#3, Column#1)], keep order:true, stats:pseudo
└─IndexReader_14 1.25 root index:IndexScan_13
└─IndexScan_13 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#3, Column#1)], keep order:false, stats:pseudo
explain select /*+ TIDB_INLJ(t2) */ * from t t1 left join t t2 on t1.a = t2.a where t2.a is null limit 5;
id count task operator info
Limit_12 5.00 root offset:0, count:5
└─Selection_13 5.00 root isnull(Column#3)
└─IndexMergeJoin_21 5.00 root left outer join, inner:IndexReader_19, outer key:Column#1, inner key:Column#3
└─IndexJoin_17 5.00 root left outer join, inner:IndexReader_16, outer key:Column#1, inner key:Column#3
├─TableReader_25 4.00 root data:TableScan_24
│ └─TableScan_24 4.00 cop[tikv] table:t1, range:[-inf,+inf], keep order:false, stats:pseudo
└─IndexReader_19 1.25 root index:IndexScan_18
└─IndexScan_18 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#3, Column#1)], keep order:true, stats:pseudo
└─IndexReader_16 1.25 root index:IndexScan_15
└─IndexScan_15 1.25 cop[tikv] table:t2, index:a, range: decided by [eq(Column#3, Column#1)], keep order:false, stats:pseudo
explain select /*+ TIDB_SMJ(t1, t2) */ * from t t1 join t t2 on t1.a = t2.a limit 5;
id count task operator info
Limit_11 5.00 root offset:0, count:5
Expand Down