Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: make IndexHashJoin support keeping the outer order #12349

Merged
merged 26 commits into from Oct 17, 2019
Merged

executor: make IndexHashJoin support keeping the outer order #12349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b1d6d7e
executor: support keeping the outer order in IndexHashJoin
XuHuaiyu Sep 24, 2019
c743eb9
refine
XuHuaiyu Sep 24, 2019
8aebe91
fix ci
XuHuaiyu Sep 24, 2019
1ded238
Merge branch 'master' of https://github.com/pingcap/tidb into indexha…
XuHuaiyu Sep 25, 2019
845f36f
refine
XuHuaiyu Sep 25, 2019
e5198eb
fix ci
XuHuaiyu Sep 25, 2019
2d566ef
Merge branch 'master' of https://github.com/pingcap/tidb into indexha…
XuHuaiyu Sep 25, 2019
4c6a7c6
for test
XuHuaiyu Sep 25, 2019
76697a7
fix explain test
XuHuaiyu Sep 25, 2019
b52e4ab
test
XuHuaiyu Sep 26, 2019
76f2181
fix ci
XuHuaiyu Sep 27, 2019
e41fd6f
address comments
XuHuaiyu Oct 10, 2019
fe3dc7f
address comments
XuHuaiyu Oct 11, 2019
ff5a6f5
address comments
XuHuaiyu Oct 11, 2019
feba35e
address comment
XuHuaiyu Oct 15, 2019
160bfba
Merge branch 'master' of https://github.com/pingcap/tidb into indexha…
XuHuaiyu Oct 15, 2019
a1a1935
fix ci
XuHuaiyu Oct 16, 2019
f69c0d8
fix ci
XuHuaiyu Oct 16, 2019
42dcfff
address comment
XuHuaiyu Oct 17, 2019
9f81815
Merge branch 'master' of https://github.com/pingcap/tidb into indexha…
XuHuaiyu Oct 17, 2019
f027606
address comment
XuHuaiyu Oct 17, 2019
88fb6a0
address comment
XuHuaiyu Oct 17, 2019
da86fdf
format
XuHuaiyu Oct 17, 2019
c1d74e3
Merge branch 'master' into indexhashjoin_order
XuHuaiyu Oct 17, 2019
55e086b
Merge branch 'master' into indexhashjoin_order
XuHuaiyu Oct 17, 2019
fee4974
Merge branch 'master' into indexhashjoin_order
sre-bot Oct 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 5 additions & 5 deletions cmd/explaintest/r/topn_push_down.result
View file
Open in desktop
Expand Up @@ -169,8 +169,8 @@ LIMIT 0, 5;
id count task operator info
Projection_13 0.00 root Column#47
└─Limit_19 0.00 root offset:0, count:5
└─IndexJoin_104 0.00 root left outer join, inner:IndexReader_103, outer key:Column#1, inner key:Column#97
├─TopN_111 0.00 root Column#47:asc, offset:0, count:5
└─IndexJoin_106 0.00 root left outer join, inner:IndexReader_105, outer key:Column#1, inner key:Column#97
├─TopN_115 0.00 root Column#47:asc, offset:0, count:5
│ └─IndexJoin_43 0.00 root inner join, inner:IndexLookUp_42, outer key:Column#1, inner key:Column#41
│ ├─IndexLookUp_86 0.00 root
│ │ ├─Selection_84 0.00 cop[tikv] eq(Column#4, 18), eq(Column#5, 1)
Expand All @@ -181,9 +181,9 @@ Projection_13 0.00 root Column#47
│ ├─IndexScan_39 1.25 cop[tikv] table:te, index:trade_id, range: decided by [eq(Column#41, Column#1)], keep order:false, stats:pseudo
│ └─Selection_41 0.03 cop[tikv] ge(Column#47, 2018-04-23 00:00:00.000000), le(Column#47, 2018-04-23 23:59:59.000000)
│ └─TableScan_40 1.25 cop[tikv] table:te, keep order:false, stats:pseudo
└─IndexReader_103 1.25 root index:Selection_102
└─Selection_102 1.25 cop[tikv] not(isnull(Column#97))
└─IndexScan_101 1.25 cop[tikv] table:p, index:relate_id, range: decided by [eq(Column#97, Column#1)], keep order:false, stats:pseudo
└─IndexReader_105 1.25 root index:Selection_104
└─Selection_104 1.25 cop[tikv] not(isnull(Column#97))
└─IndexScan_103 1.25 cop[tikv] table:p, index:relate_id, range: decided by [eq(Column#97, Column#1)], keep order:false, stats:pseudo
desc select 1 as a from dual order by a limit 1;
id count task operator info
Projection_6 1.00 root 1
Expand Down
88 changes: 44 additions & 44 deletions cmd/explaintest/r/tpch.result
View file
Open in desktop
Expand Up @@ -252,15 +252,15 @@ id count task operator info
Projection_14 10.00 root Column#18, Column#35, Column#13, Column#16
└─TopN_17 10.00 root Column#35:desc, Column#13:asc, offset:0, count:10
└─HashAgg_23 40252367.98 root group by:Column#49, Column#50, Column#51, funcs:sum(Column#45), firstrow(Column#46), firstrow(Column#47), firstrow(Column#48)
└─Projection_79 91515927.49 root mul(Column#23, minus(1, Column#24)), Column#13, Column#16, Column#18, Column#18, Column#13, Column#16
└─Projection_81 91515927.49 root mul(Column#23, minus(1, Column#24)), Column#13, Column#16, Column#18, Column#18, Column#13, Column#16
└─IndexHashJoin_38 91515927.49 root inner join, inner:IndexLookUp_28, outer key:Column#9, inner key:Column#18
├─HashRightJoin_69 22592975.51 root inner join, inner:TableReader_75, equal:[eq(Column#1, Column#10)]
│ ├─TableReader_75 1498236.00 root data:Selection_74
│ │ └─Selection_74 1498236.00 cop[tikv] eq(Column#7, "AUTOMOBILE")
│ │ └─TableScan_73 7500000.00 cop[tikv] table:customer, range:[-inf,+inf], keep order:false
│ └─TableReader_72 36870000.00 root data:Selection_71
│ └─Selection_71 36870000.00 cop[tikv] lt(Column#13, 1995-03-13 00:00:00.000000)
│ └─TableScan_70 75000000.00 cop[tikv] table:orders, range:[-inf,+inf], keep order:false
├─HashRightJoin_71 22592975.51 root inner join, inner:TableReader_77, equal:[eq(Column#1, Column#10)]
│ ├─TableReader_77 1498236.00 root data:Selection_76
│ │ └─Selection_76 1498236.00 cop[tikv] eq(Column#7, "AUTOMOBILE")
│ │ └─TableScan_75 7500000.00 cop[tikv] table:customer, range:[-inf,+inf], keep order:false
│ └─TableReader_74 36870000.00 root data:Selection_73
│ └─Selection_73 36870000.00 cop[tikv] lt(Column#13, 1995-03-13 00:00:00.000000)
│ └─TableScan_72 75000000.00 cop[tikv] table:orders, range:[-inf,+inf], keep order:false
└─IndexLookUp_28 2.20 root
├─IndexScan_25 4.05 cop[tikv] table:lineitem, index:L_ORDERKEY, L_LINENUMBER, range: decided by [eq(Column#18, Column#9)], keep order:false
└─Selection_27 2.20 cop[tikv] gt(Column#28, 1995-03-13 00:00:00.000000)
Expand Down Expand Up @@ -596,24 +596,24 @@ Sort_25 2406.00 root Column#57:asc, Column#58:desc
└─Projection_27 2406.00 root Column#53, Column#54, Column#56
└─HashAgg_30 2406.00 root group by:Column#53, Column#54, funcs:sum(Column#55), firstrow(Column#53), firstrow(Column#54)
└─Projection_31 971049283.51 root Column#50, extract("YEAR", Column#44), minus(mul(Column#22, minus(1, Column#23)), mul(Column#37, Column#21))
└─HashLeftJoin_44 971049283.51 root inner join, inner:TableReader_104, equal:[eq(Column#19, Column#35) eq(Column#18, Column#34)]
├─HashLeftJoin_56 241379546.70 root inner join, inner:TableReader_102, equal:[eq(Column#17, Column#40)]
│ ├─HashLeftJoin_77 241379546.70 root inner join, inner:TableReader_100, equal:[eq(Column#18, Column#1)]
│ │ ├─HashRightJoin_80 300005811.00 root inner join, inner:HashRightJoin_91, equal:[eq(Column#10, Column#19)]
│ │ │ ├─HashRightJoin_91 500000.00 root inner join, inner:TableReader_95, equal:[eq(Column#49, Column#13)]
│ │ │ │ ├─TableReader_95 25.00 root data:TableScan_94
│ │ │ │ │ └─TableScan_94 25.00 cop[tikv] table:nation, range:[-inf,+inf], keep order:false
│ │ │ │ └─TableReader_93 500000.00 root data:TableScan_92
│ │ │ │ └─TableScan_92 500000.00 cop[tikv] table:supplier, range:[-inf,+inf], keep order:false
│ │ │ └─TableReader_97 300005811.00 root data:TableScan_96
│ │ │ └─TableScan_96 300005811.00 cop[tikv] table:lineitem, range:[-inf,+inf], keep order:false
│ │ └─TableReader_100 8000000.00 root data:Selection_99
│ │ └─Selection_99 8000000.00 cop[tikv] like(Column#2, "%dim%", 92)
│ │ └─TableScan_98 10000000.00 cop[tikv] table:part, range:[-inf,+inf], keep order:false
│ └─TableReader_102 75000000.00 root data:TableScan_101
│ └─TableScan_101 75000000.00 cop[tikv] table:orders, range:[-inf,+inf], keep order:false
└─TableReader_104 40000000.00 root data:TableScan_103
└─TableScan_103 40000000.00 cop[tikv] table:partsupp, range:[-inf,+inf], keep order:false
└─HashLeftJoin_44 971049283.51 root inner join, inner:TableReader_106, equal:[eq(Column#19, Column#35) eq(Column#18, Column#34)]
├─HashLeftJoin_56 241379546.70 root inner join, inner:TableReader_104, equal:[eq(Column#17, Column#40)]
│ ├─HashLeftJoin_79 241379546.70 root inner join, inner:TableReader_102, equal:[eq(Column#18, Column#1)]
│ │ ├─HashRightJoin_82 300005811.00 root inner join, inner:HashRightJoin_93, equal:[eq(Column#10, Column#19)]
│ │ │ ├─HashRightJoin_93 500000.00 root inner join, inner:TableReader_97, equal:[eq(Column#49, Column#13)]
│ │ │ │ ├─TableReader_97 25.00 root data:TableScan_96
│ │ │ │ │ └─TableScan_96 25.00 cop[tikv] table:nation, range:[-inf,+inf], keep order:false
│ │ │ │ └─TableReader_95 500000.00 root data:TableScan_94
│ │ │ │ └─TableScan_94 500000.00 cop[tikv] table:supplier, range:[-inf,+inf], keep order:false
│ │ │ └─TableReader_99 300005811.00 root data:TableScan_98
│ │ │ └─TableScan_98 300005811.00 cop[tikv] table:lineitem, range:[-inf,+inf], keep order:false
│ │ └─TableReader_102 8000000.00 root data:Selection_101
│ │ └─Selection_101 8000000.00 cop[tikv] like(Column#2, "%dim%", 92)
│ │ └─TableScan_100 10000000.00 cop[tikv] table:part, range:[-inf,+inf], keep order:false
│ └─TableReader_104 75000000.00 root data:TableScan_103
│ └─TableScan_103 75000000.00 cop[tikv] table:orders, range:[-inf,+inf], keep order:false
└─TableReader_106 40000000.00 root data:TableScan_105
└─TableScan_105 40000000.00 cop[tikv] table:partsupp, range:[-inf,+inf], keep order:false
/*
Q10 Returned Item Reporting Query
The query identifies customers who might be having problems with the parts that are shipped to them.
Expand Down Expand Up @@ -1223,24 +1223,24 @@ Projection_25 1.00 root Column#2, Column#104
└─TopN_28 1.00 root Column#104:desc, Column#2:asc, offset:0, count:100
└─HashAgg_34 1.00 root group by:Column#2, funcs:count(1), firstrow(Column#2)
└─IndexHashJoin_49 7828961.66 root anti semi join, inner:IndexLookUp_39, outer key:Column#8, inner key:Column#71, other cond:ne(Column#73, Column#10)
├─IndexHashJoin_84 9786202.08 root semi join, inner:IndexLookUp_75, outer key:Column#8, inner key:Column#38, other cond:ne(Column#40, Column#1), ne(Column#40, Column#10)
│ ├─IndexMergeJoin_95 12232752.60 root inner join, inner:TableReader_93, outer key:Column#8, inner key:Column#25
│ │ ├─HashRightJoin_101 12232752.60 root inner join, inner:HashRightJoin_114, equal:[eq(Column#1, Column#10)]
│ │ │ ├─HashRightJoin_114 20000.00 root inner join, inner:TableReader_119, equal:[eq(Column#34, Column#4)]
│ │ │ │ ├─TableReader_119 1.00 root data:Selection_118
│ │ │ │ │ └─Selection_118 1.00 cop[tikv] eq(Column#35, "EGYPT")
│ │ │ │ │ └─TableScan_117 25.00 cop[tikv] table:nation, range:[-inf,+inf], keep order:false
│ │ │ │ └─TableReader_116 500000.00 root data:TableScan_115
│ │ │ │ └─TableScan_115 500000.00 cop[tikv] table:supplier, range:[-inf,+inf], keep order:false
│ │ │ └─TableReader_122 240004648.80 root data:Selection_121
│ │ │ └─Selection_121 240004648.80 cop[tikv] gt(Column#20, Column#19)
│ │ │ └─TableScan_120 300005811.00 cop[tikv] table:l1, range:[-inf,+inf], keep order:false
│ │ └─TableReader_93 0.80 root data:Selection_92
│ │ └─Selection_92 0.80 cop[tikv] eq(Column#27, "F")
│ │ └─TableScan_91 1.00 cop[tikv] table:orders, range: decided by [Column#8], keep order:true
│ └─IndexLookUp_75 4.05 root
│ ├─IndexScan_73 4.05 cop[tikv] table:l2, index:L_ORDERKEY, L_LINENUMBER, range: decided by [eq(Column#38, Column#8)], keep order:false
│ └─TableScan_74 4.05 cop[tikv] table:l2, keep order:false
├─IndexHashJoin_88 9786202.08 root semi join, inner:IndexLookUp_79, outer key:Column#8, inner key:Column#38, other cond:ne(Column#40, Column#1), ne(Column#40, Column#10)
│ ├─IndexMergeJoin_99 12232752.60 root inner join, inner:TableReader_97, outer key:Column#8, inner key:Column#25
│ │ ├─HashRightJoin_105 12232752.60 root inner join, inner:HashRightJoin_118, equal:[eq(Column#1, Column#10)]
│ │ │ ├─HashRightJoin_118 20000.00 root inner join, inner:TableReader_123, equal:[eq(Column#34, Column#4)]
│ │ │ │ ├─TableReader_123 1.00 root data:Selection_122
│ │ │ │ │ └─Selection_122 1.00 cop[tikv] eq(Column#35, "EGYPT")
│ │ │ │ │ └─TableScan_121 25.00 cop[tikv] table:nation, range:[-inf,+inf], keep order:false
│ │ │ │ └─TableReader_120 500000.00 root data:TableScan_119
│ │ │ │ └─TableScan_119 500000.00 cop[tikv] table:supplier, range:[-inf,+inf], keep order:false
│ │ │ └─TableReader_126 240004648.80 root data:Selection_125
│ │ │ └─Selection_125 240004648.80 cop[tikv] gt(Column#20, Column#19)
│ │ │ └─TableScan_124 300005811.00 cop[tikv] table:l1, range:[-inf,+inf], keep order:false
│ │ └─TableReader_97 0.80 root data:Selection_96
│ │ └─Selection_96 0.80 cop[tikv] eq(Column#27, "F")
│ │ └─TableScan_95 1.00 cop[tikv] table:orders, range: decided by [Column#8], keep order:true
│ └─IndexLookUp_79 4.05 root
│ ├─IndexScan_77 4.05 cop[tikv] table:l2, index:L_ORDERKEY, L_LINENUMBER, range: decided by [eq(Column#38, Column#8)], keep order:false
│ └─TableScan_78 4.05 cop[tikv] table:l2, keep order:false
└─IndexLookUp_39 3.24 root
├─IndexScan_36 4.05 cop[tikv] table:l3, index:L_ORDERKEY, L_LINENUMBER, range: decided by [eq(Column#71, Column#8)], keep order:false
└─Selection_38 3.24 cop[tikv] gt(Column#83, Column#82)
Expand Down
5 changes: 4 additions & 1 deletion executor/builder.go
View file
Open in desktop
Expand Up @@ -1857,7 +1857,10 @@ func (b *executorBuilder) buildIndexLookUpMergeJoin(v *plannercore.PhysicalIndex

func (b *executorBuilder) buildIndexNestedLoopHashJoin(v *plannercore.PhysicalIndexHashJoin) Executor {
e := b.buildIndexLookUpJoin(&(v.PhysicalIndexJoin)).(*IndexLookUpJoin)
idxHash := &IndexNestedLoopHashJoin{IndexLookUpJoin: *e}
idxHash := &IndexNestedLoopHashJoin{
IndexLookUpJoin: *e,
keepOuterOrder: v.KeepOuterOrder,
}
concurrency := e.ctx.GetSessionVars().IndexLookupJoinConcurrency
idxHash.joiners = make([]joiner, concurrency)
for i := 0; i < concurrency; i++ {
Expand Down