Skip to content

[pipelineX](localexchange) Adjust local exchange plan rule#30393

Merged
Gabriel39 merged 2 commits intoapache:masterfrom
Gabriel39:dev_0126
Jan 26, 2024
Merged

[pipelineX](localexchange) Adjust local exchange plan rule#30393
Gabriel39 merged 2 commits intoapache:masterfrom
Gabriel39:dev_0126

Conversation

@Gabriel39
Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Gabriel39
Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39
Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 38706 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc0d8a350a3a689a98501569c056e210c2d33ba9, data reload: false

------ Round 1 ----------------------------------
q1	17639	5402	5298	5298
q2	2037	141	131	131
q3	10785	1179	1155	1155
q4	10406	801	782	782
q5	7728	3180	3137	3137
q6	194	121	122	121
q7	843	496	489	489
q8	9248	1870	1926	1870
q9	7228	6314	6328	6314
q10	8192	3106	3076	3076
q11	406	219	204	204
q12	356	195	190	190
q13	17991	3430	3369	3369
q14	252	221	219	219
q15	549	529	497	497
q16	425	369	376	369
q17	948	475	470	470
q18	7483	7028	6748	6748
q19	1544	1261	1252	1252
q20	574	295	287	287
q21	2791	2427	2499	2427
q22	363	301	306	301
Total cold run time: 107982 ms
Total hot run time: 38706 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5351	5243	5305	5243
q2	336	217	213	213
q3	3294	3235	3200	3200
q4	2099	2076	2069	2069
q5	6066	5915	5955	5915
q6	198	121	117	117
q7	2356	1913	1869	1869
q8	3243	3370	3481	3370
q9	8859	8843	8817	8817
q10	3897	3847	3858	3847
q11	561	452	465	452
q12	803	655	661	655
q13	16882	3179	3178	3178
q14	274	253	271	253
q15	553	513	521	513
q16	522	461	474	461
q17	1908	1858	1899	1858
q18	9565	18248	9908	9908
q19	25247	1537	1522	1522
q20	4655	1936	1930	1930
q21	14477	5428	5279	5279
q22	989	554	520	520
Total cold run time: 112135 ms
Total hot run time: 61189 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 187431 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc0d8a350a3a689a98501569c056e210c2d33ba9, data reload: false

query1	921	338	334	334
query2	6556	2010	1864	1864
query3	6691	205	201	201
query4	32981	22224	22189	22189
query5	4457	365	376	365
query6	238	154	147	147
query7	4596	272	258	258
query8	213	178	173	173
query9	8442	2623	2583	2583
query10	440	231	245	231
query11	17057	15556	15797	15556
query12	118	70	67	67
query13	1821	396	376	376
query14	11055	7136	7088	7088
query15	212	183	186	183
query16	7557	276	275	275
query17	1500	495	477	477
query18	1804	256	260	256
query19	172	130	130	130
query20	75	69	70	69
query21	193	135	130	130
query22	4905	4742	4785	4742
query23	32046	31564	31727	31564
query24	13285	2826	2867	2826
query25	570	327	316	316
query26	1654	155	145	145
query27	3279	297	290	290
query28	8318	1878	1831	1831
query29	2725	665	658	658
query30	284	140	135	135
query31	930	746	762	746
query32	76	48	48	48
query33	703	222	211	211
query34	1129	486	472	472
query35	889	754	797	754
query36	1269	1219	1224	1219
query37	95	59	58	58
query38	3386	3272	3210	3210
query39	1308	1275	1267	1267
query40	343	87	86	86
query41	38	35	35	35
query42	94	86	82	82
query43	492	487	488	487
query44	1132	692	707	692
query45	192	181	174	174
query46	1078	669	665	665
query47	1656	1492	1514	1492
query48	383	313	301	301
query49	1240	302	284	284
query50	687	322	323	322
query51	5288	5159	5227	5159
query52	93	75	75	75
query53	317	266	262	262
query54	244	183	203	183
query55	84	81	75	75
query56	179	166	169	166
query57	1012	918	941	918
query58	176	161	153	153
query59	2828	2710	2670	2670
query60	208	177	180	177
query61	84	88	82	82
query62	584	360	358	358
query63	277	259	271	259
query64	6112	1784	1732	1732
query65	3362	3290	3262	3262
query66	1266	319	312	312
query67	15823	14955	15336	14955
query68	10450	510	508	508
query69	619	293	309	293
query70	1716	1480	1523	1480
query71	10472	10205	10201	10201
query72	4688	2830	2873	2830
query73	1944	316	314	314
query74	7033	6386	6361	6361
query75	4759	2280	2297	2280
query76	6642	1022	972	972
query77	850	229	245	229
query78	9462	8828	8555	8555
query79	1001	499	494	494
query80	586	327	314	314
query81	450	207	202	202
query82	229	83	80	80
query83	133	119	113	113
query84	272	68	69	68
query85	1071	323	331	323
query86	389	402	404	402
query87	3492	3364	3274	3274
query88	2730	2190	2206	2190
query89	419	367	346	346
query90	2067	191	189	189
query91	153	131	121	121
query92	51	50	44	44
query93	922	447	416	416
query94	1184	159	157	157
query95	490	467	462	462
query96	618	320	322	320
query97	4248	4155	4154	4154
query98	226	191	190	190
query99	1010	687	721	687
Total cold run time: 309598 ms
Total hot run time: 187431 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc0d8a350a3a689a98501569c056e210c2d33ba9, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.03
query3	0.23	0.04	0.04
query4	1.71	0.06	0.06
query5	0.53	0.51	0.53
query6	1.25	0.64	0.64
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.51	0.50
query10	0.56	0.56	0.57
query11	0.12	0.10	0.09
query12	0.11	0.09	0.09
query13	0.61	0.61	0.60
query14	0.78	0.81	0.81
query15	0.80	0.78	0.78
query16	0.37	0.36	0.40
query17	1.02	1.06	1.03
query18	0.26	0.24	0.26
query19	1.88	1.79	1.80
query20	0.01	0.01	0.01
query21	15.40	0.56	0.55
query22	2.42	2.20	1.36
query23	17.11	0.72	1.00
query24	2.41	1.66	1.09
query25	0.44	0.30	0.16
query26	0.43	0.14	0.14
query27	0.05	0.05	0.05
query28	10.61	0.78	0.78
query29	12.63	3.15	3.12
query30	0.57	0.51	0.50
query31	2.78	0.35	0.35
query32	3.37	0.49	0.50
query33	3.26	3.22	3.26
query34	15.81	4.29	4.29
query35	4.39	4.30	4.37
query36	1.11	1.08	1.09
query37	0.07	0.05	0.05
query38	0.04	0.02	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.16 s
Total hot run time: 30.65 s

@doris-robot
Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dc0d8a350a3a689a98501569c056e210c2d33ba9 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jan 26, 2024
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

Copy link
Copy Markdown
Member

@mrhhsg mrhhsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Gabriel39 Gabriel39 merged commit c2dd41c into apache:master Jan 26, 2024
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants