Skip to content

[fix](pipeline) Disable non-pipeline engine for stream load#35555

Merged
Gabriel39 merged 1 commit intoapache:masterfrom
liaoxin01:disable_pipeline
May 29, 2024
Merged

[fix](pipeline) Disable non-pipeline engine for stream load#35555
Gabriel39 merged 1 commit intoapache:masterfrom
liaoxin01:disable_pipeline

Conversation

@liaoxin01
Copy link
Copy Markdown
Contributor

@liaoxin01 liaoxin01 commented May 28, 2024

Proposed changes

Issue Number: close #xxx
relates to: #34812

@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@liaoxin01
Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 41791 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f81d4b8c75a7c989ba7a840603a04c9254b7638, data reload: false

------ Round 1 ----------------------------------
q1	17601	4324	4271	4271
q2	2020	198	196	196
q3	10455	1273	1235	1235
q4	10184	770	836	770
q5	7494	2677	2647	2647
q6	220	133	129	129
q7	980	600	602	600
q8	9218	2132	2094	2094
q9	10226	6645	6661	6645
q10	9256	3902	3892	3892
q11	476	243	247	243
q12	473	232	246	232
q13	17625	3256	3212	3212
q14	275	231	230	230
q15	519	479	491	479
q16	512	423	397	397
q17	996	700	729	700
q18	8391	7971	7704	7704
q19	6713	1559	1483	1483
q20	677	330	322	322
q21	5122	4019	4148	4019
q22	372	322	291	291
Total cold run time: 119805 ms
Total hot run time: 41791 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4605	4478	4527	4478
q2	376	289	272	272
q3	3175	2983	2877	2877
q4	1981	1672	1601	1601
q5	5328	5495	5521	5495
q6	208	124	125	124
q7	2202	1843	1865	1843
q8	3250	3379	3412	3379
q9	8655	8696	8748	8696
q10	4092	3832	3775	3775
q11	583	499	513	499
q12	777	610	629	610
q13	16091	3178	3152	3152
q14	293	279	269	269
q15	516	489	484	484
q16	486	445	455	445
q17	1816	1547	1508	1508
q18	8030	7666	7299	7299
q19	1657	1506	1579	1506
q20	1993	1778	1797	1778
q21	7271	4785	4762	4762
q22	572	503	505	503
Total cold run time: 73957 ms
Total hot run time: 55355 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 168797 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f81d4b8c75a7c989ba7a840603a04c9254b7638, data reload: false

query1	937	382	382	382
query2	6581	2484	2388	2388
query3	6651	205	205	205
query4	19193	17307	17433	17307
query5	4292	434	414	414
query6	248	157	157	157
query7	4592	301	292	292
query8	256	185	212	185
query9	8497	2451	2436	2436
query10	454	286	269	269
query11	10629	10200	9919	9919
query12	135	90	86	86
query13	1630	374	365	365
query14	10127	6212	6682	6212
query15	211	172	166	166
query16	7442	282	258	258
query17	1279	539	518	518
query18	1945	269	271	269
query19	202	156	153	153
query20	93	88	83	83
query21	195	134	131	131
query22	4258	4095	3893	3893
query23	33540	32955	33345	32955
query24	9308	2784	2894	2784
query25	582	345	374	345
query26	706	155	159	155
query27	2241	329	326	326
query28	5866	2086	2101	2086
query29	847	608	593	593
query30	236	151	146	146
query31	1000	766	749	749
query32	95	52	57	52
query33	605	274	258	258
query34	868	472	487	472
query35	721	612	598	598
query36	1065	904	898	898
query37	108	65	69	65
query38	2940	2799	2753	2753
query39	825	804	806	804
query40	201	126	127	126
query41	47	45	44	44
query42	133	98	92	92
query43	597	572	553	553
query44	1070	725	755	725
query45	178	165	165	165
query46	1062	714	729	714
query47	1836	1754	1782	1754
query48	362	300	295	295
query49	848	366	379	366
query50	776	390	379	379
query51	6946	6773	6788	6773
query52	101	87	89	87
query53	367	287	285	285
query54	689	432	430	430
query55	72	71	72	71
query56	280	240	244	240
query57	1096	1068	1028	1028
query58	243	198	213	198
query59	3554	3299	3185	3185
query60	277	262	262	262
query61	89	85	85	85
query62	593	441	445	441
query63	312	285	296	285
query64	8467	2238	1783	1783
query65	3189	3131	3126	3126
query66	840	331	328	328
query67	15173	15085	14892	14892
query68	4521	551	540	540
query69	447	274	275	274
query70	1166	1120	1112	1112
query71	387	277	267	267
query72	7640	2744	2570	2570
query73	713	325	325	325
query74	6029	5623	5561	5561
query75	3345	2605	2609	2605
query76	2302	980	937	937
query77	446	269	271	269
query78	10228	9926	9645	9645
query79	2402	512	517	512
query80	1164	460	444	444
query81	519	231	228	228
query82	706	95	92	92
query83	236	176	190	176
query84	255	93	91	91
query85	1580	338	326	326
query86	498	317	301	301
query87	3303	3138	3130	3130
query88	4453	2468	2450	2450
query89	486	404	393	393
query90	2041	201	195	195
query91	132	124	107	107
query92	65	49	49	49
query93	2063	526	513	513
query94	1286	200	197	197
query95	406	328	323	323
query96	592	282	272	272
query97	3172	3036	2997	2997
query98	247	230	217	217
query99	1157	846	840	840
Total cold run time: 264785 ms
Total hot run time: 168797 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.8 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f81d4b8c75a7c989ba7a840603a04c9254b7638, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.06	0.06
query4	1.68	0.09	0.10
query5	0.51	0.50	0.51
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.48
query10	0.53	0.54	0.54
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.60
query14	0.76	0.79	0.79
query15	0.82	0.81	0.81
query16	0.35	0.38	0.37
query17	1.05	1.02	0.98
query18	0.23	0.24	0.24
query19	1.77	1.81	1.79
query20	0.02	0.01	0.00
query21	15.72	0.64	0.64
query22	3.84	7.01	2.19
query23	18.30	1.33	1.25
query24	1.71	0.34	0.21
query25	0.14	0.08	0.08
query26	0.27	0.15	0.16
query27	0.08	0.08	0.08
query28	13.25	1.02	1.00
query29	13.33	3.30	3.23
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.30	0.46	0.47
query33	2.93	2.87	2.91
query34	17.22	4.40	4.45
query35	4.50	4.46	4.58
query36	0.69	0.49	0.46
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.84 s
Total hot run time: 30.8 s

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 28, 2024
@Gabriel39 Gabriel39 merged commit 40d8a98 into apache:master May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants