Skip to content

Conversation

@starocean999
Copy link
Contributor

pick from #37644

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49769 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7f632655c81c338ecb014f38cd303eba4a12a544, data reload: false

------ Round 1 ----------------------------------
q1	18182	4415	4308	4308
q2	2075	160	156	156
q3	10251	1924	1944	1924
q4	10125	1256	1318	1256
q5	8557	3918	3878	3878
q6	230	127	169	127
q7	2044	1604	1618	1604
q8	9505	2742	2706	2706
q9	13905	10243	10135	10135
q10	8595	3548	3494	3494
q11	409	243	250	243
q12	471	307	314	307
q13	18357	4004	3984	3984
q14	362	336	328	328
q15	514	464	458	458
q16	692	573	576	573
q17	1129	953	896	896
q18	7330	6905	6956	6905
q19	1779	1680	1623	1623
q20	546	311	310	310
q21	4415	4128	4116	4116
q22	535	438	447	438
Total cold run time: 120008 ms
Total hot run time: 49769 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4411	4296	4361	4296
q2	324	226	230	226
q3	4165	4160	4130	4130
q4	2757	2738	2762	2738
q5	7191	7087	7094	7087
q6	237	122	120	120
q7	3225	2931	2888	2888
q8	4340	4463	4503	4463
q9	16897	16781	16816	16781
q10	4236	4314	4287	4287
q11	759	693	697	693
q12	1033	858	866	858
q13	7033	3750	3770	3750
q14	458	426	430	426
q15	518	464	449	449
q16	739	679	680	679
q17	3866	3903	3871	3871
q18	8849	8838	8736	8736
q19	1758	1667	1686	1667
q20	2427	2169	2149	2149
q21	8563	8517	8469	8469
q22	1084	966	968	966
Total cold run time: 84870 ms
Total hot run time: 79729 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204301 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7f632655c81c338ecb014f38cd303eba4a12a544, data reload: false

query1	943	421	380	380
query2	6550	2857	2909	2857
query3	6931	207	202	202
query4	20109	18073	18064	18064
query5	19727	6611	6568	6568
query6	296	213	227	213
query7	4159	312	319	312
query8	399	497	479	479
query9	3149	2706	2620	2620
query10	415	294	297	294
query11	11281	10717	10824	10717
query12	123	82	76	76
query13	5605	709	697	697
query14	17676	13866	13273	13273
query15	366	256	256	256
query16	6478	295	266	266
query17	1697	1464	862	862
query18	2334	408	427	408
query19	212	153	164	153
query20	80	80	80	80
query21	195	96	104	96
query22	5262	5006	4975	4975
query23	32751	31906	31878	31878
query24	7032	6616	6559	6559
query25	539	455	425	425
query26	532	172	168	168
query27	1875	298	304	298
query28	6142	2382	2313	2313
query29	2959	2902	2867	2867
query30	242	167	178	167
query31	882	736	744	736
query32	72	69	62	62
query33	396	263	253	253
query34	845	473	489	473
query35	1111	954	937	937
query36	1279	1327	1209	1209
query37	91	62	60	60
query38	3089	2938	2939	2938
query39	1379	1330	1339	1330
query40	211	94	97	94
query41	46	44	47	44
query42	81	89	85	85
query43	682	627	671	627
query44	1155	726	740	726
query45	252	234	235	234
query46	1222	984	1018	984
query47	1809	1790	1669	1669
query48	1032	726	721	721
query49	620	371	372	371
query50	868	649	640	640
query51	4831	4637	4652	4637
query52	98	89	94	89
query53	464	330	322	322
query54	2684	2497	2467	2467
query55	98	74	78	74
query56	241	223	210	210
query57	1233	1140	1164	1140
query58	225	200	207	200
query59	4326	4150	4244	4150
query60	228	214	207	207
query61	96	92	96	92
query62	796	456	468	456
query63	498	356	340	340
query64	2559	1520	1345	1345
query65	3621	3581	3567	3567
query66	822	390	380	380
query67	16086	15649	14964	14964
query68	10077	685	654	654
query69	586	367	357	357
query70	1942	1518	1443	1443
query71	416	314	312	312
query72	6563	3494	3534	3494
query73	747	333	321	321
query74	6272	5888	5919	5888
query75	5366	3729	3686	3686
query76	6489	1183	1209	1183
query77	1120	260	264	260
query78	12509	12228	11485	11485
query79	12655	648	666	648
query80	815	405	405	405
query81	495	241	240	240
query82	1539	101	104	101
query83	184	140	138	138
query84	259	73	72	72
query85	903	329	333	329
query86	331	313	315	313
query87	3277	3016	3029	3016
query88	5721	2364	2375	2364
query89	477	291	324	291
query90	1966	223	211	211
query91	180	142	155	142
query92	64	53	56	53
query93	7088	557	587	557
query94	753	205	219	205
query95	1123	1062	1052	1052
query96	672	340	335	335
query97	6503	6469	6396	6396
query98	186	192	177	177
query99	2962	817	883	817
Total cold run time: 322327 ms
Total hot run time: 204301 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7f632655c81c338ecb014f38cd303eba4a12a544, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.06	0.06
query4	1.77	0.06	0.07
query5	0.54	0.53	0.52
query6	1.23	0.63	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.49
query10	0.53	0.52	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.63	0.61	0.60
query14	0.78	0.80	0.79
query15	0.80	0.77	0.77
query16	0.39	0.39	0.39
query17	0.98	1.02	1.01
query18	0.24	0.23	0.26
query19	1.95	1.86	1.79
query20	0.01	0.02	0.02
query21	15.48	0.62	0.56
query22	2.35	2.30	1.44
query23	16.73	1.07	0.93
query24	5.51	1.41	1.31
query25	0.34	0.11	0.06
query26	0.60	0.16	0.14
query27	0.04	0.04	0.05
query28	7.25	0.72	0.72
query29	12.78	2.29	2.37
query30	0.57	0.55	0.49
query31	2.81	0.41	0.37
query32	3.35	0.51	0.49
query33	3.09	3.08	3.05
query34	15.27	4.79	4.79
query35	4.83	4.84	4.84
query36	1.09	1.04	1.03
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.15	0.15
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.03	0.02	0.01
Total cold run time: 103.49 s
Total hot run time: 30.91 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 7f632655c81c338ecb014f38cd303eba4a12a544 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49494 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5901119b9097c78ac3e41ddbc3b2f7c863be01e7, data reload: false

------ Round 1 ----------------------------------
q1	18000	4347	4333	4333
q2	2063	151	146	146
q3	10402	1903	1896	1896
q4	10311	1246	1294	1246
q5	8722	3806	3987	3806
q6	245	166	126	126
q7	2053	1645	1626	1626
q8	9400	2750	2704	2704
q9	10591	10170	10234	10170
q10	8632	3472	3465	3465
q11	427	238	248	238
q12	469	301	295	295
q13	18332	3961	3994	3961
q14	358	326	321	321
q15	496	457	463	457
q16	680	570	568	568
q17	1108	947	961	947
q18	7169	6739	6897	6739
q19	1783	1684	1652	1652
q20	517	315	308	308
q21	4439	4165	4051	4051
q22	540	439	454	439
Total cold run time: 116737 ms
Total hot run time: 49494 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4319	4306	4288	4288
q2	328	231	224	224
q3	4189	4124	4121	4121
q4	2747	2758	2738	2738
q5	7228	7084	6996	6996
q6	237	118	120	118
q7	3223	2841	2867	2841
q8	4302	4454	4512	4454
q9	16840	16765	16480	16480
q10	4176	4250	4241	4241
q11	738	695	701	695
q12	1034	864	840	840
q13	6874	3703	3728	3703
q14	466	438	441	438
q15	497	457	457	457
q16	742	686	677	677
q17	3841	3895	3865	3865
q18	8761	8802	8676	8676
q19	1718	1690	1640	1640
q20	2371	2112	2102	2102
q21	8655	8542	8509	8509
q22	1090	1045	1023	1023
Total cold run time: 84376 ms
Total hot run time: 79126 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 205407 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5901119b9097c78ac3e41ddbc3b2f7c863be01e7, data reload: false

query1	914	378	417	378
query2	6491	2814	2591	2591
query3	7068	208	208	208
query4	21217	18017	18013	18013
query5	19669	6485	6493	6485
query6	296	220	214	214
query7	4155	296	301	296
query8	422	445	417	417
query9	3058	2695	2630	2630
query10	403	305	295	295
query11	11485	10824	10806	10806
query12	121	78	78	78
query13	5609	687	707	687
query14	17259	13563	13622	13563
query15	368	252	250	250
query16	6546	292	263	263
query17	1967	1452	867	867
query18	2287	402	405	402
query19	213	144	153	144
query20	85	78	90	78
query21	193	93	96	93
query22	5216	5107	5137	5107
query23	32326	32093	31899	31899
query24	7068	6556	6458	6458
query25	517	430	428	428
query26	528	156	157	156
query27	1891	293	293	293
query28	6235	2373	2334	2334
query29	3084	2742	2645	2645
query30	243	165	168	165
query31	892	748	771	748
query32	69	61	60	60
query33	394	257	245	245
query34	839	468	480	468
query35	1092	915	963	915
query36	1424	1524	1162	1162
query37	89	63	61	61
query38	3088	2904	2983	2904
query39	1373	1330	1321	1321
query40	198	100	94	94
query41	46	44	51	44
query42	91	79	82	79
query43	748	696	656	656
query44	1117	725	728	725
query45	244	245	233	233
query46	1229	952	1005	952
query47	1773	1812	2000	1812
query48	1024	713	698	698
query49	638	365	371	365
query50	870	598	649	598
query51	4736	4712	4666	4666
query52	88	79	88	79
query53	448	335	321	321
query54	2709	2487	2472	2472
query55	92	78	85	78
query56	241	213	214	213
query57	1253	1057	1088	1057
query58	223	202	194	194
query59	4103	4115	4252	4115
query60	216	203	226	203
query61	95	91	96	91
query62	840	439	476	439
query63	483	345	340	340
query64	2604	1551	1481	1481
query65	3638	3567	3558	3558
query66	832	375	376	375
query67	19148	17329	16318	16318
query68	9108	658	659	658
query69	587	350	355	350
query70	1495	1402	1398	1398
query71	409	307	320	307
query72	6478	3458	3466	3458
query73	742	337	314	314
query74	6418	5897	5966	5897
query75	5147	3780	3569	3569
query76	5373	1174	1202	1174
query77	916	255	255	255
query78	12548	13713	11909	11909
query79	7372	632	640	632
query80	676	392	400	392
query81	492	231	233	231
query82	297	106	98	98
query83	163	130	129	129
query84	258	69	71	69
query85	718	329	331	329
query86	343	303	309	303
query87	3307	2990	3008	2990
query88	3879	2342	2342	2342
query89	347	277	306	277
query90	1938	208	214	208
query91	175	140	142	140
query92	64	55	53	53
query93	991	552	557	552
query94	690	214	210	210
query95	1126	1078	1068	1068
query96	622	326	325	325
query97	6602	6358	6379	6358
query98	194	180	174	174
query99	2819	858	855	855
Total cold run time: 308215 ms
Total hot run time: 205407 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5901119b9097c78ac3e41ddbc3b2f7c863be01e7, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.04	0.05
query4	1.79	0.07	0.06
query5	0.54	0.53	0.52
query6	1.23	0.62	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.49
query10	0.53	0.54	0.56
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.63	0.61	0.60
query14	0.77	0.79	0.78
query15	0.79	0.76	0.78
query16	0.36	0.38	0.36
query17	1.03	1.03	1.00
query18	0.24	0.25	0.24
query19	1.94	1.81	1.86
query20	0.02	0.01	0.01
query21	15.46	0.57	0.56
query22	2.05	2.72	1.78
query23	16.71	1.03	1.06
query24	6.70	0.92	1.20
query25	0.37	0.06	0.09
query26	0.69	0.15	0.16
query27	0.05	0.05	0.03
query28	6.44	0.76	0.75
query29	12.61	2.40	2.34
query30	0.62	0.54	0.57
query31	2.81	0.40	0.37
query32	3.36	0.49	0.49
query33	3.11	3.07	3.07
query34	15.29	4.80	4.77
query35	4.85	4.84	4.83
query36	1.07	1.02	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.01
Total cold run time: 103.56 s
Total hot run time: 31.02 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5901119b9097c78ac3e41ddbc3b2f7c863be01e7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49737 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f4074c805ceb515c8beef66e6cdf9b9b756558e0, data reload: false

------ Round 1 ----------------------------------
q1	18013	4475	4356	4356
q2	2037	159	149	149
q3	10444	1938	1926	1926
q4	10316	1254	1323	1254
q5	8596	3913	3905	3905
q6	235	126	127	126
q7	2064	1584	1591	1584
q8	9303	2702	2689	2689
q9	10507	10143	10273	10143
q10	8667	3544	3540	3540
q11	420	250	254	250
q12	481	313	307	307
q13	18341	3966	3970	3966
q14	363	322	327	322
q15	510	475	462	462
q16	690	562	569	562
q17	1138	971	960	960
q18	7306	6947	6771	6771
q19	1793	1625	1650	1625
q20	532	310	300	300
q21	4452	4183	4102	4102
q22	544	438	454	438
Total cold run time: 116752 ms
Total hot run time: 49737 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4336	4395	4336
q2	326	225	226	225
q3	4189	4091	4154	4091
q4	2733	2736	2743	2736
q5	7138	7040	7039	7039
q6	244	122	118	118
q7	3241	2847	2837	2837
q8	4309	4470	4490	4470
q9	16881	16795	16813	16795
q10	4217	4285	4257	4257
q11	754	665	694	665
q12	1039	864	885	864
q13	7304	3745	3751	3745
q14	457	416	430	416
q15	502	463	456	456
q16	732	679	682	679
q17	3852	3891	3814	3814
q18	8932	8758	8775	8758
q19	1764	1721	1686	1686
q20	2384	2116	2138	2116
q21	8585	8466	8469	8466
q22	1045	1019	1007	1007
Total cold run time: 84982 ms
Total hot run time: 79576 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204648 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f4074c805ceb515c8beef66e6cdf9b9b756558e0, data reload: false

query1	943	431	393	393
query2	6548	2825	2631	2631
query3	6915	206	203	203
query4	20508	18186	17994	17994
query5	19730	6492	6518	6492
query6	294	226	239	226
query7	4171	307	310	307
query8	435	422	426	422
query9	3115	2667	2623	2623
query10	403	319	324	319
query11	11326	10687	10746	10687
query12	127	79	81	79
query13	5611	724	709	709
query14	17768	13593	13349	13349
query15	365	248	246	246
query16	6486	287	264	264
query17	1739	1450	894	894
query18	2303	404	408	404
query19	219	160	150	150
query20	81	80	83	80
query21	199	98	99	98
query22	5061	4824	4976	4824
query23	32717	32004	32051	32004
query24	6950	6565	6472	6472
query25	526	444	415	415
query26	529	165	161	161
query27	1871	291	294	291
query28	6215	2399	2357	2357
query29	2932	2763	2676	2676
query30	250	172	167	167
query31	914	734	736	734
query32	68	66	61	61
query33	397	268	263	263
query34	860	488	483	483
query35	1136	952	919	919
query36	1331	1356	1158	1158
query37	92	63	60	60
query38	3064	2903	2920	2903
query39	1381	1331	1339	1331
query40	211	99	93	93
query41	46	45	44	44
query42	86	89	89	89
query43	848	787	701	701
query44	1148	735	742	735
query45	251	242	235	235
query46	1230	957	993	957
query47	1894	1629	1695	1629
query48	1013	732	718	718
query49	624	379	365	365
query50	876	623	632	623
query51	4769	4691	4649	4649
query52	90	85	75	75
query53	455	326	314	314
query54	2662	2446	2442	2442
query55	84	81	88	81
query56	226	234	215	215
query57	1144	1143	1045	1045
query58	224	216	196	196
query59	4249	4185	4446	4185
query60	206	224	219	219
query61	100	98	99	98
query62	790	480	467	467
query63	490	341	348	341
query64	2624	1524	1516	1516
query65	3703	3546	3584	3546
query66	765	372	366	366
query67	17707	17037	15868	15868
query68	8372	657	660	657
query69	554	329	348	329
query70	1712	1452	1623	1452
query71	411	303	322	303
query72	6551	3471	3491	3471
query73	733	324	312	312
query74	6237	5850	5864	5850
query75	4556	3711	3663	3663
query76	4673	1163	1182	1163
query77	620	249	252	249
query78	12497	12653	11747	11747
query79	10568	654	640	640
query80	1706	394	396	394
query81	523	234	234	234
query82	1123	103	100	100
query83	183	132	137	132
query84	261	70	72	70
query85	1310	327	343	327
query86	355	326	317	317
query87	3289	3042	3120	3042
query88	5226	2364	2341	2341
query89	440	308	271	271
query90	1751	215	215	215
query91	184	139	142	139
query92	60	59	52	52
query93	5796	546	561	546
query94	852	211	211	211
query95	1086	1073	1038	1038
query96	659	330	345	330
query97	6529	6431	6406	6406
query98	187	175	171	171
query99	2907	919	815	815
Total cold run time: 316006 ms
Total hot run time: 204648 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f4074c805ceb515c8beef66e6cdf9b9b756558e0, data reload: false

query1	0.02	0.02	0.03
query2	0.06	0.03	0.03
query3	0.25	0.05	0.05
query4	1.81	0.06	0.06
query5	0.53	0.52	0.52
query6	1.24	0.60	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.53	0.49	0.48
query10	0.55	0.54	0.55
query11	0.12	0.09	0.09
query12	0.12	0.10	0.09
query13	0.62	0.60	0.60
query14	0.78	0.80	0.81
query15	0.78	0.76	0.77
query16	0.38	0.38	0.37
query17	1.04	1.02	1.02
query18	0.21	0.25	0.25
query19	1.91	1.87	1.86
query20	0.01	0.01	0.00
query21	15.50	0.56	0.56
query22	1.92	2.13	1.98
query23	17.42	0.98	0.91
query24	7.14	0.84	0.85
query25	0.37	0.10	0.06
query26	0.74	0.15	0.17
query27	0.05	0.04	0.03
query28	6.61	0.76	0.77
query29	12.60	2.30	2.32
query30	0.62	0.53	0.56
query31	2.80	0.40	0.38
query32	3.38	0.51	0.49
query33	3.14	3.11	3.10
query34	15.27	4.77	4.79
query35	4.84	4.85	4.85
query36	1.05	1.03	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.03	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.87 s
Total hot run time: 31.15 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f4074c805ceb515c8beef66e6cdf9b9b756558e0 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@morrySnow morrySnow merged commit 971a7c8 into apache:branch-2.0 Jul 17, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants