Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](decommission be) decommission check replica num #32748 #34038

Merged
merged 6 commits into from
May 10, 2024
Merged

[improvement](decommission be) decommission check replica num #32748 #34038

merged 6 commits into from
May 10, 2024

Conversation

yujun777
Copy link
Collaborator

cherry pick from #32748

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Collaborator Author

run buildall

@yujun777 yujun777 marked this pull request as draft April 24, 2024 06:54
@doris-robot
Copy link

TPC-H: Total hot run time: 49640 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d778d4d1c42a17859326a6878bd25de9da743337, data reload: false

------ Round 1 ----------------------------------
q1	17767	4372	4365	4365
q2	2022	154	138	138
q3	10483	1904	1926	1904
q4	10333	1248	1324	1248
q5	8638	3870	3923	3870
q6	233	123	122	122
q7	2025	1596	1585	1585
q8	9296	2725	2698	2698
q9	10743	10561	10388	10388
q10	8624	3504	3483	3483
q11	421	245	245	245
q12	462	298	300	298
q13	18359	3966	3988	3966
q14	362	326	327	326
q15	492	465	454	454
q16	683	602	605	602
q17	1112	976	982	976
q18	7207	6752	6834	6752
q19	1702	1556	1525	1525
q20	512	306	292	292
q21	4470	4104	4027	4027
q22	502	412	376	376
Total cold run time: 116448 ms
Total hot run time: 49640 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4299	4307	4295	4295
q2	318	218	217	217
q3	4173	4149	4142	4142
q4	2742	2721	2746	2721
q5	7235	7099	7144	7099
q6	234	114	120	114
q7	3268	2898	2835	2835
q8	4358	4482	4482	4482
q9	17159	17030	16915	16915
q10	4191	4232	4237	4232
q11	755	693	722	693
q12	1013	841	849	841
q13	7364	3709	3742	3709
q14	443	424	436	424
q15	508	463	447	447
q16	752	700	703	700
q17	3790	3820	3817	3817
q18	8759	8707	8773	8707
q19	1700	1710	1659	1659
q20	2368	2154	2087	2087
q21	8466	8459	8470	8459
q22	1025	973	944	944
Total cold run time: 84920 ms
Total hot run time: 79539 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202751 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d778d4d1c42a17859326a6878bd25de9da743337, data reload: false

query1	920	386	383	383
query2	6549	2778	2816	2778
query3	6912	197	197	197
query4	21439	18147	18031	18031
query5	19716	6508	6518	6508
query6	307	226	228	226
query7	4166	299	307	299
query8	259	253	261	253
query9	3161	2768	2696	2696
query10	425	293	304	293
query11	11387	10768	10633	10633
query12	118	76	72	72
query13	5591	686	667	667
query14	17546	13132	13509	13132
query15	383	223	233	223
query16	6450	267	255	255
query17	1735	1446	867	867
query18	2321	398	395	395
query19	191	143	140	140
query20	74	76	77	76
query21	187	96	89	89
query22	5286	5064	5119	5064
query23	32712	31779	31968	31779
query24	6968	6519	6485	6485
query25	525	433	417	417
query26	520	171	154	154
query27	1892	290	294	290
query28	6201	2376	2335	2335
query29	3025	2925	2748	2748
query30	240	161	162	161
query31	893	739	744	739
query32	64	57	59	57
query33	398	241	237	237
query34	835	464	472	464
query35	1119	910	924	910
query36	1339	1170	1178	1170
query37	93	60	60	60
query38	3071	2920	2881	2881
query39	1352	1324	1317	1317
query40	200	90	103	90
query41	42	34	33	33
query42	83	90	80	80
query43	758	695	694	694
query44	1140	728	728	728
query45	238	227	223	223
query46	1230	975	973	973
query47	1940	1774	1782	1774
query48	1025	723	706	706
query49	623	361	376	361
query50	883	587	577	577
query51	4761	4659	4693	4659
query52	90	76	69	69
query53	443	320	307	307
query54	2669	2458	2484	2458
query55	89	78	79	78
query56	203	202	201	201
query57	1173	1138	1108	1108
query58	211	210	185	185
query59	4187	4076	3765	3765
query60	202	191	195	191
query61	86	79	79	79
query62	802	522	481	481
query63	476	326	330	326
query64	2514	1599	1350	1350
query65	3631	3544	3570	3544
query66	753	374	380	374
query67	18537	15876	15502	15502
query68	8601	679	689	679
query69	580	330	340	330
query70	1510	1411	1402	1402
query71	400	292	303	292
query72	6450	3415	3398	3398
query73	731	319	313	313
query74	6370	5792	5826	5792
query75	4632	3628	3598	3598
query76	4819	1173	1219	1173
query77	627	244	248	244
query78	12942	11925	11579	11579
query79	8768	632	641	632
query80	855	399	385	385
query81	496	230	229	229
query82	1327	97	95	95
query83	161	132	135	132
query84	251	70	66	66
query85	776	304	284	284
query86	322	305	336	305
query87	3283	3054	3024	3024
query88	4886	2299	2289	2289
query89	371	285	292	285
query90	1917	208	199	199
query91	168	137	130	130
query92	54	47	52	47
query93	5870	586	552	552
query94	727	198	200	198
query95	1117	1075	1027	1027
query96	639	320	323	320
query97	6448	6387	6269	6269
query98	188	175	167	167
query99	2981	919	933	919
Total cold run time: 314626 ms
Total hot run time: 202751 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d778d4d1c42a17859326a6878bd25de9da743337, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.81	0.06	0.06
query5	0.54	0.53	0.52
query6	1.33	0.66	0.62
query7	0.02	0.02	0.01
query8	0.04	0.02	0.02
query9	0.54	0.48	0.49
query10	0.53	0.54	0.54
query11	0.11	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.61
query14	0.80	0.78	0.79
query15	0.79	0.76	0.75
query16	0.36	0.38	0.36
query17	1.00	1.01	1.02
query18	0.24	0.24	0.25
query19	1.90	1.86	1.88
query20	0.01	0.01	0.01
query21	15.49	0.56	0.58
query22	2.06	2.17	1.88
query23	16.99	1.10	0.92
query24	4.44	0.74	1.10
query25	0.38	0.10	0.04
query26	0.49	0.16	0.14
query27	0.05	0.04	0.03
query28	9.11	0.75	0.72
query29	12.62	2.36	2.26
query30	0.59	0.49	0.49
query31	2.80	0.38	0.38
query32	3.41	0.50	0.51
query33	3.07	3.08	3.08
query34	15.27	4.81	4.78
query35	4.85	4.86	4.85
query36	1.05	1.02	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.14 s
Total hot run time: 30.76 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d778d4d1c42a17859326a6878bd25de9da743337 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@yujun777 yujun777 force-pushed the pick_32748_to_upstream_branch-2.0 branch from d778d4d to 63fadb7 Compare April 28, 2024 07:07
@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49359 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 63fadb7e41215931002fe28e51a7e436c692ea3e, data reload: false

------ Round 1 ----------------------------------
q1	18004	4397	4310	4310
q2	2017	147	138	138
q3	10310	1926	1942	1926
q4	10391	1237	1328	1237
q5	8471	3883	3896	3883
q6	230	122	122	122
q7	1993	1609	1582	1582
q8	9331	2702	2702	2702
q9	10669	10466	10206	10206
q10	8612	3513	3503	3503
q11	424	243	246	243
q12	455	303	308	303
q13	18362	3930	4026	3930
q14	351	318	326	318
q15	513	462	452	452
q16	658	574	577	574
q17	1111	949	948	948
q18	7268	6764	6925	6764
q19	1703	1565	1477	1477
q20	513	303	323	303
q21	4440	4087	4039	4039
q22	481	407	399	399
Total cold run time: 116307 ms
Total hot run time: 49359 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4329	4291	4291
q2	320	225	223	223
q3	4156	4132	4140	4132
q4	2754	2742	2737	2737
q5	7184	7158	7129	7129
q6	233	119	116	116
q7	3264	2899	2825	2825
q8	4359	4454	4462	4454
q9	16818	16629	16628	16628
q10	4218	4243	4258	4243
q11	761	672	646	646
q12	1022	831	869	831
q13	7148	3739	3739	3739
q14	447	421	430	421
q15	502	460	453	453
q16	741	677	678	677
q17	3823	3882	3863	3863
q18	8816	8859	8692	8692
q19	1681	1707	1630	1630
q20	2383	2142	2111	2111
q21	8473	8395	8477	8395
q22	1066	935	988	935
Total cold run time: 84494 ms
Total hot run time: 79171 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202967 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 63fadb7e41215931002fe28e51a7e436c692ea3e, data reload: false

query1	921	388	375	375
query2	6545	2942	2662	2662
query3	6925	197	195	195
query4	20374	17988	17986	17986
query5	19708	6490	6448	6448
query6	281	220	227	220
query7	4147	287	297	287
query8	283	233	239	233
query9	3156	2702	2643	2643
query10	399	284	284	284
query11	11353	10703	10722	10703
query12	122	71	72	71
query13	5568	687	673	673
query14	17291	13149	13618	13149
query15	381	218	236	218
query16	6472	270	251	251
query17	1773	1458	858	858
query18	2336	401	400	400
query19	201	150	145	145
query20	79	77	72	72
query21	186	95	98	95
query22	5272	5035	4998	4998
query23	32511	31852	31827	31827
query24	7038	6476	6462	6462
query25	498	435	406	406
query26	520	164	152	152
query27	1894	295	294	294
query28	6141	2406	2369	2369
query29	2927	2741	2654	2654
query30	243	162	160	160
query31	912	748	730	730
query32	66	59	62	59
query33	415	255	243	243
query34	845	466	482	466
query35	1135	897	904	897
query36	1379	1270	1268	1268
query37	88	64	60	60
query38	3111	2963	2910	2910
query39	1361	1322	1319	1319
query40	203	98	96	96
query41	35	34	34	34
query42	85	84	80	80
query43	650	768	721	721
query44	1142	733	723	723
query45	240	229	226	226
query46	1210	977	941	941
query47	1702	1731	1828	1731
query48	1009	719	697	697
query49	626	366	386	366
query50	845	605	633	605
query51	4715	4677	4657	4657
query52	82	81	77	77
query53	436	320	317	317
query54	2660	2444	2477	2444
query55	94	70	76	70
query56	195	212	223	212
query57	1166	1185	1089	1089
query58	213	209	193	193
query59	4100	4300	4038	4038
query60	211	201	200	200
query61	89	83	82	82
query62	847	486	492	486
query63	470	334	334	334
query64	2398	1592	1359	1359
query65	3651	3575	3593	3575
query66	747	368	396	368
query67	16484	15727	15097	15097
query68	8776	659	675	659
query69	577	342	366	342
query70	1585	1426	1319	1319
query71	405	304	337	304
query72	6469	3444	3452	3444
query73	731	316	322	316
query74	6406	5849	5863	5849
query75	5254	3684	3766	3684
query76	5399	1124	1122	1122
query77	913	251	244	244
query78	12642	11683	11715	11683
query79	8286	618	637	618
query80	1256	399	376	376
query81	484	229	230	229
query82	1669	96	99	96
query83	170	129	128	128
query84	253	69	68	68
query85	851	301	303	301
query86	345	331	292	292
query87	3229	3028	3021	3021
query88	5077	2310	2341	2310
query89	377	310	284	284
query90	1966	210	206	206
query91	172	130	147	130
query92	61	54	50	50
query93	5582	602	605	602
query94	692	204	207	204
query95	1113	1076	1059	1059
query96	633	331	327	327
query97	6403	6581	6595	6581
query98	190	186	170	170
query99	2980	890	889	889
Total cold run time: 312108 ms
Total hot run time: 202967 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 63fadb7e41215931002fe28e51a7e436c692ea3e, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.24	0.06	0.05
query4	1.80	0.08	0.06
query5	0.53	0.52	0.52
query6	1.23	0.64	0.61
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.53	0.49	0.47
query10	0.53	0.54	0.53
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.61
query14	0.78	0.79	0.79
query15	0.78	0.75	0.77
query16	0.39	0.36	0.36
query17	0.99	1.03	1.01
query18	0.21	0.24	0.25
query19	1.95	1.85	1.84
query20	0.01	0.01	0.01
query21	15.50	0.55	0.55
query22	1.98	2.18	1.47
query23	16.96	0.96	0.96
query24	6.57	1.02	0.99
query25	0.37	0.11	0.04
query26	0.66	0.16	0.15
query27	0.05	0.04	0.04
query28	6.81	0.73	0.73
query29	12.63	2.14	2.30
query30	0.62	0.54	0.55
query31	2.80	0.39	0.37
query32	3.38	0.49	0.50
query33	3.10	3.08	3.06
query34	15.25	4.79	4.79
query35	4.84	4.84	4.81
query36	1.06	1.02	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.13
query41	0.08	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.01	0.02
Total cold run time: 103.95 s
Total hot run time: 30.46 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 63fadb7e41215931002fe28e51a7e436c692ea3e with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

@yujun777 yujun777 force-pushed the pick_32748_to_upstream_branch-2.0 branch from 63fadb7 to 8f8df91 Compare April 29, 2024 09:06
@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49632 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f8df91ec2d05deda486e0427d88822862fc6fea, data reload: false

------ Round 1 ----------------------------------
q1	17650	4364	4321	4321
q2	2036	156	142	142
q3	10341	1846	1890	1846
q4	10099	1240	1332	1240
q5	8484	3919	3952	3919
q6	229	120	123	120
q7	2049	1617	1586	1586
q8	9522	2701	2710	2701
q9	14249	10505	10405	10405
q10	8618	3555	3499	3499
q11	408	241	247	241
q12	467	299	300	299
q13	18357	4003	3988	3988
q14	349	326	312	312
q15	497	458	462	458
q16	655	572	560	560
q17	1107	953	877	877
q18	7322	6804	6925	6804
q19	1668	1576	1561	1561
q20	525	293	293	293
q21	4381	4117	4076	4076
q22	481	384	394	384
Total cold run time: 119494 ms
Total hot run time: 49632 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4284	4293	4281	4281
q2	318	221	217	217
q3	4180	4162	4098	4098
q4	2744	2738	2724	2724
q5	7170	7124	7064	7064
q6	236	117	120	117
q7	3261	2802	2843	2802
q8	4351	4479	4464	4464
q9	17497	16991	17092	16991
q10	4260	4248	4248	4248
q11	744	698	645	645
q12	1011	846	828	828
q13	6810	3727	3747	3727
q14	451	426	427	426
q15	509	454	454	454
q16	737	684	688	684
q17	3856	3897	3816	3816
q18	8826	8729	8831	8729
q19	1750	1684	1681	1681
q20	2411	2169	2136	2136
q21	8386	8519	8531	8519
q22	986	920	981	920
Total cold run time: 84778 ms
Total hot run time: 79571 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204003 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8f8df91ec2d05deda486e0427d88822862fc6fea, data reload: false

query1	916	394	376	376
query2	6522	2891	2641	2641
query3	6914	196	199	196
query4	21347	18068	18017	18017
query5	19717	6536	6502	6502
query6	296	225	236	225
query7	4158	312	306	306
query8	282	266	243	243
query9	3118	2678	2625	2625
query10	407	293	305	293
query11	11380	10682	10659	10659
query12	124	72	73	72
query13	5587	686	658	658
query14	17448	13528	13481	13481
query15	373	229	219	219
query16	6469	278	255	255
query17	1727	1442	873	873
query18	2315	411	414	411
query19	210	148	150	148
query20	72	80	75	75
query21	188	97	95	95
query22	5054	5119	4976	4976
query23	32599	31958	31915	31915
query24	7031	6636	6525	6525
query25	522	435	429	429
query26	532	163	158	158
query27	1881	301	292	292
query28	6129	2366	2322	2322
query29	2888	2879	2620	2620
query30	243	164	164	164
query31	934	738	739	738
query32	68	63	56	56
query33	400	262	252	252
query34	841	468	472	468
query35	1112	936	959	936
query36	1278	1237	1249	1237
query37	88	62	64	62
query38	3079	2900	2921	2900
query39	1374	1330	1319	1319
query40	196	85	95	85
query41	44	36	34	34
query42	88	88	79	79
query43	664	666	720	666
query44	1112	733	734	733
query45	242	229	230	229
query46	1216	941	968	941
query47	1863	1724	1976	1724
query48	979	715	723	715
query49	618	373	390	373
query50	871	602	596	596
query51	4815	4661	4657	4657
query52	90	72	82	72
query53	440	322	313	313
query54	2615	2416	2454	2416
query55	82	79	78	78
query56	230	217	230	217
query57	1206	1107	1091	1091
query58	219	198	202	198
query59	4120	3891	3763	3763
query60	211	190	187	187
query61	91	83	85	83
query62	804	495	436	436
query63	472	339	351	339
query64	2556	1603	1366	1366
query65	3674	3567	3569	3567
query66	824	377	391	377
query67	17579	16331	15437	15437
query68	10170	655	662	655
query69	555	332	363	332
query70	1982	1381	1449	1381
query71	412	288	315	288
query72	6473	3468	3429	3429
query73	735	322	321	321
query74	6253	5867	5873	5867
query75	5407	3635	3696	3635
query76	6563	1135	1174	1135
query77	1127	250	254	250
query78	12937	12480	19078	12480
query79	6952	627	641	627
query80	609	408	389	389
query81	489	235	232	232
query82	244	90	108	90
query83	177	145	130	130
query84	261	71	73	71
query85	782	298	296	296
query86	346	303	286	286
query87	3241	3001	3081	3001
query88	3599	2292	2307	2292
query89	337	299	273	273
query90	1984	216	216	216
query91	178	129	145	129
query92	60	59	53	53
query93	978	580	587	580
query94	699	211	209	209
query95	1125	1061	1063	1061
query96	644	318	320	318
query97	6554	6479	6516	6479
query98	173	168	168	168
query99	2947	838	819	819
Total cold run time: 308537 ms
Total hot run time: 204003 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8f8df91ec2d05deda486e0427d88822862fc6fea, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.02
query3	0.24	0.05	0.04
query4	1.78	0.06	0.06
query5	0.52	0.53	0.52
query6	1.23	0.62	0.61
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.52	0.48	0.48
query10	0.53	0.53	0.55
query11	0.13	0.09	0.08
query12	0.11	0.09	0.10
query13	0.61	0.61	0.61
query14	0.79	0.78	0.80
query15	0.78	0.75	0.77
query16	0.37	0.37	0.37
query17	1.01	0.98	1.01
query18	0.21	0.28	0.21
query19	1.96	1.87	1.87
query20	0.01	0.01	0.01
query21	15.46	0.55	0.54
query22	2.27	2.10	1.77
query23	17.36	0.97	0.95
query24	6.53	1.38	1.39
query25	0.37	0.10	0.07
query26	0.69	0.15	0.15
query27	0.04	0.03	0.04
query28	6.02	0.73	0.73
query29	12.62	2.38	2.29
query30	0.59	0.52	0.54
query31	2.82	0.39	0.38
query32	3.37	0.50	0.49
query33	3.10	3.07	3.07
query34	15.24	4.83	4.82
query35	4.90	4.85	4.81
query36	1.05	1.03	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.03	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.76 s
Total hot run time: 31.32 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8f8df91ec2d05deda486e0427d88822862fc6fea with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@yujun777 yujun777 force-pushed the pick_32748_to_upstream_branch-2.0 branch from 8f8df91 to 4e5d1f3 Compare May 8, 2024 02:32
@yujun777
Copy link
Collaborator Author

yujun777 commented May 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49402 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4e5d1f39d95098e5830126801f146efb2390bce8, data reload: false

------ Round 1 ----------------------------------
q1	17742	4365	4358	4358
q2	2035	150	143	143
q3	10440	1901	1912	1901
q4	10380	1230	1309	1230
q5	8379	3918	3914	3914
q6	235	122	121	121
q7	2015	1595	1587	1587
q8	9271	2714	2718	2714
q9	10568	10463	10198	10198
q10	8662	3502	3505	3502
q11	422	248	246	246
q12	470	306	302	302
q13	18355	3946	4075	3946
q14	350	317	321	317
q15	506	447	458	447
q16	678	561	564	561
q17	1116	970	939	939
q18	7193	6835	6733	6733
q19	1705	1552	1530	1530
q20	545	307	297	297
q21	4460	4073	4031	4031
q22	502	385	394	385
Total cold run time: 116029 ms
Total hot run time: 49402 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4269	4284	4279	4279
q2	317	224	218	218
q3	4143	4109	4134	4109
q4	2748	2752	2733	2733
q5	7081	7053	7088	7053
q6	237	117	115	115
q7	3224	2872	2787	2787
q8	4371	4438	4461	4438
q9	16855	16655	16758	16655
q10	4211	4267	4260	4260
q11	769	671	698	671
q12	1029	847	824	824
q13	6685	3756	3736	3736
q14	450	411	415	411
q15	503	453	463	453
q16	729	687	724	687
q17	3687	3847	3744	3744
q18	8831	8702	8688	8688
q19	1718	1725	1670	1670
q20	2362	2121	2151	2121
q21	8467	8374	8416	8374
q22	1012	923	929	923
Total cold run time: 83698 ms
Total hot run time: 78949 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202818 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4e5d1f39d95098e5830126801f146efb2390bce8, data reload: false

query1	928	387	379	379
query2	6543	2917	2908	2908
query3	6919	196	204	196
query4	20754	18089	17955	17955
query5	19717	6473	6545	6473
query6	287	213	239	213
query7	4150	306	305	305
query8	253	275	269	269
query9	3144	2699	2617	2617
query10	418	266	298	266
query11	11315	10680	10685	10680
query12	119	72	71	71
query13	5589	698	680	680
query14	17445	13412	12917	12917
query15	360	225	215	215
query16	6469	266	252	252
query17	1738	1439	888	888
query18	2301	408	416	408
query19	211	154	148	148
query20	78	73	79	73
query21	183	95	93	93
query22	5308	5039	5069	5039
query23	32587	31632	31752	31632
query24	7025	6596	6491	6491
query25	507	427	424	424
query26	526	166	167	166
query27	1875	302	303	302
query28	6214	2402	2349	2349
query29	2865	2646	2757	2646
query30	238	158	156	156
query31	882	726	746	726
query32	70	63	55	55
query33	398	255	239	239
query34	854	476	476	476
query35	1115	940	902	902
query36	1272	1175	1235	1175
query37	88	58	59	58
query38	3114	2912	2955	2912
query39	1355	1325	1304	1304
query40	201	89	91	89
query41	37	40	35	35
query42	84	90	80	80
query43	704	612	703	612
query44	1138	727	730	727
query45	246	223	227	223
query46	1223	975	956	956
query47	1999	1758	1903	1758
query48	990	724	695	695
query49	615	365	375	365
query50	859	609	607	607
query51	4750	4611	4686	4611
query52	93	80	80	80
query53	441	324	320	320
query54	2654	2473	2446	2446
query55	89	80	91	80
query56	216	201	197	197
query57	1173	1061	1130	1061
query58	210	214	214	214
query59	4390	4181	4086	4086
query60	203	202	219	202
query61	92	86	84	84
query62	853	484	503	484
query63	467	335	335	335
query64	2262	1534	1436	1436
query65	3629	3547	3552	3547
query66	770	367	381	367
query67	15888	16991	15791	15791
query68	8452	651	666	651
query69	562	354	331	331
query70	1570	1301	1291	1291
query71	401	303	314	303
query72	6481	3427	3468	3427
query73	733	332	334	332
query74	6305	5793	5876	5793
query75	4487	3752	3647	3647
query76	4728	1172	1215	1172
query77	625	247	254	247
query78	12446	11769	11519	11519
query79	9101	667	659	659
query80	1675	400	401	400
query81	498	233	227	227
query82	1667	100	97	97
query83	165	130	128	128
query84	260	73	70	70
query85	871	301	294	294
query86	329	302	290	290
query87	3260	2982	2989	2982
query88	5189	2309	2310	2309
query89	392	276	311	276
query90	1928	196	195	195
query91	167	134	132	132
query92	56	58	52	52
query93	6543	559	561	559
query94	722	200	204	200
query95	1122	1053	1066	1053
query96	648	322	340	322
query97	6409	6338	6354	6338
query98	190	176	168	168
query99	2981	808	951	808
Total cold run time: 312453 ms
Total hot run time: 202818 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.38 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4e5d1f39d95098e5830126801f146efb2390bce8, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.04
query4	1.80	0.06	0.07
query5	0.53	0.54	0.51
query6	1.26	0.64	0.62
query7	0.02	0.01	0.01
query8	0.02	0.02	0.02
query9	0.52	0.48	0.49
query10	0.54	0.54	0.54
query11	0.13	0.08	0.09
query12	0.12	0.09	0.09
query13	0.61	0.63	0.61
query14	0.80	0.76	0.82
query15	0.78	0.76	0.75
query16	0.37	0.37	0.37
query17	1.02	0.98	0.98
query18	0.21	0.27	0.25
query19	1.92	1.85	1.87
query20	0.02	0.02	0.02
query21	15.49	0.54	0.53
query22	1.99	2.12	2.08
query23	17.35	0.97	0.90
query24	4.21	1.74	1.33
query25	0.38	0.14	0.05
query26	0.55	0.16	0.16
query27	0.03	0.04	0.03
query28	7.86	0.74	0.73
query29	12.65	2.23	2.28
query30	0.60	0.54	0.55
query31	2.79	0.40	0.39
query32	3.38	0.50	0.49
query33	3.07	3.03	3.12
query34	15.24	4.76	4.79
query35	4.87	4.78	4.84
query36	1.10	1.02	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 102.95 s
Total hot run time: 31.38 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4e5d1f39d95098e5830126801f146efb2390bce8 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.0 seconds inserted 10000000 Rows, about 476K ops/s

@yujun777
Copy link
Collaborator Author

yujun777 commented May 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49417 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fddf6456d2dbe43d18635802dee024a051d5a46b, data reload: false

------ Round 1 ----------------------------------
q1	17803	4405	4352	4352
q2	2025	147	137	137
q3	10292	1864	1897	1864
q4	10406	1253	1320	1253
q5	8413	3859	3893	3859
q6	224	122	125	122
q7	2014	1559	1599	1559
q8	9291	2710	2706	2706
q9	11005	10308	10420	10308
q10	8635	3526	3479	3479
q11	416	233	245	233
q12	458	304	297	297
q13	18383	3983	4021	3983
q14	348	332	319	319
q15	492	454	451	451
q16	685	569	568	568
q17	1114	955	943	943
q18	7227	6766	6923	6766
q19	1695	1574	1503	1503
q20	526	302	286	286
q21	4453	4102	4037	4037
q22	476	392	393	392
Total cold run time: 116381 ms
Total hot run time: 49417 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4308	4259	4281	4259
q2	315	218	225	218
q3	4139	4134	4129	4129
q4	2756	2716	2750	2716
q5	7190	7062	7092	7062
q6	230	119	117	117
q7	3205	2798	2803	2798
q8	4353	4464	4468	4464
q9	17229	17184	16873	16873
q10	4228	4306	4288	4288
q11	745	677	675	675
q12	1017	874	850	850
q13	6945	3746	3718	3718
q14	449	429	418	418
q15	494	459	456	456
q16	740	696	678	678
q17	3753	3848	3824	3824
q18	8672	8818	8679	8679
q19	1717	1690	1639	1639
q20	2378	2147	2131	2131
q21	8482	8494	8478	8478
q22	1012	974	957	957
Total cold run time: 84357 ms
Total hot run time: 79427 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202212 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fddf6456d2dbe43d18635802dee024a051d5a46b, data reload: false

query1	916	390	380	380
query2	6521	2892	2654	2654
query3	6915	201	196	196
query4	20024	17979	17995	17979
query5	19719	6451	6552	6451
query6	283	214	222	214
query7	4149	295	304	295
query8	250	250	261	250
query9	3110	2638	2564	2564
query10	407	288	298	288
query11	11355	10603	10715	10603
query12	122	68	70	68
query13	5576	678	691	678
query14	17417	13373	13404	13373
query15	380	216	218	216
query16	6470	275	256	256
query17	1759	1470	878	878
query18	2316	398	403	398
query19	209	147	154	147
query20	76	76	75	75
query21	183	101	93	93
query22	5274	4991	5133	4991
query23	32365	31839	31951	31839
query24	6960	6443	6459	6443
query25	526	402	420	402
query26	610	163	153	153
query27	2034	295	291	291
query28	6152	2361	2345	2345
query29	2949	2668	2634	2634
query30	238	161	161	161
query31	906	745	724	724
query32	65	59	60	59
query33	396	243	250	243
query34	844	452	476	452
query35	1120	895	948	895
query36	1267	1284	1250	1250
query37	91	61	58	58
query38	3085	2901	2950	2901
query39	1350	1319	1323	1319
query40	192	94	98	94
query41	43	40	35	35
query42	81	84	84	84
query43	768	646	660	646
query44	1122	723	720	720
query45	238	226	224	224
query46	1226	965	975	965
query47	1953	1605	1740	1605
query48	1037	711	696	696
query49	616	362	389	362
query50	859	604	597	597
query51	4791	4610	4653	4610
query52	83	72	76	72
query53	438	310	314	310
query54	2647	2452	2494	2452
query55	85	78	82	78
query56	214	207	214	207
query57	1312	1153	1093	1093
query58	215	208	205	205
query59	3970	4012	3856	3856
query60	200	185	196	185
query61	91	87	85	85
query62	811	508	476	476
query63	469	329	330	329
query64	2546	1518	1371	1371
query65	3642	3522	3537	3522
query66	817	378	373	373
query67	15981	15252	15869	15252
query68	8568	653	651	651
query69	562	347	340	340
query70	1606	1452	1285	1285
query71	391	302	300	300
query72	6616	3442	3412	3412
query73	730	315	312	312
query74	6256	5869	5931	5869
query75	4544	3772	3679	3679
query76	4730	1143	1173	1143
query77	658	238	253	238
query78	12829	11677	11684	11677
query79	8650	639	641	639
query80	1414	397	397	397
query81	511	232	226	226
query82	1741	98	95	95
query83	167	134	132	132
query84	254	68	66	66
query85	925	291	291	291
query86	352	296	299	296
query87	3237	2959	3041	2959
query88	5162	2305	2317	2305
query89	378	279	280	279
query90	1882	210	212	210
query91	166	130	131	130
query92	56	57	47	47
query93	5853	557	543	543
query94	761	202	203	202
query95	1110	1082	1049	1049
query96	654	320	324	320
query97	6553	6367	6517	6367
query98	189	169	171	169
query99	2885	864	885	864
Total cold run time: 311226 ms
Total hot run time: 202212 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.97 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fddf6456d2dbe43d18635802dee024a051d5a46b, data reload: false

query1	0.02	0.03	0.03
query2	0.07	0.02	0.02
query3	0.24	0.05	0.05
query4	1.81	0.06	0.07
query5	0.53	0.53	0.52
query6	1.25	0.63	0.62
query7	0.02	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.48	0.49
query10	0.52	0.53	0.55
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.63	0.62	0.62
query14	0.78	0.79	0.79
query15	0.78	0.76	0.75
query16	0.36	0.38	0.37
query17	1.02	1.02	0.98
query18	0.22	0.26	0.25
query19	1.94	1.76	1.82
query20	0.02	0.02	0.02
query21	15.45	0.54	0.55
query22	2.09	2.53	1.92
query23	17.36	1.12	0.79
query24	4.87	1.52	1.19
query25	0.37	0.06	0.05
query26	0.66	0.17	0.15
query27	0.05	0.04	0.04
query28	7.58	0.74	0.77
query29	12.78	2.37	2.20
query30	0.66	0.53	0.50
query31	2.81	0.39	0.38
query32	3.36	0.49	0.49
query33	3.09	3.15	3.05
query34	15.27	4.79	4.82
query35	4.90	4.83	4.84
query36	1.05	1.02	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.07	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.01	0.01
Total cold run time: 103.75 s
Total hot run time: 30.97 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fddf6456d2dbe43d18635802dee024a051d5a46b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@yujun777 yujun777 force-pushed the pick_32748_to_upstream_branch-2.0 branch from fddf645 to 3458926 Compare May 8, 2024 08:54
@yujun777
Copy link
Collaborator Author

yujun777 commented May 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49780 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3458926926ea809e3e8905879f1e163734a9c025, data reload: false

------ Round 1 ----------------------------------
q1	17687	4475	4345	4345
q2	2098	148	143	143
q3	10395	1902	1923	1902
q4	10117	1289	1327	1289
q5	8552	3874	3921	3874
q6	230	124	123	123
q7	2047	1568	1605	1568
q8	9285	2720	2725	2720
q9	10691	10393	10188	10188
q10	8612	3539	3531	3531
q11	417	241	239	239
q12	477	314	297	297
q13	18374	3995	4027	3995
q14	357	331	311	311
q15	518	469	453	453
q16	674	589	587	587
q17	1144	982	958	958
q18	7244	6915	6922	6915
q19	1713	1630	1565	1565
q20	540	318	314	314
q21	4467	4115	4072	4072
q22	497	391	398	391
Total cold run time: 116136 ms
Total hot run time: 49780 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4333	4289	4336	4289
q2	321	229	219	219
q3	4171	4159	4172	4159
q4	2768	2759	2772	2759
q5	7163	7112	7144	7112
q6	237	122	120	120
q7	3252	2879	2889	2879
q8	4403	4473	4486	4473
q9	16958	16826	17438	16826
q10	4253	4274	4385	4274
q11	746	707	698	698
q12	1020	854	872	854
q13	6884	3723	3741	3723
q14	455	438	418	418
q15	506	457	452	452
q16	735	695	676	676
q17	3834	3880	3904	3880
q18	8853	8707	8842	8707
q19	1710	1737	1672	1672
q20	2395	2137	2114	2114
q21	8479	8548	8488	8488
q22	1026	948	967	948
Total cold run time: 84502 ms
Total hot run time: 79740 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203264 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3458926926ea809e3e8905879f1e163734a9c025, data reload: false

query1	940	389	382	382
query2	6542	2853	2687	2687
query3	6918	206	203	203
query4	20885	18078	18089	18078
query5	19740	6509	6500	6500
query6	284	214	226	214
query7	4158	301	307	301
query8	255	245	238	238
query9	3136	2687	2604	2604
query10	420	299	329	299
query11	11257	10626	10724	10626
query12	120	70	80	70
query13	5581	703	684	684
query14	17401	13794	13486	13486
query15	384	237	223	223
query16	6441	277	256	256
query17	1726	1444	864	864
query18	2322	405	403	403
query19	200	140	142	140
query20	80	80	74	74
query21	190	92	88	88
query22	5217	4928	4983	4928
query23	32629	31873	31929	31873
query24	7000	6531	6505	6505
query25	515	436	410	410
query26	526	173	160	160
query27	1870	292	291	291
query28	6191	2381	2328	2328
query29	2856	2760	2715	2715
query30	245	166	164	164
query31	902	752	738	738
query32	69	66	58	58
query33	402	258	238	238
query34	854	461	479	461
query35	1120	936	872	872
query36	1347	1404	1331	1331
query37	90	64	61	61
query38	3107	2956	2942	2942
query39	1370	1322	1339	1322
query40	204	90	98	90
query41	37	35	34	34
query42	81	86	86	86
query43	895	643	661	643
query44	1155	703	698	698
query45	242	236	228	228
query46	1232	951	947	947
query47	1962	1727	1832	1727
query48	1021	725	695	695
query49	627	359	380	359
query50	873	604	600	600
query51	4807	4675	4682	4675
query52	89	78	76	76
query53	455	314	318	314
query54	2643	2474	2492	2474
query55	90	89	84	84
query56	227	204	202	202
query57	1280	1068	1074	1068
query58	210	202	190	190
query59	3898	4182	3748	3748
query60	202	195	183	183
query61	87	81	83	81
query62	783	497	544	497
query63	479	334	332	332
query64	2429	1479	1459	1459
query65	3649	3568	3550	3550
query66	742	373	382	373
query67	16617	15586	15402	15402
query68	10127	649	665	649
query69	562	335	340	335
query70	1932	1288	1412	1288
query71	419	309	314	309
query72	6415	3408	3412	3408
query73	729	321	320	320
query74	6284	5812	5805	5805
query75	5377	3671	3717	3671
query76	6478	1134	1165	1134
query77	1099	259	252	252
query78	12492	12306	11790	11790
query79	11594	632	642	632
query80	710	384	384	384
query81	491	228	227	227
query82	854	99	96	96
query83	173	136	131	131
query84	254	69	70	69
query85	809	296	299	296
query86	355	341	301	301
query87	3283	3061	3050	3050
query88	5108	2343	2353	2343
query89	418	293	281	281
query90	1989	214	210	210
query91	174	148	134	134
query92	59	55	60	55
query93	5189	555	544	544
query94	719	202	209	202
query95	1111	1049	1055	1049
query96	651	333	328	328
query97	6519	6332	6397	6332
query98	188	181	176	176
query99	3006	909	831	831
Total cold run time: 317874 ms
Total hot run time: 203264 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.85 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3458926926ea809e3e8905879f1e163734a9c025, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.80	0.10	0.10
query5	0.53	0.53	0.51
query6	1.30	0.62	0.60
query7	0.01	0.00	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.47
query10	0.55	0.53	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.10
query13	0.62	0.62	0.61
query14	0.78	0.80	0.81
query15	0.79	0.76	0.74
query16	0.35	0.37	0.35
query17	1.02	1.00	1.01
query18	0.23	0.27	0.25
query19	1.84	1.88	1.79
query20	0.02	0.02	0.01
query21	15.47	0.53	0.54
query22	2.41	2.35	1.99
query23	17.16	1.05	0.89
query24	5.24	2.49	1.69
query25	0.39	0.06	0.07
query26	0.82	0.16	0.15
query27	0.05	0.04	0.04
query28	5.82	0.77	0.80
query29	12.68	2.37	2.35
query30	0.59	0.56	0.53
query31	2.81	0.40	0.38
query32	3.40	0.50	0.49
query33	3.06	3.05	3.04
query34	15.26	4.81	4.80
query35	4.88	4.85	4.84
query36	1.06	1.02	1.03
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.13
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 102.48 s
Total hot run time: 31.85 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3458926926ea809e3e8905879f1e163734a9c025 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@yujun777
Copy link
Collaborator Author

yujun777 commented May 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50349 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f9406e16f0e41c7c68d5bf8ddadef60ea475a573, data reload: false

------ Round 1 ----------------------------------
q1	17536	4394	4318	4318
q2	2023	153	142	142
q3	10278	1939	1953	1939
q4	10104	1262	1321	1262
q5	8494	4053	3936	3936
q6	233	125	122	122
q7	2043	1601	1598	1598
q8	9259	2738	2732	2732
q9	10766	11060	10729	10729
q10	8634	3550	3558	3550
q11	432	238	244	238
q12	460	298	307	298
q13	18361	4029	4044	4029
q14	353	331	338	331
q15	507	472	464	464
q16	678	572	570	570
q17	1128	963	974	963
q18	7377	6767	6956	6767
q19	1759	1613	1562	1562
q20	522	310	308	308
q21	4487	4133	4111	4111
q22	499	397	380	380
Total cold run time: 115933 ms
Total hot run time: 50349 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4327	4315	4335	4315
q2	320	220	217	217
q3	4190	4191	4150	4150
q4	2762	2753	2754	2753
q5	7219	7110	7076	7076
q6	236	127	121	121
q7	3259	2875	2805	2805
q8	4347	4454	4520	4454
q9	16833	16897	16697	16697
q10	4236	4253	4274	4253
q11	755	678	680	678
q12	1041	864	866	864
q13	7022	3769	3782	3769
q14	447	436	425	425
q15	509	455	460	455
q16	732	681	676	676
q17	3856	3813	3896	3813
q18	8802	8754	8791	8754
q19	1755	1726	1652	1652
q20	2384	2180	2110	2110
q21	8550	8484	8597	8484
q22	1016	920	956	920
Total cold run time: 84598 ms
Total hot run time: 79441 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202673 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f9406e16f0e41c7c68d5bf8ddadef60ea475a573, data reload: false

query1	918	393	375	375
query2	6524	2789	2671	2671
query3	6917	205	201	201
query4	20126	17987	17934	17934
query5	19727	6478	6528	6478
query6	283	209	212	209
query7	4152	311	302	302
query8	248	223	238	223
query9	3146	2749	2642	2642
query10	402	274	307	274
query11	11350	10655	10740	10655
query12	115	73	70	70
query13	5589	684	687	684
query14	17614	13114	13456	13114
query15	362	216	223	216
query16	6467	269	255	255
query17	1578	1479	877	877
query18	2251	402	397	397
query19	204	143	151	143
query20	79	76	77	76
query21	186	90	90	90
query22	5220	5019	4991	4991
query23	32383	31958	31984	31958
query24	6988	6590	6499	6499
query25	506	426	419	419
query26	524	161	159	159
query27	1896	294	291	291
query28	6120	2382	2316	2316
query29	2865	2701	2638	2638
query30	235	161	162	161
query31	898	725	719	719
query32	63	61	57	57
query33	406	253	254	253
query34	852	462	487	462
query35	1096	952	898	898
query36	1407	1409	1296	1296
query37	95	64	62	62
query38	3067	2902	2924	2902
query39	1357	1346	1330	1330
query40	211	95	95	95
query41	39	36	38	36
query42	91	85	88	85
query43	787	760	682	682
query44	1126	714	722	714
query45	241	229	225	225
query46	1226	960	965	960
query47	1903	1761	1625	1625
query48	1014	707	702	702
query49	618	367	373	367
query50	853	632	607	607
query51	4727	4712	4681	4681
query52	78	73	72	72
query53	441	321	317	317
query54	2661	2446	2448	2446
query55	99	78	88	78
query56	220	229	210	210
query57	1179	1100	1087	1087
query58	215	193	197	193
query59	4162	3734	3962	3734
query60	198	217	195	195
query61	85	81	87	81
query62	763	503	487	487
query63	469	333	335	333
query64	2521	1482	1454	1454
query65	3642	3568	3570	3568
query66	797	375	370	370
query67	16716	17123	15539	15539
query68	8755	662	658	658
query69	569	342	352	342
query70	1582	1403	1487	1403
query71	397	303	314	303
query72	6535	3400	3448	3400
query73	730	325	321	321
query74	6440	5910	5832	5832
query75	5279	3688	3675	3675
query76	5350	1142	1110	1110
query77	900	259	244	244
query78	12594	11551	11739	11551
query79	8060	644	648	644
query80	1354	387	385	385
query81	492	235	237	235
query82	1696	98	101	98
query83	181	129	129	129
query84	264	69	66	66
query85	859	292	293	292
query86	339	294	292	292
query87	3192	2968	3039	2968
query88	4780	2365	2341	2341
query89	390	294	299	294
query90	1954	205	217	205
query91	180	135	133	133
query92	57	52	49	49
query93	6397	574	564	564
query94	708	205	207	205
query95	1099	1036	1054	1036
query96	650	332	328	328
query97	6538	6400	6316	6316
query98	191	173	171	171
query99	2970	925	843	843
Total cold run time: 312780 ms
Total hot run time: 202673 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.15 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f9406e16f0e41c7c68d5bf8ddadef60ea475a573, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.05
query4	1.80	0.10	0.09
query5	0.53	0.52	0.52
query6	1.23	0.61	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.47
query10	0.53	0.53	0.53
query11	0.11	0.09	0.09
query12	0.12	0.08	0.08
query13	0.62	0.62	0.62
query14	0.77	0.79	0.79
query15	0.79	0.76	0.76
query16	0.39	0.36	0.36
query17	0.98	1.03	1.00
query18	0.24	0.26	0.24
query19	1.92	1.85	1.86
query20	0.01	0.01	0.01
query21	15.49	0.57	0.52
query22	2.25	2.09	1.60
query23	16.94	0.99	1.12
query24	6.09	1.46	1.33
query25	0.36	0.15	0.06
query26	0.72	0.15	0.15
query27	0.04	0.04	0.03
query28	6.28	0.76	0.78
query29	12.61	2.29	2.32
query30	0.56	0.50	0.48
query31	2.82	0.37	0.36
query32	3.41	0.50	0.49
query33	3.11	3.07	3.10
query34	15.23	4.81	4.79
query35	4.86	4.85	4.85
query36	1.07	1.01	1.00
query37	0.06	0.04	0.05
query38	0.03	0.02	0.03
query39	0.02	0.01	0.01
query40	0.15	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.14 s
Total hot run time: 31.15 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f9406e16f0e41c7c68d5bf8ddadef60ea475a573 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@yujun777 yujun777 marked this pull request as ready for review May 9, 2024 06:56
@dataroaring dataroaring merged commit 7ed116b into apache:branch-2.0 May 10, 2024
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@yujun777 @doris-robot @dataroaring