Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](fe ut) fix TabletRepairAndBalanceTest #31397

Merged
merged 6 commits into from
Feb 28, 2024

Conversation

yujun777
Copy link
Collaborator

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41483 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3320667139635a0d43effb881fd00a118a7f4e20, data reload: false

------ Round 1 ----------------------------------
q1	18247	5283	5138	5138
q2	2794	150	143	143
q3	12309	1036	1026	1026
q4	5228	1004	1023	1004
q5	7872	3155	3202	3155
q6	196	135	132	132
q7	1267	791	774	774
q8	9254	2082	2068	2068
q9	7458	6619	6607	6607
q10	8327	2633	2628	2628
q11	405	205	204	204
q12	741	323	328	323
q13	17919	3571	3612	3571
q14	307	253	248	248
q15	593	512	495	495
q16	470	407	418	407
q17	920	871	859	859
q18	7167	6780	6631	6631
q19	1550	1502	1491	1491
q20	481	274	283	274
q21	6093	4018	3967	3967
q22	578	352	338	338
Total cold run time: 110176 ms
Total hot run time: 41483 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4885	4846	4872	4846
q2	300	185	191	185
q3	3623	3597	3595	3595
q4	2543	2528	2548	2528
q5	5787	5770	5741	5741
q6	216	128	129	128
q7	2257	1653	1690	1653
q8	3044	3108	3131	3108
q9	8743	8726	8727	8726
q10	4352	4248	4239	4239
q11	537	365	379	365
q12	806	552	546	546
q13	4210	3433	3422	3422
q14	278	238	246	238
q15	599	503	508	503
q16	495	465	463	463
q17	1694	1624	1610	1610
q18	8326	7659	7672	7659
q19	1630	1645	1649	1645
q20	2106	1834	1845	1834
q21	6595	6232	6175	6175
q22	589	508	522	508
Total cold run time: 63615 ms
Total hot run time: 59717 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 176622 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3320667139635a0d43effb881fd00a118a7f4e20, data reload: false

query1	922	349	345	345
query2	6519	1770	1733	1733
query3	6725	208	205	205
query4	22925	21235	21044	21044
query5	4336	379	382	379
query6	253	167	167	167
query7	4609	291	302	291
query8	254	196	204	196
query9	8446	2863	2855	2855
query10	408	221	224	221
query11	14937	14565	14400	14400
query12	139	82	85	82
query13	1705	429	420	420
query14	9375	7740	7743	7740
query15	211	186	192	186
query16	7544	254	257	254
query17	1404	552	526	526
query18	1945	270	267	267
query19	196	142	152	142
query20	87	82	81	81
query21	187	119	126	119
query22	4881	4725	4675	4675
query23	32576	31463	31451	31451
query24	12671	3384	3479	3384
query25	619	364	362	362
query26	1814	159	167	159
query27	3035	317	316	316
query28	6584	1822	1826	1822
query29	1097	632	633	632
query30	274	135	146	135
query31	907	752	763	752
query32	97	59	50	50
query33	708	242	249	242
query34	1062	503	495	495
query35	919	844	826	826
query36	987	913	876	876
query37	144	60	63	60
query38	3258	3178	3182	3178
query39	1399	1332	1305	1305
query40	284	108	109	108
query41	39	37	36	36
query42	105	99	97	97
query43	471	440	471	440
query44	1066	695	701	695
query45	201	184	174	174
query46	1046	764	754	754
query47	1636	1520	1528	1520
query48	417	347	345	345
query49	1164	309	298	298
query50	761	371	377	371
query51	4490	4366	4304	4304
query52	104	96	94	94
query53	393	305	310	305
query54	304	225	231	225
query55	84	89	83	83
query56	224	206	201	201
query57	1039	916	968	916
query58	224	200	198	198
query59	2253	2173	2074	2074
query60	250	214	211	211
query61	86	90	84	84
query62	581	378	392	378
query63	340	308	304	304
query64	6313	3086	3124	3086
query65	3296	3282	3241	3241
query66	1335	328	322	322
query67	14616	14500	14221	14221
query68	5106	549	551	549
query69	522	368	369	368
query70	1230	1262	1273	1262
query71	430	257	255	255
query72	6349	2803	2625	2625
query73	692	311	312	311
query74	6871	6469	6471	6469
query75	3214	2567	2597	2567
query76	3323	1092	1197	1092
query77	350	243	245	243
query78	9481	8793	8753	8753
query79	979	523	499	499
query80	514	356	368	356
query81	435	202	205	202
query82	243	83	86	83
query83	143	121	125	121
query84	222	79	80	79
query85	1019	357	345	345
query86	291	315	291	291
query87	3467	3277	3255	3255
query88	2718	2311	2301	2301
query89	451	368	368	368
query90	1847	170	172	170
query91	153	132	127	127
query92	60	50	50	50
query93	1006	502	488	488
query94	1111	177	178	177
query95	435	338	343	338
query96	573	281	263	263
query97	4428	4323	4257	4257
query98	216	209	198	198
query99	1095	702	728	702
Total cold run time: 268255 ms
Total hot run time: 176622 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3320667139635a0d43effb881fd00a118a7f4e20, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.02
query3	0.23	0.07	0.07
query4	1.65	0.07	0.08
query5	0.48	0.49	0.47
query6	1.37	0.62	0.62
query7	0.02	0.01	0.02
query8	0.04	0.02	0.02
query9	0.54	0.46	0.46
query10	0.49	0.50	0.50
query11	0.13	0.10	0.09
query12	0.12	0.10	0.10
query13	0.60	0.59	0.59
query14	0.79	0.80	0.78
query15	0.83	0.80	0.81
query16	0.33	0.33	0.32
query17	0.93	0.89	0.89
query18	0.18	0.18	0.18
query19	1.78	1.72	1.65
query20	0.01	0.01	0.02
query21	15.41	0.67	0.60
query22	3.23	3.41	2.96
query23	17.68	1.04	0.94
query24	1.99	0.58	0.56
query25	0.63	0.05	0.06
query26	0.15	0.14	0.15
query27	0.05	0.06	0.05
query28	11.88	0.83	0.84
query29	12.58	3.44	3.31
query30	0.52	0.50	0.47
query31	2.80	0.35	0.36
query32	3.35	0.48	0.47
query33	3.15	3.13	3.15
query34	15.35	4.50	4.54
query35	4.51	4.50	4.50
query36	1.06	0.94	0.94
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.17	0.14	0.15
query41	0.08	0.02	0.01
query42	0.03	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.38 s
Total hot run time: 31.73 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3320667139635a0d43effb881fd00a118a7f4e20 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.2 seconds inserted 10000000 Rows, about 581K ops/s

@yujun777 yujun777 marked this pull request as draft February 26, 2024 06:13
@yujun777 yujun777 marked this pull request as ready for review February 26, 2024 07:52
@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41458 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 67e0f0c1af47a32d9768b4c0aa4cdccae88e0569, data reload: false

------ Round 1 ----------------------------------
q1	18230	5236	5150	5150
q2	2417	161	135	135
q3	11480	1046	1053	1046
q4	4969	986	986	986
q5	7642	3160	3202	3160
q6	197	139	138	138
q7	1257	791	771	771
q8	9246	2068	2048	2048
q9	7558	6583	6555	6555
q10	8328	2653	2638	2638
q11	408	229	209	209
q12	795	340	333	333
q13	17947	3616	3620	3616
q14	297	260	258	258
q15	610	524	505	505
q16	480	418	407	407
q17	926	870	862	862
q18	7383	6682	6649	6649
q19	1535	1490	1495	1490
q20	554	287	271	271
q21	6460	3900	3951	3900
q22	863	336	331	331
Total cold run time: 109582 ms
Total hot run time: 41458 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4856	4902	4868	4868
q2	292	187	189	187
q3	3594	3592	3602	3592
q4	2570	2541	2540	2540
q5	5761	5759	5774	5759
q6	215	126	129	126
q7	2250	1673	1688	1673
q8	3061	3074	3125	3074
q9	8762	8760	8712	8712
q10	6920	4245	4247	4245
q11	498	378	392	378
q12	770	567	560	560
q13	4215	3411	3416	3411
q14	276	231	238	231
q15	590	510	500	500
q16	486	434	462	434
q17	1670	1617	1613	1613
q18	8348	7706	7662	7662
q19	1639	1630	1635	1630
q20	2108	1859	1845	1845
q21	6725	6282	6259	6259
q22	582	496	517	496
Total cold run time: 66188 ms
Total hot run time: 59795 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175937 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 67e0f0c1af47a32d9768b4c0aa4cdccae88e0569, data reload: false

query1	942	353	347	347
query2	6513	1718	1695	1695
query3	6694	205	203	203
query4	23023	21026	21107	21026
query5	4305	368	367	367
query6	254	163	163	163
query7	4600	300	294	294
query8	246	193	201	193
query9	8435	2801	2787	2787
query10	416	211	221	211
query11	14841	14445	14386	14386
query12	137	82	85	82
query13	1688	418	421	418
query14	8920	7638	7677	7638
query15	211	188	187	187
query16	7102	257	264	257
query17	1159	544	516	516
query18	2092	264	263	263
query19	199	145	145	145
query20	86	83	82	82
query21	189	123	119	119
query22	4971	4697	4804	4697
query23	32499	31550	31563	31550
query24	12638	3368	3398	3368
query25	641	360	355	355
query26	1871	152	163	152
query27	3008	308	299	299
query28	6720	1822	1808	1808
query29	1234	605	613	605
query30	281	138	147	138
query31	905	748	757	748
query32	94	58	54	54
query33	726	233	221	221
query34	1057	494	497	494
query35	905	803	825	803
query36	1006	936	888	888
query37	262	66	64	64
query38	3309	3172	3178	3172
query39	1367	1308	1310	1308
query40	287	107	103	103
query41	39	38	35	35
query42	102	98	99	98
query43	504	451	437	437
query44	1077	684	697	684
query45	195	188	176	176
query46	1030	796	765	765
query47	1633	1482	1555	1482
query48	424	352	347	347
query49	1202	303	300	300
query50	760	371	366	366
query51	4406	4319	4290	4290
query52	105	97	91	91
query53	401	302	304	302
query54	292	219	224	219
query55	87	81	84	81
query56	223	199	193	193
query57	1040	942	909	909
query58	207	195	194	194
query59	2249	2264	2096	2096
query60	238	213	211	211
query61	92	81	79	79
query62	595	373	353	353
query63	325	280	285	280
query64	6409	2981	3106	2981
query65	3299	3258	3235	3235
query66	1382	321	317	317
query67	14468	14305	14199	14199
query68	5040	553	528	528
query69	486	355	354	354
query70	1230	1242	1206	1206
query71	354	252	263	252
query72	6213	2809	2609	2609
query73	689	310	310	310
query74	6879	6533	6454	6454
query75	3158	2546	2549	2546
query76	3046	1147	1197	1147
query77	342	234	240	234
query78	9303	8900	8739	8739
query79	951	495	487	487
query80	513	352	363	352
query81	439	206	211	206
query82	240	84	87	84
query83	140	121	117	117
query84	246	76	80	76
query85	1046	339	337	337
query86	283	314	295	295
query87	3458	3284	3271	3271
query88	2700	2297	2284	2284
query89	444	352	348	348
query90	2010	164	163	163
query91	148	134	126	126
query92	54	48	54	48
query93	1000	497	481	481
query94	1145	184	175	175
query95	435	339	341	339
query96	571	262	265	262
query97	4430	4268	4266	4266
query98	225	207	198	198
query99	1053	756	717	717
Total cold run time: 266888 ms
Total hot run time: 175937 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 67e0f0c1af47a32d9768b4c0aa4cdccae88e0569, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.23	0.06	0.08
query4	1.65	0.08	0.08
query5	0.48	0.47	0.49
query6	1.38	0.61	0.60
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.52	0.43	0.46
query10	0.49	0.48	0.49
query11	0.14	0.09	0.09
query12	0.12	0.09	0.10
query13	0.60	0.60	0.59
query14	0.77	0.78	0.78
query15	0.81	0.78	0.78
query16	0.31	0.34	0.34
query17	0.91	0.91	0.90
query18	0.17	0.18	0.15
query19	1.73	1.66	1.70
query20	0.02	0.01	0.01
query21	15.42	0.59	0.54
query22	3.08	3.60	2.76
query23	17.66	1.00	0.95
query24	1.98	0.60	0.55
query25	0.62	0.06	0.06
query26	0.15	0.14	0.13
query27	0.05	0.06	0.05
query28	11.83	0.83	0.83
query29	12.67	3.24	3.30
query30	0.51	0.49	0.52
query31	2.77	0.37	0.38
query32	3.33	0.47	0.48
query33	3.12	3.11	3.09
query34	15.39	4.49	4.50
query35	4.49	4.51	4.52
query36	1.08	0.95	0.94
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.19	0.17	0.18
query41	0.07	0.02	0.01
query42	0.03	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.06 s
Total hot run time: 31.37 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 67e0f0c1af47a32d9768b4c0aa4cdccae88e0569 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       15.7 seconds inserted 10000000 Rows, about 636K ops/s

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37452 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 54a00d2da0e4a787ba28a23be5d95fce651fcbe7, data reload: false

------ Round 1 ----------------------------------
q1	17696	4028	4013	4013
q2	2040	138	131	131
q3	10587	927	931	927
q4	4652	934	938	934
q5	7620	2821	2988	2821
q6	179	122	119	119
q7	1218	746	733	733
q8	9237	1950	1983	1950
q9	7118	6222	6217	6217
q10	8191	2517	2470	2470
q11	415	212	208	208
q12	770	308	304	304
q13	17932	3204	3198	3198
q14	281	251	248	248
q15	534	492	485	485
q16	495	404	413	404
q17	931	849	802	802
q18	6695	5989	6074	5989
q19	1577	1492	1497	1492
q20	546	271	263	263
q21	6469	3450	3556	3450
q22	811	296	294	294
Total cold run time: 105994 ms
Total hot run time: 37452 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4003	4005	3985	3985
q2	301	218	214	214
q3	2975	2898	2893	2893
q4	1845	1776	1807	1776
q5	5091	5077	5030	5030
q6	196	115	113	113
q7	2103	1717	1716	1716
q8	3077	3150	3173	3150
q9	8205	8159	8230	8159
q10	6071	3676	3635	3635
q11	515	417	435	417
q12	686	523	527	523
q13	11535	3061	3055	3055
q14	284	261	248	248
q15	530	480	490	480
q16	492	433	464	433
q17	1683	1663	1656	1656
q18	7891	7413	7435	7413
q19	2218	1617	1621	1617
q20	2092	1890	1879	1879
q21	4822	4569	4511	4511
q22	501	438	441	438
Total cold run time: 67116 ms
Total hot run time: 53341 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169519 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 54a00d2da0e4a787ba28a23be5d95fce651fcbe7, data reload: false

query1	945	343	329	329
query2	6554	1747	1774	1747
query3	6704	204	204	204
query4	23231	20561	20538	20538
query5	4310	357	358	357
query6	282	162	167	162
query7	4609	308	311	308
query8	251	196	194	194
query9	8467	2197	2194	2194
query10	426	221	224	221
query11	14650	14216	14217	14216
query12	139	82	81	81
query13	1650	432	438	432
query14	8436	6499	6290	6290
query15	214	170	169	169
query16	7077	260	256	256
query17	1041	530	521	521
query18	1897	275	269	269
query19	196	168	146	146
query20	85	80	86	80
query21	199	125	119	119
query22	4751	4562	4614	4562
query23	30352	29938	29834	29834
query24	11879	3092	3156	3092
query25	664	355	351	351
query26	1888	155	164	155
query27	3062	335	325	325
query28	7013	1796	1767	1767
query29	1263	565	561	561
query30	288	132	139	132
query31	886	673	687	673
query32	92	61	60	60
query33	722	238	232	232
query34	1002	496	498	496
query35	849	722	742	722
query36	944	859	824	824
query37	188	64	65	64
query38	3112	2977	2974	2974
query39	1307	1254	1257	1254
query40	275	104	100	100
query41	38	39	37	37
query42	106	99	104	99
query43	451	423	420	420
query44	1085	698	702	698
query45	186	182	170	170
query46	1045	780	775	775
query47	1435	1424	1460	1424
query48	434	369	355	355
query49	1194	323	290	290
query50	776	388	375	375
query51	4348	4288	4303	4288
query52	101	90	89	89
query53	332	263	268	263
query54	299	225	234	225
query55	90	85	88	85
query56	232	197	216	197
query57	980	940	858	858
query58	212	190	195	190
query59	2234	2199	2136	2136
query60	236	212	226	212
query61	89	84	92	84
query62	592	342	352	342
query63	283	256	255	255
query64	6458	2964	3124	2964
query65	3231	3218	3205	3205
query66	1418	328	321	321
query67	14241	14343	14266	14266
query68	5004	572	583	572
query69	548	374	366	366
query70	1267	1160	1210	1160
query71	386	259	252	252
query72	6247	2836	2696	2696
query73	706	327	329	327
query74	6259	6046	5984	5984
query75	3075	2444	2400	2400
query76	2860	1054	1164	1054
query77	348	249	231	231
query78	9073	8515	8419	8419
query79	980	547	535	535
query80	582	366	363	363
query81	450	197	197	197
query82	425	83	85	83
query83	240	119	122	119
query84	282	82	80	80
query85	1140	363	345	345
query86	297	297	266	266
query87	3236	3094	3101	3094
query88	2814	2492	2402	2402
query89	389	316	321	316
query90	1969	172	168	168
query91	157	124	124	124
query92	53	49	46	46
query93	1035	536	521	521
query94	1062	179	180	179
query95	440	339	348	339
query96	572	271	276	271
query97	4235	4101	4124	4101
query98	222	212	192	192
query99	1046	747	683	683
Total cold run time: 261425 ms
Total hot run time: 169519 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 54a00d2da0e4a787ba28a23be5d95fce651fcbe7, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.67	0.10	0.10
query5	0.52	0.50	0.52
query6	1.23	0.69	0.68
query7	0.02	0.01	0.02
query8	0.03	0.02	0.02
query9	0.57	0.51	0.51
query10	0.58	0.58	0.56
query11	0.13	0.10	0.10
query12	0.12	0.10	0.09
query13	0.62	0.61	0.62
query14	0.78	0.80	0.80
query15	0.84	0.80	0.80
query16	0.36	0.38	0.37
query17	0.97	0.98	0.98
query18	0.25	0.25	0.26
query19	1.84	1.74	1.75
query20	0.01	0.00	0.01
query21	15.39	0.60	0.61
query22	3.72	4.19	2.52
query23	17.53	1.05	0.96
query24	2.23	0.31	0.31
query25	0.44	0.12	0.09
query26	0.16	0.14	0.16
query27	0.06	0.03	0.04
query28	12.29	0.84	0.83
query29	12.55	3.44	3.51
query30	0.60	0.53	0.54
query31	2.78	0.35	0.35
query32	3.38	0.47	0.48
query33	3.18	3.22	3.21
query34	15.35	4.79	4.74
query35	4.73	4.75	4.79
query36	1.14	1.05	1.06
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.16	0.13	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 106.81 s
Total hot run time: 32.51 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 54a00d2da0e4a787ba28a23be5d95fce651fcbe7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.3 seconds inserted 10000000 Rows, about 578K ops/s

@yujun777 yujun777 force-pushed the fix-TabletRepairAndBalanceTest branch from 54a00d2 to 3478ef8 Compare February 27, 2024 02:36
@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37734 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3478ef8a34b0f138f7759e4e1fd5f7f28e34fd8b, data reload: false

------ Round 1 ----------------------------------
q1	17736	4021	4049	4021
q2	2044	143	138	138
q3	10580	968	942	942
q4	4656	927	920	920
q5	7606	2909	2986	2909
q6	182	124	123	123
q7	1243	751	741	741
q8	9234	2011	2041	2011
q9	7172	6306	6271	6271
q10	8168	2467	2487	2467
q11	408	215	221	215
q12	738	307	316	307
q13	17941	3183	3223	3183
q14	274	243	264	243
q15	534	500	492	492
q16	469	403	414	403
q17	927	889	873	873
q18	6692	5870	5984	5870
q19	1590	1517	1531	1517
q20	553	270	277	270
q21	6554	3520	3598	3520
q22	806	303	298	298
Total cold run time: 106107 ms
Total hot run time: 37734 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4168	4026	4039	4026
q2	300	218	212	212
q3	2971	2950	2910	2910
q4	1866	1833	1781	1781
q5	5076	5119	5170	5119
q6	204	117	116	116
q7	2140	1729	1752	1729
q8	3114	3200	3213	3200
q9	8282	8238	8236	8236
q10	6105	3678	3654	3654
q11	505	442	429	429
q12	696	538	584	538
q13	10949	3065	3072	3065
q14	281	249	244	244
q15	540	489	486	486
q16	491	477	471	471
q17	1711	1689	1688	1688
q18	7854	7398	7547	7398
q19	1677	1665	1645	1645
q20	2095	1889	1912	1889
q21	4817	4632	4616	4616
q22	505	444	451	444
Total cold run time: 66347 ms
Total hot run time: 53896 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169536 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3478ef8a34b0f138f7759e4e1fd5f7f28e34fd8b, data reload: false

query1	938	339	334	334
query2	6553	1735	1758	1735
query3	6703	205	204	204
query4	23298	20555	20469	20469
query5	4304	371	366	366
query6	264	166	169	166
query7	4609	307	305	305
query8	272	200	206	200
query9	8459	2284	2266	2266
query10	415	234	219	219
query11	14747	14212	14079	14079
query12	139	83	87	83
query13	1653	435	431	431
query14	8684	6682	6499	6499
query15	216	174	178	174
query16	7090	263	248	248
query17	1040	521	512	512
query18	1897	264	266	264
query19	188	151	157	151
query20	90	79	89	79
query21	191	129	127	127
query22	4629	4551	4488	4488
query23	30548	29859	29879	29859
query24	11882	3215	3090	3090
query25	660	356	353	353
query26	1822	154	163	154
query27	2982	328	327	327
query28	6999	1832	1831	1831
query29	1215	572	564	564
query30	283	138	143	138
query31	880	679	685	679
query32	93	61	59	59
query33	709	240	224	224
query34	1025	493	496	493
query35	855	757	763	757
query36	952	831	859	831
query37	256	65	63	63
query38	3117	2990	2990	2990
query39	1335	1278	1260	1260
query40	285	103	102	102
query41	39	37	35	35
query42	109	98	96	96
query43	457	439	426	426
query44	1073	705	726	705
query45	188	176	171	171
query46	1031	807	799	799
query47	1470	1438	1363	1363
query48	422	344	356	344
query49	1189	307	308	307
query50	773	388	385	385
query51	4389	4251	4286	4251
query52	106	93	93	93
query53	339	269	269	269
query54	299	221	230	221
query55	91	90	80	80
query56	224	204	197	197
query57	951	873	913	873
query58	216	203	205	203
query59	2243	2212	2218	2212
query60	244	235	225	225
query61	86	83	82	82
query62	601	349	373	349
query63	282	259	263	259
query64	6491	3005	3146	3005
query65	3252	3238	3196	3196
query66	1431	322	314	314
query67	14184	14046	14114	14046
query68	4930	554	567	554
query69	536	382	390	382
query70	1230	1231	1241	1231
query71	325	254	251	251
query72	6180	2777	2653	2653
query73	719	334	327	327
query74	6308	6023	5950	5950
query75	3049	2430	2414	2414
query76	2758	1023	1166	1023
query77	357	233	233	233
query78	8935	8430	8475	8430
query79	1573	532	538	532
query80	1080	359	354	354
query81	500	197	196	196
query82	941	86	89	86
query83	230	127	124	124
query84	286	81	80	80
query85	1340	355	343	343
query86	388	305	291	291
query87	3288	3105	3147	3105
query88	2832	2367	2365	2365
query89	402	359	328	328
query90	1930	169	173	169
query91	157	123	128	123
query92	57	53	49	49
query93	1256	511	503	503
query94	1284	186	181	181
query95	451	352	343	343
query96	581	266	263	263
query97	4217	4168	4130	4130
query98	210	210	194	194
query99	1057	663	650	650
Total cold run time: 263844 ms
Total hot run time: 169536 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.22 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3478ef8a34b0f138f7759e4e1fd5f7f28e34fd8b, data reload: false

query1	0.04	0.03	0.04
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.67	0.10	0.09
query5	0.53	0.51	0.51
query6	1.22	0.66	0.67
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.51
query10	0.58	0.57	0.57
query11	0.12	0.10	0.10
query12	0.13	0.10	0.10
query13	0.61	0.61	0.61
query14	0.77	0.81	0.80
query15	0.84	0.82	0.82
query16	0.37	0.37	0.39
query17	1.00	1.00	1.00
query18	0.27	0.26	0.26
query19	1.83	1.71	1.69
query20	0.02	0.01	0.01
query21	15.40	0.65	0.56
query22	3.66	4.56	2.58
query23	17.47	1.03	1.00
query24	2.05	0.72	0.21
query25	0.44	0.07	0.08
query26	0.19	0.17	0.18
query27	0.06	0.04	0.05
query28	12.07	0.84	0.83
query29	12.56	3.19	3.52
query30	0.57	0.54	0.52
query31	2.79	0.36	0.36
query32	3.34	0.48	0.47
query33	3.21	3.18	3.26
query34	15.36	4.73	4.74
query35	4.77	4.73	4.73
query36	1.17	1.07	1.08
query37	0.06	0.05	0.05
query38	0.04	0.02	0.03
query39	0.02	0.02	0.01
query40	0.17	0.13	0.13
query41	0.08	0.01	0.01
query42	0.02	0.01	0.02
query43	0.04	0.02	0.02
Total cold run time: 106.47 s
Total hot run time: 32.22 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3478ef8a34b0f138f7759e4e1fd5f7f28e34fd8b with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.1 seconds inserted 10000000 Rows, about 584K ops/s

Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 28, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@yiguolei yiguolei merged commit 94ea17a into apache:master Feb 28, 2024
28 of 30 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Feb 28, 2024
dataroaring pushed a commit that referenced this pull request Jun 7, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.12-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants