Skip to content

Conversation

@Jibing-Li
Copy link
Contributor

backport: #39101

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li
Copy link
Contributor Author

run buildall

@Jibing-Li Jibing-Li marked this pull request as ready for review August 15, 2024 05:01
@doris-robot
Copy link

TPC-H: Total hot run time: 49787 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6d614c417b7f463304a3ce68d10ea56770f4f98d, data reload: false

------ Round 1 ----------------------------------
q1	17852	4339	4349	4339
q2	2073	159	156	156
q3	10335	1889	1906	1889
q4	10367	1244	1295	1244
q5	8521	3922	3902	3902
q6	228	125	126	125
q7	2068	1621	1613	1613
q8	9496	2727	2694	2694
q9	13747	10470	10307	10307
q10	8614	3520	3514	3514
q11	413	250	253	250
q12	464	310	303	303
q13	18354	3990	4072	3990
q14	361	325	319	319
q15	503	467	463	463
q16	670	583	569	569
q17	1130	944	903	903
q18	7247	6825	6982	6825
q19	1718	1556	1548	1548
q20	547	311	291	291
q21	4383	4155	4143	4143
q22	479	400	411	400
Total cold run time: 119570 ms
Total hot run time: 49787 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4306	4277	4286	4277
q2	322	224	226	224
q3	4126	4168	4109	4109
q4	2745	2748	2753	2748
q5	7240	7154	7092	7092
q6	244	120	123	120
q7	3263	2886	2786	2786
q8	4365	4457	4442	4442
q9	17360	17237	17120	17120
q10	4274	4301	4294	4294
q11	736	670	691	670
q12	1027	878	864	864
q13	8965	3751	3765	3751
q14	451	427	423	423
q15	503	467	467	467
q16	746	682	680	680
q17	3834	3890	3819	3819
q18	9664	8662	8763	8662
q19	1737	1708	1692	1692
q20	2391	2141	2138	2138
q21	8531	8785	8821	8785
q22	1065	1114	1081	1081
Total cold run time: 87895 ms
Total hot run time: 80244 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201445 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6d614c417b7f463304a3ce68d10ea56770f4f98d, data reload: false

query1	2868	392	449	392
query2	7829	2240	2021	2021
query3	6926	197	202	197
query4	20209	18041	17920	17920
query5	19741	6585	6515	6515
query6	426	216	240	216
query7	5209	298	300	298
query8	273	244	242	242
query9	3145	2706	2664	2664
query10	473	304	296	296
query11	11363	10670	10891	10670
query12	127	76	76	76
query13	5614	662	661	661
query14	19053	13176	13435	13176
query15	371	227	232	227
query16	6434	282	264	264
query17	1380	1477	879	879
query18	2269	409	409	409
query19	208	151	155	151
query20	80	77	82	77
query21	183	99	89	89
query22	5280	5098	5015	5015
query23	32421	31991	31901	31901
query24	6877	6526	6514	6514
query25	530	420	444	420
query26	525	171	161	161
query27	1722	295	296	295
query28	6141	2346	2264	2264
query29	2878	2804	2600	2600
query30	243	173	165	165
query31	924	729	773	729
query32	70	62	63	62
query33	396	258	267	258
query34	843	462	470	462
query35	1138	908	958	908
query36	1412	1209	1182	1182
query37	95	63	62	62
query38	3110	2914	2918	2914
query39	1377	1346	1322	1322
query40	207	94	95	94
query41	39	38	40	38
query42	83	79	85	79
query43	601	668	676	668
query44	1129	721	731	721
query45	244	231	230	230
query46	1237	944	980	944
query47	1799	1772	1800	1772
query48	986	673	704	673
query49	619	377	376	376
query50	855	625	585	585
query51	4785	4612	4668	4612
query52	86	81	91	81
query53	448	325	317	317
query54	2655	2419	2448	2419
query55	103	74	88	74
query56	227	224	201	201
query57	1287	1043	1190	1043
query58	224	213	208	208
query59	3689	3371	3484	3371
query60	215	217	199	199
query61	103	105	105	105
query62	801	447	480	447
query63	485	335	351	335
query64	2535	1548	1493	1493
query65	3627	3516	3566	3516
query66	793	385	385	385
query67	16308	14900	15939	14900
query68	8934	641	659	641
query69	601	360	347	347
query70	1579	1488	1406	1406
query71	433	303	317	303
query72	6578	3507	3520	3507
query73	751	328	323	323
query74	6323	5808	5865	5808
query75	5482	3763	3634	3634
query76	5431	1100	1133	1100
query77	975	256	260	256
query78	12876	11904	11840	11840
query79	7536	647	644	644
query80	1003	406	424	406
query81	485	240	231	231
query82	1589	98	98	98
query83	180	138	132	132
query84	259	74	71	71
query85	895	328	331	328
query86	336	301	337	301
query87	3296	2993	3101	2993
query88	4947	2304	2310	2304
query89	377	284	270	270
query90	1921	213	218	213
query91	176	123	135	123
query92	59	53	52	52
query93	4179	542	552	542
query94	709	205	216	205
query95	1111	1087	1047	1047
query96	635	333	326	326
query97	6535	6389	6547	6389
query98	198	183	177	177
query99	2994	962	868	868
Total cold run time: 315716 ms
Total hot run time: 201445 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6d614c417b7f463304a3ce68d10ea56770f4f98d, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.24	0.04	0.05
query4	1.79	0.06	0.08
query5	0.55	0.52	0.52
query6	1.28	0.67	0.60
query7	0.01	0.02	0.01
query8	0.04	0.03	0.02
query9	0.52	0.48	0.47
query10	0.53	0.52	0.54
query11	0.12	0.09	0.09
query12	0.12	0.09	0.08
query13	0.62	0.61	0.61
query14	0.79	0.80	0.78
query15	0.77	0.76	0.75
query16	0.37	0.36	0.36
query17	0.98	1.02	1.00
query18	0.23	0.26	0.22
query19	1.86	1.88	1.82
query20	0.02	0.01	0.01
query21	15.50	0.57	0.55
query22	2.39	2.02	1.98
query23	17.34	0.95	0.95
query24	4.87	2.13	1.75
query25	0.39	0.07	0.05
query26	0.74	0.16	0.15
query27	0.06	0.03	0.04
query28	6.75	0.74	0.72
query29	12.61	2.25	2.18
query30	0.61	0.54	0.47
query31	2.81	0.39	0.37
query32	3.39	0.49	0.50
query33	3.08	3.07	3.11
query34	15.25	4.83	4.81
query35	4.86	4.85	4.85
query36	1.07	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.07	0.01	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.04 s
Total hot run time: 31.69 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6d614c417b7f463304a3ce68d10ea56770f4f98d with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

…he#39101)

Drop column stats after schema change. So the auto analyze could be
triggered and will collect stats under new schema.
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49575 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4e635d5a91e83a1545091e230d3af2ebf5c4cb5f, data reload: false

------ Round 1 ----------------------------------
q1	17609	4398	4392	4392
q2	2074	158	147	147
q3	10258	1921	1933	1921
q4	10120	1267	1338	1267
q5	8448	3915	3921	3915
q6	239	124	122	122
q7	2026	1617	1611	1611
q8	9532	2755	2723	2723
q9	13678	10244	10106	10106
q10	8718	3551	3548	3548
q11	401	253	249	249
q12	468	296	304	296
q13	18356	3966	4061	3966
q14	362	327	334	327
q15	499	452	447	447
q16	671	589	574	574
q17	1141	922	918	918
q18	7258	6762	6891	6762
q19	1734	1565	1475	1475
q20	525	311	318	311
q21	4438	4197	4106	4106
q22	494	392	405	392
Total cold run time: 119049 ms
Total hot run time: 49575 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4374	4292	4352	4292
q2	320	233	220	220
q3	4180	4138	4115	4115
q4	2753	2756	2760	2756
q5	7254	7149	7093	7093
q6	239	123	121	121
q7	3188	2853	2907	2853
q8	4364	4454	4450	4450
q9	16876	16741	16767	16741
q10	4216	4302	4297	4297
q11	756	693	672	672
q12	1048	856	848	848
q13	6994	3719	3710	3710
q14	446	433	430	430
q15	493	454	457	454
q16	738	697	676	676
q17	3779	3885	3875	3875
q18	8651	8728	8821	8728
q19	1717	1691	1686	1686
q20	2414	2154	2134	2134
q21	8570	8459	8406	8406
q22	1010	947	967	947
Total cold run time: 84380 ms
Total hot run time: 79504 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202261 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4e635d5a91e83a1545091e230d3af2ebf5c4cb5f, data reload: false

query1	939	387	441	387
query2	6551	2216	2084	2084
query3	6916	205	197	197
query4	20431	18109	18052	18052
query5	19734	6446	6513	6446
query6	286	209	235	209
query7	4164	294	315	294
query8	281	242	251	242
query9	3205	2758	2676	2676
query10	420	308	291	291
query11	11448	10797	10957	10797
query12	126	79	76	76
query13	5601	649	655	649
query14	17651	13892	13737	13737
query15	376	222	228	222
query16	6460	296	263	263
query17	1698	1454	904	904
query18	2283	446	408	408
query19	216	152	145	145
query20	78	79	79	79
query21	196	101	103	101
query22	5432	5076	5105	5076
query23	32513	31977	31885	31885
query24	7012	6482	6506	6482
query25	527	440	433	433
query26	534	159	168	159
query27	1875	305	300	300
query28	6083	2320	2289	2289
query29	2982	2572	2724	2572
query30	242	172	169	169
query31	896	736	726	726
query32	69	66	59	59
query33	409	260	259	259
query34	860	476	472	472
query35	1149	933	937	933
query36	1186	1148	1121	1121
query37	88	59	60	59
query38	3084	2978	2971	2971
query39	1378	1341	1323	1323
query40	205	96	95	95
query41	39	37	35	35
query42	83	77	81	77
query43	631	640	574	574
query44	1137	715	729	715
query45	244	235	234	234
query46	1224	1005	974	974
query47	1773	1663	1649	1649
query48	987	688	666	666
query49	622	388	364	364
query50	865	610	586	586
query51	4733	4638	4681	4638
query52	95	79	84	79
query53	443	314	330	314
query54	2660	2494	2497	2494
query55	92	84	89	84
query56	244	231	215	215
query57	1206	1070	1115	1070
query58	224	206	216	206
query59	3583	3375	3149	3149
query60	214	212	204	204
query61	95	93	96	93
query62	786	449	472	449
query63	487	344	342	342
query64	2306	1497	1503	1497
query65	3619	3573	3537	3537
query66	824	388	388	388
query67	15662	15021	16861	15021
query68	10062	664	673	664
query69	568	371	380	371
query70	1762	1373	1453	1373
query71	412	310	312	310
query72	6558	3506	3470	3470
query73	752	323	321	321
query74	6319	5842	5912	5842
query75	5339	3734	3674	3674
query76	6398	1126	1172	1126
query77	1045	259	258	258
query78	12479	12277	11717	11717
query79	11750	666	671	666
query80	757	415	420	415
query81	489	240	230	230
query82	1095	98	100	98
query83	159	135	138	135
query84	264	74	73	73
query85	866	327	319	319
query86	334	295	298	295
query87	3235	3058	3076	3058
query88	5165	2325	2322	2322
query89	425	316	295	295
query90	1970	224	221	221
query91	154	122	127	122
query92	59	54	51	51
query93	5851	561	595	561
query94	745	217	211	211
query95	1125	1072	1058	1058
query96	656	329	327	327
query97	6423	6324	6450	6324
query98	197	174	174	174
query99	2835	921	884	884
Total cold run time: 316710 ms
Total hot run time: 202261 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4e635d5a91e83a1545091e230d3af2ebf5c4cb5f, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.04
query4	1.77	0.06	0.06
query5	0.53	0.54	0.52
query6	1.25	0.61	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.51	0.50	0.47
query10	0.54	0.54	0.54
query11	0.13	0.09	0.09
query12	0.12	0.09	0.08
query13	0.62	0.62	0.62
query14	0.78	0.78	0.77
query15	0.77	0.76	0.77
query16	0.37	0.37	0.36
query17	1.00	1.02	1.01
query18	0.21	0.27	0.23
query19	1.95	1.83	1.87
query20	0.02	0.01	0.01
query21	15.47	0.55	0.54
query22	2.28	2.06	1.49
query23	17.21	0.94	0.94
query24	8.35	0.40	0.54
query25	0.38	0.13	0.05
query26	0.72	0.15	0.14
query27	0.03	0.05	0.04
query28	5.88	0.79	0.77
query29	12.73	2.33	2.29
query30	0.57	0.54	0.53
query31	2.80	0.38	0.38
query32	3.40	0.49	0.49
query33	3.06	3.05	3.08
query34	15.23	4.79	4.76
query35	4.83	4.82	4.84
query36	1.07	1.01	1.01
query37	0.06	0.05	0.04
query38	0.05	0.03	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.36 s
Total hot run time: 30.02 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4e635d5a91e83a1545091e230d3af2ebf5c4cb5f with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@Jibing-Li Jibing-Li merged commit 47fcfbd into apache:branch-2.0 Aug 15, 2024
@Jibing-Li Jibing-Li deleted the sc2.0 branch August 15, 2024 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants