Skip to content

Conversation

@wuwenchi
Copy link
Contributor

Proposed changes

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37320 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e0fc9b9ad3b9fd011d529a388703e9ea7f52048, data reload: false

------ Round 1 ----------------------------------
q1	17680	4861	5095	4861
q2	2045	143	133	133
q3	10696	941	904	904
q4	4700	735	747	735
q5	7687	2782	2762	2762
q6	186	123	122	122
q7	1117	714	705	705
q8	9351	1981	1976	1976
q9	7221	6346	6320	6320
q10	8092	2406	2400	2400
q11	419	205	202	202
q12	713	262	281	262
q13	18027	3294	3282	3282
q14	283	244	254	244
q15	543	495	488	488
q16	460	389	406	389
q17	938	491	502	491
q18	7012	6219	6100	6100
q19	1582	1455	1356	1356
q20	583	350	309	309
q21	6827	2997	3094	2997
q22	815	309	282	282
Total cold run time: 106977 ms
Total hot run time: 37320 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5317	4830	5071	4830
q2	338	235	231	231
q3	3018	2642	2640	2640
q4	1787	1567	1555	1555
q5	5259	5160	5282	5160
q6	195	117	116	116
q7	2107	1761	1818	1761
q8	3144	3160	3271	3160
q9	8721	8569	10564	8569
q10	5497	3513	3503	3503
q11	537	457	448	448
q12	758	583	581	581
q13	16949	3086	3063	3063
q14	285	262	280	262
q15	525	493	487	487
q16	503	453	454	453
q17	1922	1716	1639	1639
q18	8957	11531	8618	8618
q19	24045	1570	1544	1544
q20	4614	1929	1884	1884
q21	15713	4842	4737	4737
q22	3086	430	495	430
Total cold run time: 113277 ms
Total hot run time: 55671 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173019 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e0fc9b9ad3b9fd011d529a388703e9ea7f52048, data reload: false

query1	954	332	326	326
query2	6587	1983	2050	1983
query3	6706	200	197	197
query4	33908	22192	22043	22043
query5	4448	348	438	348
query6	240	161	158	158
query7	4606	265	258	258
query8	245	181	181	181
query9	8529	2262	2264	2262
query10	415	202	200	200
query11	17828	15457	15275	15275
query12	127	69	69	69
query13	1671	372	380	372
query14	9183	6604	6503	6503
query15	223	183	183	183
query16	5696	251	247	247
query17	952	476	481	476
query18	1800	264	249	249
query19	176	137	132	132
query20	72	75	69	69
query21	198	142	133	133
query22	4985	4789	4489	4489
query23	31207	30310	30235	30235
query24	10452	2801	2818	2801
query25	549	320	317	317
query26	900	145	154	145
query27	2999	288	281	281
query28	6133	1842	1816	1816
query29	946	633	612	612
query30	281	141	136	136
query31	923	712	736	712
query32	76	53	50	50
query33	596	217	214	214
query34	839	463	465	463
query35	846	772	749	749
query36	1433	1160	1274	1160
query37	90	61	57	57
query38	3278	3155	3159	3155
query39	1306	1257	1257	1257
query40	201	88	84	84
query41	38	36	35	35
query42	84	76	88	76
query43	530	477	492	477
query44	1081	692	685	685
query45	192	176	180	176
query46	1040	644	657	644
query47	1607	1567	1545	1545
query48	388	312	316	312
query49	1095	297	286	286
query50	676	322	317	317
query51	5252	5166	5188	5166
query52	100	73	74	73
query53	321	263	259	259
query54	234	186	191	186
query55	80	74	77	74
query56	170	162	163	162
query57	977	908	930	908
query58	180	171	164	164
query59	2768	2499	2618	2499
query60	199	180	177	177
query61	87	84	82	82
query62	580	375	341	341
query63	277	260	267	260
query64	4028	1770	1720	1720
query65	3302	3216	3238	3216
query66	1175	330	307	307
query67	14535	14026	14021	14021
query68	3750	488	489	488
query69	447	312	317	312
query70	1637	1565	1560	1560
query71	277	216	219	216
query72	3746	2828	2846	2828
query73	705	317	310	310
query74	6615	6262	6221	6221
query75	2860	2288	2292	2288
query76	2632	1013	978	978
query77	364	230	228	228
query78	9134	8785	8542	8542
query79	2767	491	496	491
query80	1943	329	311	311
query81	529	200	200	200
query82	710	86	82	82
query83	241	120	124	120
query84	288	77	76	76
query85	1936	343	343	343
query86	507	433	415	415
query87	3441	3282	3308	3282
query88	4194	2167	2173	2167
query89	437	341	356	341
query90	1922	183	188	183
query91	157	116	118	116
query92	54	43	45	43
query93	5441	449	429	429
query94	1251	162	162	162
query95	497	465	457	457
query96	641	308	315	308
query97	4282	4140	4114	4114
query98	209	186	184	184
query99	966	659	680	659
Total cold run time: 278180 ms
Total hot run time: 173019 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e0fc9b9ad3b9fd011d529a388703e9ea7f52048, data reload: false

query1	0.04	0.02	0.03
query2	0.06	0.03	0.02
query3	0.22	0.06	0.06
query4	1.69	0.09	0.10
query5	0.53	0.51	0.52
query6	1.26	0.66	0.64
query7	0.01	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.49	0.50
query10	0.54	0.55	0.55
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.61	0.60	0.60
query14	0.78	0.82	0.79
query15	0.80	0.79	0.78
query16	0.39	0.39	0.40
query17	1.04	0.98	1.03
query18	0.21	0.24	0.22
query19	1.84	1.77	1.82
query20	0.01	0.01	0.01
query21	15.40	0.58	0.58
query22	2.62	2.24	1.76
query23	17.41	0.76	0.71
query24	2.57	1.25	1.29
query25	0.37	0.19	0.16
query26	0.25	0.14	0.14
query27	0.05	0.04	0.05
query28	10.87	0.86	0.82
query29	12.57	3.18	3.25
query30	0.63	0.54	0.56
query31	2.79	0.35	0.35
query32	3.35	0.49	0.48
query33	3.22	3.23	3.23
query34	15.78	4.24	4.27
query35	4.37	4.36	4.27
query36	1.10	1.05	1.07
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.02	0.02	0.01
query43	0.03	0.01	0.02
Total cold run time: 104.61 s
Total hot run time: 31.19 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1e0fc9b9ad3b9fd011d529a388703e9ea7f52048 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

@wuwenchi
Copy link
Contributor Author

run buildall

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37083 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7875ad0ff51be668cf4bba78eb4f63f72ff5841d, data reload: false

------ Round 1 ----------------------------------
q1	17937	4708	4528	4528
q2	2420	149	133	133
q3	11625	947	917	917
q4	5805	745	737	737
q5	7884	2864	2922	2864
q6	187	122	121	121
q7	1157	754	740	740
q8	9392	1987	2017	1987
q9	7231	6365	6288	6288
q10	8108	2431	2425	2425
q11	417	214	208	208
q12	726	277	285	277
q13	18015	3280	3306	3280
q14	277	257	242	242
q15	529	497	487	487
q16	460	392	397	392
q17	938	509	517	509
q18	6742	6108	5919	5919
q19	1574	1399	1318	1318
q20	623	331	316	316
q21	7005	3101	3094	3094
q22	806	301	312	301
Total cold run time: 109858 ms
Total hot run time: 37083 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4492	4474	4504	4474
q2	327	232	224	224
q3	2973	2840	2776	2776
q4	1854	1625	1641	1625
q5	5169	5220	5253	5220
q6	189	113	115	113
q7	2187	1765	1733	1733
q8	3095	3260	3263	3260
q9	8318	8312	8211	8211
q10	5726	3572	3576	3572
q11	544	489	456	456
q12	738	587	584	584
q13	9424	3103	3055	3055
q14	265	242	256	242
q15	530	487	483	483
q16	510	469	461	461
q17	1828	1637	1732	1637
q18	7949	7799	7519	7519
q19	7691	1529	1567	1529
q20	2125	1905	1904	1904
q21	4798	4582	4497	4497
q22	555	468	480	468
Total cold run time: 71287 ms
Total hot run time: 54043 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172934 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7875ad0ff51be668cf4bba78eb4f63f72ff5841d, data reload: false

query1	948	340	327	327
query2	6548	1915	1877	1877
query3	6694	200	194	194
query4	28050	22064	22061	22061
query5	4453	418	338	338
query6	247	153	157	153
query7	4613	260	255	255
query8	237	166	170	166
query9	9062	2317	2310	2310
query10	408	198	208	198
query11	17002	15451	15322	15322
query12	121	67	63	63
query13	1670	384	365	365
query14	9168	6466	6462	6462
query15	212	179	177	177
query16	6425	253	247	247
query17	1902	468	456	456
query18	1920	264	243	243
query19	174	128	130	128
query20	70	72	67	67
query21	194	136	128	128
query22	5026	4693	4839	4693
query23	31124	30339	30243	30243
query24	12385	2761	2738	2738
query25	608	305	306	305
query26	2012	133	134	133
query27	3079	282	280	280
query28	7541	1857	1858	1857
query29	2206	632	608	608
query30	286	132	138	132
query31	924	709	722	709
query32	91	48	47	47
query33	708	212	205	205
query34	1165	457	458	457
query35	865	777	746	746
query36	1259	1197	1219	1197
query37	176	59	65	59
query38	3293	3146	3150	3146
query39	1278	1246	1234	1234
query40	322	84	82	82
query41	36	34	36	34
query42	85	85	75	75
query43	476	461	479	461
query44	1108	685	688	685
query45	196	179	171	171
query46	1052	650	634	634
query47	1587	1475	1543	1475
query48	383	303	307	303
query49	1274	287	278	278
query50	680	307	307	307
query51	5249	5182	5119	5119
query52	90	81	78	78
query53	317	247	252	247
query54	265	177	180	177
query55	75	73	72	72
query56	173	158	163	158
query57	962	895	893	893
query58	191	146	154	146
query59	2325	2333	2353	2333
query60	192	178	171	171
query61	87	86	86	86
query62	669	363	352	352
query63	271	250	260	250
query64	5644	1764	1746	1746
query65	3271	3205	3201	3201
query66	1486	316	311	311
query67	14416	14475	14202	14202
query68	3825	493	478	478
query69	423	304	301	301
query70	1517	1464	1483	1464
query71	299	211	211	211
query72	3746	2871	2869	2869
query73	681	315	311	311
query74	6583	6269	6282	6269
query75	2791	2340	2292	2292
query76	2895	943	936	936
query77	430	223	228	223
query78	9019	8868	8817	8817
query79	1456	488	488	488
query80	1269	319	308	308
query81	486	190	196	190
query82	1018	82	79	79
query83	222	112	120	112
query84	230	72	73	72
query85	1449	360	336	336
query86	412	416	400	400
query87	3437	3333	3228	3228
query88	2954	2196	2182	2182
query89	412	368	341	341
query90	2038	187	183	183
query91	167	118	134	118
query92	51	43	42	42
query93	947	441	407	407
query94	1450	160	159	159
query95	503	442	452	442
query96	600	321	315	315
query97	4240	4126	4097	4097
query98	206	200	184	184
query99	958	750	707	707
Total cold run time: 273740 ms
Total hot run time: 172934 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.77 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7875ad0ff51be668cf4bba78eb4f63f72ff5841d, data reload: false

query1	0.04	0.02	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.68	0.09	0.09
query5	0.52	0.51	0.52
query6	1.18	0.66	0.65
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.56	0.49	0.51
query10	0.55	0.55	0.55
query11	0.12	0.09	0.09
query12	0.11	0.09	0.10
query13	0.60	0.61	0.61
query14	0.79	0.81	0.80
query15	0.81	0.78	0.76
query16	0.37	0.40	0.37
query17	1.04	0.99	0.97
query18	0.22	0.26	0.25
query19	1.89	1.75	1.76
query20	0.01	0.01	0.01
query21	15.42	0.56	0.54
query22	3.21	2.38	1.73
query23	17.24	0.81	0.69
query24	2.65	1.58	1.01
query25	0.38	0.26	0.14
query26	0.52	0.14	0.14
query27	0.05	0.05	0.04
query28	11.52	0.83	0.84
query29	12.51	3.10	3.34
query30	0.61	0.60	0.52
query31	2.79	0.34	0.35
query32	3.37	0.48	0.48
query33	3.20	3.16	3.21
query34	15.82	4.34	4.33
query35	4.32	4.28	4.32
query36	1.10	1.06	1.05
query37	0.06	0.05	0.05
query38	0.03	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.01
query42	0.02	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.94 s
Total hot run time: 30.77 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 7875ad0ff51be668cf4bba78eb4f63f72ff5841d with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.7 seconds inserted 10000000 Rows, about 729K ops/s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 30, 2024
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 0cf1abb into apache:master Jan 30, 2024
yiguolei pushed a commit that referenced this pull request Jan 30, 2024
@wm1581066 wm1581066 added the usercase Important user case type label label Jan 31, 2024
morningman pushed a commit that referenced this pull request Feb 1, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.5-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants