Skip to content

[chore](ci) collect core and adjust log dir#31418

Merged
liaoxin01 merged 6 commits intoapache:masterfrom
hello-stephen:0226-ci-log
Feb 27, 2024
Merged

[chore](ci) collect core and adjust log dir#31418
liaoxin01 merged 6 commits intoapache:masterfrom
hello-stephen:0226-ci-log

Conversation

@hello-stephen
Copy link
Copy Markdown
Contributor

@hello-stephen hello-stephen commented Feb 26, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 26, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@hello-stephen
Copy link
Copy Markdown
Contributor Author

hello-stephen commented Feb 26, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 26, 2024

TPC-H: Total hot run time: 37584 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

------ Round 1 ----------------------------------
q1	17730	4009	4010	4009
q2	2072	137	130	130
q3	10586	981	926	926
q4	4657	920	910	910
q5	7611	2896	2980	2896
q6	179	121	120	120
q7	1221	755	730	730
q8	9251	1981	2005	1981
q9	7129	6227	6257	6227
q10	8162	2483	2481	2481
q11	420	207	195	195
q12	768	292	308	292
q13	17960	3165	3211	3165
q14	271	246	238	238
q15	529	526	504	504
q16	484	410	398	398
q17	914	882	875	875
q18	6584	5915	6066	5915
q19	1564	1497	1515	1497
q20	545	276	268	268
q21	6687	3561	3556	3556
q22	804	303	271	271
Total cold run time: 106128 ms
Total hot run time: 37584 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4000	4048	3980	3980
q2	299	219	219	219
q3	2969	2921	2902	2902
q4	1837	1801	1795	1795
q5	5056	5078	5048	5048
q6	200	116	115	115
q7	2170	1699	1718	1699
q8	3108	3178	3171	3171
q9	8271	8268	8196	8196
q10	6071	3655	3627	3627
q11	505	426	424	424
q12	698	572	546	546
q13	8072	3059	3057	3057
q14	277	251	252	251
q15	533	490	492	490
q16	521	476	463	463
q17	1677	1670	1671	1670
q18	7777	7391	7284	7284
q19	1965	1617	1607	1607
q20	2107	1927	1876	1876
q21	4797	4580	4574	4574
q22	510	456	409	409
Total cold run time: 63420 ms
Total hot run time: 53403 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 26, 2024

TPC-DS: Total hot run time: 168719 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

query1	916	343	331	331
query2	6555	1789	1706	1706
query3	6695	206	205	205
query4	23316	20448	20609	20448
query5	4233	356	367	356
query6	262	181	170	170
query7	4604	308	300	300
query8	269	187	186	186
query9	8440	2199	2176	2176
query10	421	224	222	222
query11	14517	14090	14273	14090
query12	134	84	82	82
query13	1654	437	424	424
query14	8749	6284	6339	6284
query15	205	170	171	170
query16	7091	268	263	263
query17	1019	518	500	500
query18	1912	283	263	263
query19	190	144	153	144
query20	83	80	81	80
query21	202	122	114	114
query22	4768	4458	4507	4458
query23	30527	29797	29846	29797
query24	11905	3105	3138	3105
query25	650	354	370	354
query26	1822	149	164	149
query27	3015	331	322	322
query28	6972	1784	1768	1768
query29	1231	579	557	557
query30	288	131	145	131
query31	873	671	695	671
query32	96	62	57	57
query33	720	233	228	228
query34	1032	490	493	490
query35	839	747	749	747
query36	950	812	871	812
query37	173	60	61	60
query38	3126	3027	2989	2989
query39	1304	1250	1246	1246
query40	277	100	102	100
query41	37	36	34	34
query42	102	101	91	91
query43	467	435	421	421
query44	1061	693	698	693
query45	184	181	175	175
query46	1028	794	774	774
query47	1498	1451	1421	1421
query48	413	348	344	344
query49	1187	298	309	298
query50	769	385	385	385
query51	4349	4228	4271	4228
query52	100	95	86	86
query53	340	261	266	261
query54	289	226	241	226
query55	84	80	82	80
query56	219	200	209	200
query57	949	833	888	833
query58	215	199	191	191
query59	2307	2097	2168	2097
query60	239	214	211	211
query61	85	84	90	84
query62	595	358	362	358
query63	294	260	264	260
query64	6534	2996	3059	2996
query65	3218	3202	3179	3179
query66	1441	321	312	312
query67	14130	14047	14116	14047
query68	3836	586	589	586
query69	511	376	371	371
query70	1184	1201	1199	1199
query71	322	250	261	250
query72	6060	2803	2677	2677
query73	711	325	327	325
query74	6333	6042	5987	5987
query75	3027	2456	2376	2376
query76	2406	1045	1174	1045
query77	328	229	234	229
query78	8937	8445	8508	8445
query79	1644	527	516	516
query80	1124	355	348	348
query81	478	200	195	195
query82	925	84	81	81
query83	224	119	119	119
query84	280	78	81	78
query85	1346	372	347	347
query86	374	309	269	269
query87	3230	3095	3159	3095
query88	2833	2416	2401	2401
query89	393	325	363	325
query90	1850	169	168	168
query91	158	145	126	126
query92	55	52	51	51
query93	1192	522	509	509
query94	1213	183	178	178
query95	440	335	339	335
query96	572	272	272	272
query97	4264	4123	4153	4123
query98	207	206	191	191
query99	1048	666	662	662
Total cold run time: 261674 ms
Total hot run time: 168719 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 26, 2024

ClickBench: Total hot run time: 31.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.68	0.10	0.10
query5	0.52	0.52	0.52
query6	1.31	0.68	0.66
query7	0.02	0.02	0.01
query8	0.03	0.02	0.02
query9	0.56	0.52	0.52
query10	0.56	0.57	0.56
query11	0.13	0.10	0.10
query12	0.12	0.10	0.10
query13	0.62	0.61	0.61
query14	0.80	0.82	0.78
query15	0.84	0.82	0.82
query16	0.36	0.36	0.37
query17	0.98	1.03	1.02
query18	0.26	0.25	0.26
query19	1.85	1.74	1.70
query20	0.01	0.02	0.02
query21	15.40	0.60	0.57
query22	3.37	4.05	1.68
query23	17.57	1.02	0.91
query24	2.00	0.56	0.22
query25	0.43	0.18	0.05
query26	0.17	0.14	0.16
query27	0.05	0.04	0.04
query28	12.23	0.87	0.84
query29	12.54	3.25	3.27
query30	0.61	0.54	0.54
query31	2.78	0.36	0.36
query32	3.33	0.49	0.48
query33	3.15	3.23	3.21
query34	15.34	4.78	4.78
query35	4.77	4.70	4.74
query36	1.13	1.07	1.06
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 106.24 s
Total hot run time: 31.36 s

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 26, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       15.1 seconds inserted 10000000 Rows, about 662K ops/s

@hello-stephen
Copy link
Copy Markdown
Contributor Author

hello-stephen commented Feb 27, 2024

run performance

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-H: Total hot run time: 37474 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

------ Round 1 ----------------------------------
q1	17755	4057	3994	3994
q2	2053	143	131	131
q3	10579	936	924	924
q4	4654	916	919	916
q5	7581	2835	2959	2835
q6	176	122	119	119
q7	1240	748	740	740
q8	9248	1970	2001	1970
q9	7140	6259	6231	6231
q10	8196	2497	2499	2497
q11	409	206	204	204
q12	723	300	311	300
q13	17929	3207	3187	3187
q14	265	253	264	253
q15	532	494	491	491
q16	475	402	406	402
q17	928	846	844	844
q18	6896	5880	6041	5880
q19	1563	1507	1491	1491
q20	539	273	264	264
q21	6719	3549	3526	3526
q22	816	280	275	275
Total cold run time: 106416 ms
Total hot run time: 37474 ms

----- Round 2, with runtime_filter_mode=off -----
q1	3996	4002	3974	3974
q2	304	223	221	221
q3	2930	2935	2923	2923
q4	1841	1823	1816	1816
q5	5125	5102	5060	5060
q6	198	113	112	112
q7	2087	1696	1716	1696
q8	3102	3178	3176	3176
q9	8260	8244	8186	8186
q10	6092	3622	3621	3621
q11	510	436	433	433
q12	674	536	515	515
q13	7528	3034	3041	3034
q14	274	247	257	247
q15	546	490	488	488
q16	520	464	460	460
q17	1676	1669	1659	1659
q18	7854	7623	7456	7456
q19	1631	1628	1609	1609
q20	2075	1892	1878	1878
q21	4779	4555	4589	4555
q22	515	446	454	446
Total cold run time: 62517 ms
Total hot run time: 53565 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-DS: Total hot run time: 169858 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

query1	919	333	323	323
query2	6575	1821	1708	1708
query3	6704	205	204	204
query4	23327	20550	20456	20456
query5	4258	355	354	354
query6	260	169	168	168
query7	4622	290	293	290
query8	282	186	195	186
query9	8542	2266	2252	2252
query10	412	214	215	214
query11	14540	14223	14143	14143
query12	134	82	80	80
query13	1657	424	436	424
query14	8980	6617	6429	6429
query15	213	180	178	178
query16	7093	253	246	246
query17	1061	520	495	495
query18	1921	268	267	267
query19	198	148	148	148
query20	83	85	83	83
query21	200	134	128	128
query22	4864	4788	4811	4788
query23	30559	29925	29856	29856
query24	11773	3052	3086	3052
query25	653	347	341	341
query26	1784	152	160	152
query27	2961	317	308	308
query28	6942	1838	1819	1819
query29	1203	555	561	555
query30	288	129	140	129
query31	852	689	690	689
query32	90	58	59	58
query33	715	224	216	216
query34	1054	473	484	473
query35	850	766	755	755
query36	936	867	876	867
query37	171	61	64	61
query38	3123	2998	2982	2982
query39	1307	1269	1262	1262
query40	278	108	99	99
query41	39	35	35	35
query42	109	97	103	97
query43	451	428	431	428
query44	1055	709	717	709
query45	190	176	170	170
query46	1044	798	796	796
query47	1551	1480	1361	1361
query48	417	346	347	346
query49	1178	296	277	277
query50	781	382	376	376
query51	4325	4358	4249	4249
query52	101	87	85	85
query53	332	258	270	258
query54	295	220	221	220
query55	83	79	86	79
query56	210	190	206	190
query57	999	886	906	886
query58	206	193	198	193
query59	2254	2232	2184	2184
query60	230	218	226	218
query61	86	87	83	83
query62	615	368	356	356
query63	279	251	256	251
query64	6561	3085	3100	3085
query65	3262	3220	3206	3206
query66	1438	319	307	307
query67	14551	14233	14289	14233
query68	5038	580	557	557
query69	530	377	374	374
query70	1230	1180	1207	1180
query71	391	253	254	253
query72	6286	2787	2657	2657
query73	702	324	315	315
query74	6371	6189	6087	6087
query75	3043	2398	2423	2398
query76	2785	1029	1154	1029
query77	360	232	230	230
query78	8988	8645	8618	8618
query79	1292	529	526	526
query80	971	351	351	351
query81	485	194	197	194
query82	995	86	85	85
query83	225	119	115	115
query84	271	75	87	75
query85	1320	355	345	345
query86	368	303	292	292
query87	3264	3143	3072	3072
query88	2760	2337	2332	2332
query89	385	312	372	312
query90	1858	170	166	166
query91	153	125	122	122
query92	56	48	52	48
query93	1148	519	510	510
query94	1201	181	176	176
query95	437	326	331	326
query96	578	263	265	263
query97	4255	4099	4109	4099
query98	218	196	187	187
query99	1062	650	648	648
Total cold run time: 264052 ms
Total hot run time: 169858 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

ClickBench: Total hot run time: 32.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b4497f850ff4e2d2d3ed18a951850cd821b1f5b7, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.02	0.02
query3	0.24	0.06	0.07
query4	1.68	0.10	0.10
query5	0.52	0.51	0.52
query6	1.24	0.69	0.67
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.54	0.51	0.51
query10	0.56	0.57	0.57
query11	0.13	0.10	0.10
query12	0.12	0.10	0.10
query13	0.62	0.61	0.62
query14	0.79	0.79	0.80
query15	0.84	0.82	0.82
query16	0.37	0.40	0.37
query17	0.98	0.99	1.00
query18	0.26	0.24	0.25
query19	1.81	1.69	1.76
query20	0.01	0.02	0.01
query21	15.40	0.62	0.62
query22	3.39	4.30	2.42
query23	17.36	1.05	1.02
query24	2.06	0.56	0.57
query25	0.49	0.09	0.08
query26	0.17	0.17	0.16
query27	0.05	0.05	0.05
query28	11.84	0.84	0.82
query29	12.49	3.36	3.35
query30	0.62	0.53	0.53
query31	2.77	0.36	0.36
query32	3.36	0.48	0.49
query33	3.19	3.17	3.21
query34	15.37	4.75	4.74
query35	4.71	4.73	4.75
query36	1.15	1.07	1.07
query37	0.07	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.18	0.12	0.12
query41	0.08	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.73 s
Total hot run time: 32.63 s

@hello-stephen
Copy link
Copy Markdown
Contributor Author

hello-stephen commented Feb 27, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-H: Total hot run time: 38035 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 45b69039ec2bde13a591c1335399f799d9bc31a7, data reload: false

------ Round 1 ----------------------------------
q1	17855	4161	4135	4135
q2	2539	144	141	141
q3	11043	971	972	971
q4	4983	971	963	963
q5	7963	2887	3045	2887
q6	181	121	120	120
q7	1248	762	761	761
q8	9258	2025	2017	2017
q9	7220	6264	6308	6264
q10	8175	2502	2487	2487
q11	421	205	218	205
q12	765	317	312	312
q13	17927	3233	3212	3212
q14	291	254	250	250
q15	539	505	494	494
q16	468	409	425	409
q17	919	887	877	877
q18	6752	6018	5918	5918
q19	1575	1501	1507	1501
q20	556	281	281	281
q21	6591	3536	3566	3536
q22	805	294	294	294
Total cold run time: 108074 ms
Total hot run time: 38035 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4013	3993	3987	3987
q2	306	215	226	215
q3	2973	2948	2916	2916
q4	1850	1800	1820	1800
q5	5151	5124	5207	5124
q6	200	114	112	112
q7	2143	1698	1731	1698
q8	3128	3179	3202	3179
q9	8285	8216	8302	8216
q10	6125	3658	3654	3654
q11	517	448	454	448
q12	710	546	543	543
q13	10166	3068	3047	3047
q14	263	257	254	254
q15	543	497	489	489
q16	520	477	456	456
q17	1704	1681	1665	1665
q18	7795	7547	7333	7333
q19	1646	1635	1633	1633
q20	2090	1915	1913	1913
q21	4783	4619	4622	4619
q22	514	453	440	440
Total cold run time: 65425 ms
Total hot run time: 53741 ms

@hello-stephen
Copy link
Copy Markdown
Contributor Author

hello-stephen commented Feb 27, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-H: Total hot run time: 37912 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aed526f47fa0fab78cf4a3364f3e330a3c074c55, data reload: false

------ Round 1 ----------------------------------
q1	17946	4151	4148	4148
q2	2274	140	137	137
q3	11340	959	965	959
q4	5094	973	975	973
q5	7950	2907	3034	2907
q6	179	119	122	119
q7	1232	765	742	742
q8	9252	1972	2009	1972
q9	7297	6312	6282	6282
q10	8198	2495	2491	2491
q11	416	204	206	204
q12	750	306	313	306
q13	17948	3217	3207	3207
q14	279	264	244	244
q15	526	498	496	496
q16	473	415	401	401
q17	937	868	839	839
q18	6773	6270	5924	5924
q19	1550	1501	1518	1501
q20	551	275	277	275
q21	6511	3499	3567	3499
q22	791	294	286	286
Total cold run time: 108267 ms
Total hot run time: 37912 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4023	4035	4013	4013
q2	300	210	215	210
q3	2984	2954	2922	2922
q4	1843	1845	1816	1816
q5	5132	5109	5161	5109
q6	204	114	114	114
q7	2157	1757	1732	1732
q8	3131	3182	3197	3182
q9	8262	8255	8282	8255
q10	6085	3645	3670	3645
q11	499	436	427	427
q12	694	531	538	531
q13	10754	3062	3038	3038
q14	266	255	241	241
q15	541	495	489	489
q16	519	459	479	459
q17	1706	1680	1664	1664
q18	7818	7430	7482	7430
q19	1636	1634	1618	1618
q20	2109	1907	1898	1898
q21	4745	4568	4539	4539
q22	497	469	433	433
Total cold run time: 65905 ms
Total hot run time: 53765 ms

@hello-stephen hello-stephen changed the title [chore](ci) adjust log [chore](ci) collect core and adjust log dir Feb 27, 2024
@hello-stephen
Copy link
Copy Markdown
Contributor Author

hello-stephen commented Feb 27, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-H: Total hot run time: 38044 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b1ef8a0db94ecf907c14cb3e275419aceaf9fb7, data reload: false

------ Round 1 ----------------------------------
q1	18133	4232	4133	4133
q2	2289	151	169	151
q3	11426	981	952	952
q4	4775	950	960	950
q5	7963	2960	3065	2960
q6	185	127	123	123
q7	1269	785	771	771
q8	9634	2057	2061	2057
q9	7148	6250	6271	6250
q10	8163	2486	2503	2486
q11	414	207	209	207
q12	762	310	314	310
q13	17931	3195	3232	3195
q14	280	250	244	244
q15	536	494	491	491
q16	474	400	414	400
q17	929	887	885	885
q18	6534	6100	5945	5945
q19	1546	1515	1500	1500
q20	542	273	276	273
q21	6158	3470	3541	3470
q22	805	291	308	291
Total cold run time: 107896 ms
Total hot run time: 38044 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4000	4001	4001	4001
q2	308	215	212	212
q3	2966	2939	2905	2905
q4	1845	1827	1802	1802
q5	5135	5092	5111	5092
q6	197	114	115	114
q7	2141	1692	1736	1692
q8	3082	3161	3192	3161
q9	8208	8258	8235	8235
q10	6185	3637	3633	3633
q11	509	434	429	429
q12	673	526	515	515
q13	4866	3067	3048	3048
q14	271	247	246	246
q15	546	490	500	490
q16	515	466	469	466
q17	1693	1685	1664	1664
q18	7808	7464	7481	7464
q19	1638	1630	1631	1630
q20	2090	1896	1883	1883
q21	4785	4635	4569	4569
q22	517	421	436	421
Total cold run time: 59978 ms
Total hot run time: 53672 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

TPC-DS: Total hot run time: 169041 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b1ef8a0db94ecf907c14cb3e275419aceaf9fb7, data reload: false

query1	938	341	330	330
query2	6532	1791	1786	1786
query3	6712	203	200	200
query4	23254	20509	20599	20509
query5	4302	362	362	362
query6	263	169	168	168
query7	4606	294	304	294
query8	269	191	222	191
query9	8370	2307	2298	2298
query10	423	222	218	218
query11	14416	14124	14048	14048
query12	133	82	83	82
query13	1634	455	429	429
query14	8380	6476	6507	6476
query15	212	175	179	175
query16	7094	262	248	248
query17	1039	523	518	518
query18	1905	268	266	266
query19	186	152	152	152
query20	85	78	79	78
query21	199	123	115	115
query22	4448	4579	4538	4538
query23	30538	29867	29923	29867
query24	11729	3149	3077	3077
query25	653	355	356	355
query26	1880	157	160	157
query27	3050	320	314	314
query28	7049	1845	1840	1840
query29	1183	566	563	563
query30	285	131	145	131
query31	852	678	678	678
query32	93	61	56	56
query33	717	235	226	226
query34	1042	479	509	479
query35	825	735	742	735
query36	963	868	864	864
query37	138	61	65	61
query38	3095	2984	2943	2943
query39	1309	1251	1267	1251
query40	281	101	99	99
query41	38	37	37	37
query42	101	97	100	97
query43	465	423	424	423
query44	1060	714	723	714
query45	191	183	170	170
query46	1047	789	796	789
query47	1466	1445	1430	1430
query48	422	347	349	347
query49	1150	297	295	295
query50	766	391	388	388
query51	4352	4222	4259	4222
query52	109	92	102	92
query53	335	271	271	271
query54	297	227	221	221
query55	88	86	84	84
query56	227	205	204	204
query57	950	871	890	871
query58	211	195	191	191
query59	2198	1914	2113	1914
query60	242	221	211	211
query61	85	88	82	82
query62	580	362	361	361
query63	280	260	254	254
query64	6559	2949	3105	2949
query65	3245	3221	3193	3193
query66	1453	323	307	307
query67	14385	13986	14048	13986
query68	5084	578	588	578
query69	554	382	403	382
query70	1249	1282	1215	1215
query71	387	260	259	259
query72	6273	2795	2670	2670
query73	706	324	323	323
query74	6253	6016	5888	5888
query75	3061	2409	2414	2409
query76	2832	1041	1148	1041
query77	347	233	230	230
query78	8954	8586	8464	8464
query79	1281	530	525	525
query80	979	360	364	360
query81	484	192	194	192
query82	952	85	86	85
query83	223	120	120	120
query84	278	78	78	78
query85	1322	349	331	331
query86	383	300	297	297
query87	3258	3076	3101	3076
query88	2763	2373	2335	2335
query89	415	332	321	321
query90	1852	163	169	163
query91	153	125	123	123
query92	60	49	48	48
query93	1235	528	523	523
query94	1282	180	177	177
query95	435	343	333	333
query96	572	268	268	268
query97	4269	4153	4131	4131
query98	225	199	196	196
query99	1073	620	649	620
Total cold run time: 262578 ms
Total hot run time: 169041 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

ClickBench: Total hot run time: 32.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1b1ef8a0db94ecf907c14cb3e275419aceaf9fb7, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.02
query3	0.24	0.07	0.06
query4	1.66	0.10	0.11
query5	0.51	0.51	0.51
query6	1.27	0.67	0.68
query7	0.02	0.02	0.01
query8	0.04	0.02	0.02
query9	0.54	0.51	0.53
query10	0.58	0.56	0.55
query11	0.13	0.10	0.09
query12	0.13	0.10	0.11
query13	0.63	0.61	0.61
query14	0.80	0.79	0.81
query15	0.84	0.83	0.81
query16	0.38	0.36	0.38
query17	0.97	0.98	0.99
query18	0.27	0.25	0.25
query19	1.78	1.71	1.75
query20	0.02	0.01	0.01
query21	15.41	0.63	0.62
query22	3.57	3.42	2.55
query23	17.81	1.11	1.15
query24	2.15	0.42	0.35
query25	0.45	0.15	0.08
query26	0.18	0.16	0.16
query27	0.05	0.05	0.05
query28	11.96	0.84	0.81
query29	12.62	3.35	3.35
query30	0.61	0.57	0.53
query31	2.79	0.37	0.36
query32	3.32	0.48	0.49
query33	3.23	3.21	3.21
query34	15.36	4.75	4.69
query35	4.75	4.71	4.74
query36	1.14	1.06	1.06
query37	0.07	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 106.73 s
Total hot run time: 32.57 s

@doris-robot
Copy link
Copy Markdown

doris-robot commented Feb 27, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1b1ef8a0db94ecf907c14cb3e275419aceaf9fb7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       17.2 seconds inserted 10000000 Rows, about 581K ops/s

shuke987
shuke987 approved these changes Feb 27, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@shuke987 shuke987 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 27, 2024

PR approved by anyone and no changes requested.

liaoxin01
liaoxin01 approved these changes Feb 27, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 27, 2024

PR approved by at least one committer and no changes requested.

@liaoxin01 liaoxin01 merged commit 4c7d7dd into apache:master Feb 27, 2024
@hello-stephen hello-stephen deleted the 0226-ci-log branch February 27, 2024 13:54
seawinde pushed a commit to seawinde/doris that referenced this pull request Feb 28, 2024
@hello-stephen @seawinde
[chore](ci) collect core and adjust log dir (apache#31418)
644506b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

+1 more reviewer

@shuke987 shuke987 shuke987 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @liaoxin01 @shuke987