Skip to content

[feat](load) introduce quorum success write to tolerate slow node(part II) #51208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[feat](load) introduce quorum success write to tolerate slow node(part II) #51208

wants to merge 1 commit into from

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented May 24, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@sollhui
Copy link
Contributor Author

sollhui commented May 24, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33801 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f2d9d151c3d400fc84c99b4a108914ffcba2809, data reload: false

------ Round 1 ----------------------------------
q1	26367	5762	5173	5173
q2	2115	284	187	187
q3	10554	1244	697	697
q4	10240	983	533	533
q5	7746	2401	2314	2314
q6	182	164	138	138
q7	920	754	613	613
q8	9302	1256	1061	1061
q9	6764	5055	4992	4992
q10	6857	2331	1876	1876
q11	499	314	282	282
q12	342	359	207	207
q13	17775	3728	3061	3061
q14	229	230	214	214
q15	525	476	492	476
q16	438	440	387	387
q17	636	860	388	388
q18	7627	7168	7067	7067
q19	2206	1254	568	568
q20	324	343	225	225
q21	3817	3135	2367	2367
q22	1060	1001	975	975
Total cold run time: 116525 ms
Total hot run time: 33801 ms

----- Round 2, with runtime_filter_mode=off -----
validate_password_policy	NONE	NONE	0
version	5.7.99	5.7.99	0
version_comment	Doris version doris-0.0.0--47fed59812	Doris version doris-0.0.0--47fed59812	0
wait_timeout	28800	28800	0
workload_group			0
============================================
show table status;
Name	Engine	Version	Row_format	Rows	Avg_row_length	Data_length	Max_data_length	Index_length	Data_free	Auto_increment	Create_time	Update_time	Check_time	Collation	Checksum	Create_options	Comment
revenue0	View	NULL	NULL	-1	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:24	NULL	NULL	utf-8	NULL	NULL	
partsupp	Doris	NULL	NULL	80000000	56	4534120086	NULL	44625495	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:44:20	NULL	utf-8	NULL	NULL	
part	Doris	NULL	NULL	20000000	37	748811035	NULL	1935627	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:57	NULL	utf-8	NULL	NULL	
nation	Doris	NULL	NULL	25	138	3473	NULL	366	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
customer	Doris	NULL	NULL	15000000	92	1381653732	NULL	4374759	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:43	NULL	utf-8	NULL	NULL	
lineitem	Doris	NULL	NULL	600037902	33	19843441616	NULL	61784740	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:38:59	NULL	utf-8	NULL	NULL	
supplier	Doris	NULL	NULL	1000000	87	87519212	NULL	194931	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:25	NULL	utf-8	NULL	NULL	
region	Doris	NULL	NULL	5	240	1201	NULL	147	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	5286	5095	5080	5080
q2	240	330	229	229
q3	2118	2717	2277	2277
q4	1362	1817	1484	1484
q5	q6	q7	q8	q9	q10	q11	q12	q13	q14	q15	q16	q17	q18	q19	q20	q21	q22	Total cold run time: 9006 ms
Total hot run time: 9070 ms

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 8f2d9d1 to 13bbcd9 Compare May 25, 2025 07:56
@sollhui
Copy link
Contributor Author

sollhui commented May 25, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34168 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ec98b14bf873a26423500f89fd0177b31369c29c, data reload: false

------ Round 1 ----------------------------------
q1	26828	5092	5033	5033
q2	2134	281	186	186
q3	10492	1244	709	709
q4	10238	988	519	519
q5	7647	2379	2344	2344
q6	186	170	134	134
q7	934	755	618	618
q8	9322	1265	1066	1066
q9	6930	5142	5230	5142
q10	6876	2313	1892	1892
q11	485	290	290	290
q12	349	363	226	226
q13	17806	3750	3101	3101
q14	233	246	222	222
q15	533	471	498	471
q16	449	438	390	390
q17	639	873	377	377
q18	7625	7219	7236	7219
q19	1415	942	569	569
q20	350	341	240	240
q21	4173	3292	2430	2430
q22	1024	1011	990	990
Total cold run time: 116668 ms
Total hot run time: 34168 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5135	5076	5073	5073
q2	257	335	231	231
q3	2183	2683	2342	2342
q4	1385	1780	1535	1535
q5	4377	4138	4165	4138
q6	216	171	130	130
q7	1864	1816	1703	1703
q8	2489	2386	2501	2386
q9	6842	6759	6842	6759
q10	2932	3081	2651	2651
q11	572	506	493	493
q12	657	728	616	616
q13	3389	3811	3124	3124
q14	275	286	277	277
q15	507	477	462	462
q16	444	486	430	430
q17	1132	1696	1343	1343
q18	7366	7207	7033	7033
q19	799	823	851	823
q20	1952	1948	1814	1814
q21	4850	4465	4411	4411
q22	1083	1050	997	997
Total cold run time: 50706 ms
Total hot run time: 48771 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189772 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec98b14bf873a26423500f89fd0177b31369c29c, data reload: false

query1	1099	590	567	567
query2	7673	2994	3020	2994
query3	6744	232	232	232
query4	31915	23969	23044	23044
query5	4635	623	481	481
query6	322	219	205	205
query7	4636	532	316	316
query8	304	262	246	246
query9	8652	2690	2668	2668
query10	464	364	297	297
query11	15737	15220	14878	14878
query12	196	122	118	118
query13	1681	573	422	422
query14	9981	6463	6638	6463
query15	211	196	168	168
query16	7644	653	494	494
query17	1504	742	640	640
query18	2033	418	327	327
query19	275	215	172	172
query20	126	133	122	122
query21	212	135	110	110
query22	4726	4839	4710	4710
query23	34201	33080	33112	33080
query24	8441	2452	2428	2428
query25	535	479	409	409
query26	1237	278	175	175
query27	2641	541	362	362
query28	4295	2185	2150	2150
query29	743	609	450	450
query30	295	232	199	199
query31	936	842	779	779
query32	76	62	68	62
query33	577	380	319	319
query34	843	918	576	576
query35	771	849	748	748
query36	974	1012	890	890
query37	126	109	83	83
query38	4228	4173	4134	4134
query39	1485	1593	1404	1404
query40	213	131	122	122
query41	68	54	55	54
query42	136	116	112	112
query43	532	515	486	486
query44	1402	875	855	855
query45	181	173	165	165
query46	881	1052	684	684
query47	1750	1809	1742	1742
query48	420	443	350	350
query49	795	521	447	447
query50	693	723	422	422
query51	4185	4170	4151	4151
query52	118	118	108	108
query53	235	273	190	190
query54	615	582	525	525
query55	107	86	93	86
query56	310	316	309	309
query57	1198	1189	1120	1120
query58	278	268	280	268
query59	2593	2749	2675	2675
query60	358	339	338	338
query61	131	128	123	123
query62	795	725	696	696
query63	238	201	195	195
query64	4211	1051	686	686
query65	4368	4258	4279	4258
query66	1046	436	326	326
query67	15963	15541	15750	15541
query68	7674	940	564	564
query69	496	333	288	288
query70	1270	1170	1142	1142
query71	501	336	319	319
query72	5515	4731	4766	4731
query73	750	646	376	376
query74	9160	9012	8904	8904
query75	3907	3227	2751	2751
query76	3756	1230	800	800
query77	780	412	326	326
query78	9625	9466	9104	9104
query79	2131	982	615	615
query80	623	544	461	461
query81	513	258	232	232
query82	467	137	99	99
query83	268	256	324	256
query84	253	117	99	99
query85	824	404	324	324
query86	389	319	286	286
query87	4407	4545	4363	4363
query88	3751	2392	2378	2378
query89	398	336	305	305
query90	1911	228	221	221
query91	158	174	113	113
query92	76	66	58	58
query93	1629	1037	596	596
query94	636	429	308	308
query95	391	304	301	301
query96	527	591	297	297
query97	2742	2787	2693	2693
query98	244	221	217	217
query99	1358	1388	1289	1289
Total cold run time: 284124 ms
Total hot run time: 189772 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.55 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec98b14bf873a26423500f89fd0177b31369c29c, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.11
query3	0.26	0.19	0.19
query4	1.59	0.19	0.20
query5	0.47	0.46	0.46
query6	1.18	0.67	0.65
query7	0.03	0.01	0.02
query8	0.05	0.04	0.04
query9	0.59	0.52	0.52
query10	0.59	0.58	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.61
query14	0.80	0.83	0.82
query15	0.90	0.86	0.87
query16	0.39	0.40	0.41
query17	1.05	1.09	1.10
query18	0.19	0.18	0.18
query19	1.96	1.89	1.84
query20	0.01	0.01	0.02
query21	15.41	0.88	0.56
query22	0.76	1.35	0.79
query23	14.76	1.39	0.67
query24	6.91	1.79	0.77
query25	0.50	0.25	0.08
query26	0.42	0.17	0.14
query27	0.05	0.06	0.06
query28	10.45	0.93	0.45
query29	12.60	3.97	3.28
query30	0.25	0.09	0.06
query31	2.83	0.61	0.40
query32	3.23	0.60	0.47
query33	3.04	3.09	3.10
query34	15.83	5.24	4.57
query35	4.53	4.52	4.55
query36	0.70	0.50	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 103.92 s
Total hot run time: 29.55 s

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from ec98b14 to 2e7feec Compare May 26, 2025 09:53
@sollhui
Copy link
Contributor Author

sollhui commented May 26, 2025

run buildall

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 2b6c229 to ed084c0 Compare May 26, 2025 13:14
@sollhui
Copy link
Contributor Author

sollhui commented May 26, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33598 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 19a76f772f61ff7f21db2c2a9dd4b48040efb25e, data reload: false

------ Round 1 ----------------------------------
q1	26085	5329	4992	4992
q2	2076	271	182	182
q3	10396	1274	706	706
q4	10221	980	530	530
q5	7538	2370	2313	2313
q6	177	165	133	133
q7	919	728	599	599
q8	9293	1291	1072	1072
q9	6773	5073	5144	5073
q10	6865	2300	1879	1879
q11	498	281	270	270
q12	332	350	206	206
q13	17777	3641	2999	2999
q14	230	224	214	214
q15	534	491	470	470
q16	422	437	370	370
q17	597	870	358	358
q18	7541	7304	7210	7210
q19	1388	961	537	537
q20	327	333	223	223
q21	3622	3161	2309	2309
q22	1020	1015	953	953
Total cold run time: 114631 ms
Total hot run time: 33598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5066	5035	5176	5035
q2	243	324	225	225
q3	2150	2669	2261	2261
q4	1360	1763	1353	1353
q5	4426	4160	4106	4106
q6	206	168	130	130
q7	1912	1829	1657	1657
q8	2472	2424	2373	2373
q9	6764	6768	6785	6768
q10	2882	3061	2617	2617
q11	561	485	516	485
q12	650	755	631	631
q13	3258	3648	3122	3122
q14	265	273	274	273
q15	510	462	467	462
q16	412	478	433	433
q17	1091	1483	1333	1333
q18	7483	7079	6983	6983
q19	790	855	999	855
q20	1910	1944	1791	1791
q21	4673	4320	4191	4191
q22	1058	1035	1015	1015
Total cold run time: 50142 ms
Total hot run time: 48099 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186974 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 19a76f772f61ff7f21db2c2a9dd4b48040efb25e, data reload: false

query1	1090	553	558	553
query2	7351	2725	2711	2711
query3	6763	224	228	224
query4	30984	23709	23609	23609
query5	5138	598	438	438
query6	311	208	198	198
query7	4632	503	305	305
query8	296	252	238	238
query9	8625	2596	2597	2596
query10	530	320	262	262
query11	15421	14979	14708	14708
query12	175	115	106	106
query13	1661	540	409	409
query14	9518	6133	6122	6122
query15	201	192	171	171
query16	7600	623	455	455
query17	1332	714	584	584
query18	2001	419	307	307
query19	196	191	165	165
query20	143	127	128	127
query21	211	134	114	114
query22	4877	4895	4782	4782
query23	33891	32792	33137	32792
query24	8478	2413	2356	2356
query25	538	464	393	393
query26	1228	270	150	150
query27	2714	513	345	345
query28	4274	2116	2109	2109
query29	726	565	459	459
query30	281	215	197	197
query31	898	819	765	765
query32	72	65	62	62
query33	555	359	313	313
query34	779	841	533	533
query35	777	798	738	738
query36	933	1014	896	896
query37	113	99	73	73
query38	4086	4042	4046	4042
query39	1434	1410	1415	1410
query40	207	117	110	110
query41	55	53	55	53
query42	128	112	107	107
query43	509	517	487	487
query44	1308	843	834	834
query45	176	166	171	166
query46	840	1027	640	640
query47	1760	1795	1703	1703
query48	388	422	326	326
query49	779	490	467	467
query50	644	680	412	412
query51	4179	4103	4158	4103
query52	111	111	107	107
query53	230	252	187	187
query54	585	600	518	518
query55	89	96	80	80
query56	297	305	321	305
query57	1175	1217	1107	1107
query58	274	248	264	248
query59	2634	2725	2580	2580
query60	338	328	307	307
query61	141	124	120	120
query62	775	727	678	678
query63	234	199	195	195
query64	4326	1005	662	662
query65	4335	4210	4213	4210
query66	1060	406	317	317
query67	15934	15828	15512	15512
query68	7933	891	528	528
query69	525	320	271	271
query70	1224	1104	1146	1104
query71	469	335	318	318
query72	5761	4675	4676	4675
query73	677	585	353	353
query74	8964	9071	8578	8578
query75	3866	3194	2666	2666
query76	3819	1184	759	759
query77	776	385	304	304
query78	9533	9521	9002	9002
query79	2188	872	584	584
query80	585	550	434	434
query81	491	266	222	222
query82	471	127	98	98
query83	266	251	234	234
query84	253	115	91	91
query85	747	361	307	307
query86	388	326	281	281
query87	4355	4496	4319	4319
query88	3806	2315	2288	2288
query89	377	327	296	296
query90	1980	209	203	203
query91	144	149	110	110
query92	76	57	57	57
query93	1781	997	588	588
query94	662	411	279	279
query95	372	314	290	290
query96	497	584	280	280
query97	2738	2769	2639	2639
query98	221	205	213	205
query99	1351	1414	1253	1253
Total cold run time: 281640 ms
Total hot run time: 186974 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.52 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 19a76f772f61ff7f21db2c2a9dd4b48040efb25e, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.09
query3	0.24	0.19	0.20
query4	1.62	0.18	0.20
query5	0.45	0.44	0.45
query6	1.55	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.51	0.52
query10	0.57	0.59	0.56
query11	0.16	0.10	0.11
query12	0.15	0.11	0.12
query13	0.61	0.61	0.60
query14	0.78	0.81	0.80
query15	0.87	0.86	0.85
query16	0.40	0.39	0.39
query17	1.06	1.03	1.07
query18	0.18	0.18	0.18
query19	1.91	1.80	1.82
query20	0.02	0.01	0.01
query21	15.39	0.88	0.53
query22	0.75	1.27	0.71
query23	14.84	1.36	0.63
query24	7.44	1.80	0.41
query25	0.50	0.13	0.06
query26	0.55	0.18	0.13
query27	0.05	0.05	0.05
query28	9.72	0.88	0.43
query29	12.52	4.06	3.39
query30	0.25	0.10	0.07
query31	2.83	0.61	0.39
query32	3.23	0.54	0.47
query33	3.07	3.01	3.12
query34	15.61	5.02	4.42
query35	4.47	4.49	4.44
query36	0.66	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.71 s
Total hot run time: 28.52 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.00% (0/215) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.03% (14933/26651)
Line Coverage 44.81% (132912/296611)
Region Coverage 43.88% (66801/152221)
Branch Coverage 38.49% (34268/89026)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/215) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.00% (15478/26234)
Line Coverage 47.99% (142338/296598)
Region Coverage 45.09% (80599/178743)
Branch Coverage 38.71% (39347/101658)

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 19a76f7 to 1b59229 Compare May 27, 2025 17:08
@sollhui
Copy link
Contributor Author

sollhui commented May 27, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34603 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b5922946cad21fc5f668eb752cf9e136c4000de, data reload: false

------ Round 1 ----------------------------------
q1	26180	5202	5071	5071
q2	2075	277	187	187
q3	10388	1257	689	689
q4	10235	1008	561	561
q5	7546	2342	2312	2312
q6	177	165	134	134
q7	906	767	622	622
q8	9312	1301	1082	1082
q9	6933	5060	5085	5060
q10	6873	2299	1916	1916
q11	489	295	267	267
q12	355	359	226	226
q13	17799	3711	3117	3117
q14	250	233	212	212
q15	520	490	481	481
q16	426	428	376	376
q17	593	856	369	369
q18	7521	7177	7160	7160
q19	1791	944	545	545
q20	342	335	229	229
q21	3841	3491	2989	2989
q22	1051	1020	998	998
Total cold run time: 115603 ms
Total hot run time: 34603 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5230	5190	5170	5170
q2	239	336	234	234
q3	2135	2679	2244	2244
q4	1331	1757	1527	1527
q5	4410	4134	4187	4134
q6	221	169	129	129
q7	1942	1822	1650	1650
q8	2458	2486	2412	2412
q9	6752	6684	6703	6684
q10	2870	3072	2663	2663
q11	552	499	499	499
q12	643	765	597	597
q13	3299	3847	3144	3144
q14	264	292	271	271
q15	515	478	459	459
q16	419	463	435	435
q17	1116	1477	1336	1336
q18	7292	7170	7163	7163
q19	776	790	990	790
q20	1927	2006	1811	1811
q21	4904	4535	4341	4341
q22	1070	1042	1007	1007
Total cold run time: 50365 ms
Total hot run time: 48700 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187532 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b5922946cad21fc5f668eb752cf9e136c4000de, data reload: false

query1	1086	560	549	549
query2	7678	2829	2910	2829
query3	6766	217	224	217
query4	31749	23820	23352	23352
query5	5077	606	454	454
query6	297	190	196	190
query7	4637	496	295	295
query8	295	256	253	253
query9	8623	2660	2643	2643
query10	511	311	276	276
query11	15551	15025	14820	14820
query12	169	113	114	113
query13	1657	526	409	409
query14	9631	6337	6198	6198
query15	208	185	164	164
query16	7641	643	441	441
query17	1135	684	558	558
query18	2003	417	313	313
query19	197	189	176	176
query20	125	116	116	116
query21	209	121	104	104
query22	4783	4765	4609	4609
query23	33978	33172	33248	33172
query24	8486	2412	2389	2389
query25	514	469	391	391
query26	1238	282	150	150
query27	2701	519	337	337
query28	4305	2131	2121	2121
query29	708	553	425	425
query30	287	214	195	195
query31	933	816	776	776
query32	80	74	63	63
query33	551	354	312	312
query34	771	852	519	519
query35	781	818	729	729
query36	935	957	866	866
query37	112	104	76	76
query38	4130	4168	4067	4067
query39	1568	1410	1417	1410
query40	213	122	115	115
query41	62	58	60	58
query42	133	119	116	116
query43	530	495	469	469
query44	1384	849	832	832
query45	187	174	169	169
query46	846	1035	646	646
query47	1756	1784	1684	1684
query48	399	431	332	332
query49	809	545	464	464
query50	646	672	428	428
query51	4117	4120	4070	4070
query52	121	111	101	101
query53	229	259	194	194
query54	605	580	510	510
query55	90	88	92	88
query56	313	311	331	311
query57	1191	1183	1127	1127
query58	274	250	271	250
query59	2568	2706	2555	2555
query60	328	359	302	302
query61	122	120	120	120
query62	765	746	659	659
query63	222	189	191	189
query64	4165	1002	672	672
query65	4350	4236	4246	4236
query66	1042	419	319	319
query67	15529	15557	15516	15516
query68	7771	894	519	519
query69	490	302	272	272
query70	1223	1165	1127	1127
query71	468	325	308	308
query72	5732	4747	4948	4747
query73	839	673	351	351
query74	8891	9198	8606	8606
query75	3934	3208	2707	2707
query76	3803	1205	782	782
query77	799	393	309	309
query78	9379	9485	8937	8937
query79	1890	937	592	592
query80	580	514	456	456
query81	521	265	232	232
query82	431	124	103	103
query83	252	253	230	230
query84	237	108	85	85
query85	736	375	314	314
query86	331	309	284	284
query87	4423	4501	4362	4362
query88	3204	2310	2313	2310
query89	380	331	299	299
query90	2013	218	221	218
query91	145	140	109	109
query92	79	62	57	57
query93	1050	976	576	576
query94	682	404	305	305
query95	381	300	279	279
query96	504	568	285	285
query97	2700	2780	2643	2643
query98	234	219	206	206
query99	1381	1397	1313	1313
Total cold run time: 280585 ms
Total hot run time: 187532 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1b5922946cad21fc5f668eb752cf9e136c4000de, data reload: false

query1	0.03	0.04	0.03
query2	0.12	0.11	0.11
query3	0.24	0.20	0.18
query4	1.59	0.19	0.19
query5	0.47	0.45	0.45
query6	1.17	0.66	0.67
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.61	0.51	0.52
query10	0.56	0.58	0.56
query11	0.15	0.12	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	0.80	0.80	0.82
query15	0.88	0.85	0.87
query16	0.39	0.37	0.37
query17	1.05	1.04	1.09
query18	0.20	0.19	0.18
query19	1.93	1.82	1.82
query20	0.02	0.02	0.01
query21	15.40	0.88	0.55
query22	0.76	1.09	0.62
query23	15.06	1.36	0.60
query24	6.76	1.60	0.88
query25	0.53	0.20	0.14
query26	0.54	0.17	0.13
query27	0.05	0.06	0.05
query28	9.46	0.90	0.45
query29	12.66	3.95	3.27
query30	0.26	0.09	0.07
query31	2.82	0.59	0.40
query32	3.23	0.55	0.47
query33	3.04	3.08	3.13
query34	15.73	5.02	4.49
query35	4.54	4.53	4.51
query36	0.68	0.51	0.48
query37	0.09	0.06	0.07
query38	0.06	0.05	0.03
query39	0.03	0.03	0.02
query40	0.16	0.14	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.03 s
Total hot run time: 29.14 s

@sollhui
Copy link
Contributor Author

sollhui commented May 28, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33915 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 68eac91f80a7eaf09d38b12a57c36e8fa475e194, data reload: false

------ Round 1 ----------------------------------
q1	26064	5086	5108	5086
q2	2059	293	202	202
q3	10473	1251	731	731
q4	10249	1004	536	536
q5	7656	2380	2326	2326
q6	189	161	131	131
q7	905	730	604	604
q8	9324	1312	1142	1142
q9	6829	5108	5079	5079
q10	6879	2316	1880	1880
q11	490	291	278	278
q12	351	350	213	213
q13	17794	3705	3132	3132
q14	225	232	207	207
q15	534	501	495	495
q16	413	428	375	375
q17	598	909	374	374
q18	7727	7318	6979	6979
q19	1611	979	568	568
q20	336	326	220	220
q21	3862	3197	2374	2374
q22	1089	1047	983	983
Total cold run time: 115657 ms
Total hot run time: 33915 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5144	5112	5106	5106
q2	239	331	231	231
q3	2198	2667	2299	2299
q4	1383	1853	1452	1452
q5	4587	4337	4403	4337
q6	207	166	124	124
q7	2005	1876	1719	1719
q8	2556	2549	2496	2496
q9	7187	7250	7111	7111
q10	3010	3202	2690	2690
q11	598	499	499	499
q12	670	775	616	616
q13	3475	3887	3262	3262
q14	280	310	280	280
q15	514	483	484	483
q16	461	489	454	454
q17	1141	1559	1362	1362
q18	7724	7566	7487	7487
q19	811	830	1087	830
q20	2003	2059	1866	1866
q21	4764	4292	4186	4186
q22	1096	1006	990	990
Total cold run time: 52053 ms
Total hot run time: 49880 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185641 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 68eac91f80a7eaf09d38b12a57c36e8fa475e194, data reload: false

query1	1010	479	503	479
query2	6563	1797	1822	1797
query3	6751	222	224	222
query4	26272	23381	22986	22986
query5	4348	632	464	464
query6	296	211	202	202
query7	4625	507	298	298
query8	313	273	239	239
query9	8630	2685	2681	2681
query10	477	323	274	274
query11	15339	15165	15017	15017
query12	156	119	107	107
query13	1667	544	415	415
query14	9039	6003	6044	6003
query15	203	184	168	168
query16	7309	633	481	481
query17	1207	739	579	579
query18	2039	407	300	300
query19	193	213	170	170
query20	126	121	120	120
query21	217	135	108	108
query22	4084	4149	3933	3933
query23	33766	33038	33081	33038
query24	8445	2385	2418	2385
query25	541	447	395	395
query26	1221	273	189	189
query27	2718	503	337	337
query28	4316	2154	2133	2133
query29	760	547	433	433
query30	286	218	183	183
query31	912	841	721	721
query32	75	72	63	63
query33	553	375	343	343
query34	819	850	540	540
query35	759	800	715	715
query36	950	957	854	854
query37	115	101	80	80
query38	4003	4121	3970	3970
query39	1477	1420	1406	1406
query40	213	119	104	104
query41	77	60	67	60
query42	129	113	115	113
query43	488	537	470	470
query44	1333	809	809	809
query45	176	182	177	177
query46	844	1036	649	649
query47	1765	1741	1703	1703
query48	386	412	326	326
query49	774	501	414	414
query50	631	703	398	398
query51	4179	4234	4089	4089
query52	112	116	99	99
query53	227	262	183	183
query54	582	586	514	514
query55	84	84	85	84
query56	315	301	293	293
query57	1144	1130	1070	1070
query58	268	252	256	252
query59	2518	2715	2604	2604
query60	345	313	303	303
query61	122	126	124	124
query62	787	729	738	729
query63	239	187	186	186
query64	4283	1005	664	664
query65	4338	4245	4287	4245
query66	1150	411	319	319
query67	15785	15619	15405	15405
query68	8085	895	523	523
query69	494	299	273	273
query70	1217	1125	1101	1101
query71	485	328	340	328
query72	5751	4687	4628	4628
query73	690	579	361	361
query74	9100	9086	8989	8989
query75	3912	3208	2703	2703
query76	3743	1221	762	762
query77	785	374	293	293
query78	10050	10198	9321	9321
query79	2511	816	587	587
query80	575	535	429	429
query81	500	255	212	212
query82	449	135	99	99
query83	284	260	234	234
query84	300	114	93	93
query85	786	365	313	313
query86	389	316	292	292
query87	4344	4388	4335	4335
query88	3596	2268	2265	2265
query89	395	308	279	279
query90	1867	201	206	201
query91	139	151	112	112
query92	74	60	57	57
query93	1757	923	573	573
query94	675	407	311	311
query95	367	281	280	280
query96	489	571	276	276
query97	2692	2718	2670	2670
query98	228	218	208	208
query99	1448	1410	1319	1319
Total cold run time: 274051 ms
Total hot run time: 185641 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 68eac91f80a7eaf09d38b12a57c36e8fa475e194, data reload: false

query1	0.03	0.03	0.03
query2	0.12	0.10	0.10
query3	0.25	0.19	0.18
query4	1.60	0.19	0.20
query5	0.43	0.42	0.42
query6	1.17	0.65	0.66
query7	0.03	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.51
query10	0.57	0.60	0.57
query11	0.15	0.11	0.10
query12	0.15	0.11	0.12
query13	0.61	0.61	0.60
query14	0.77	0.80	0.82
query15	0.86	0.87	0.88
query16	0.38	0.38	0.38
query17	1.02	1.05	1.06
query18	0.22	0.21	0.21
query19	1.94	1.82	1.82
query20	0.01	0.01	0.02
query21	15.40	0.89	0.53
query22	0.75	1.23	0.62
query23	14.95	1.36	0.64
query24	7.75	1.06	0.28
query25	0.28	0.07	0.08
query26	0.58	0.16	0.14
query27	0.05	0.05	0.06
query28	8.76	0.92	0.47
query29	12.57	4.01	3.29
query30	0.25	0.10	0.07
query31	2.81	0.59	0.38
query32	3.24	0.54	0.47
query33	3.08	3.03	3.19
query34	15.75	5.16	4.52
query35	4.52	4.52	4.58
query36	0.66	0.51	0.48
query37	0.09	0.07	0.06
query38	0.05	0.05	0.04
query39	0.04	0.03	0.02
query40	0.18	0.15	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 102.85 s
Total hot run time: 28.54 s

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 68eac91 to 7d76676 Compare May 28, 2025 09:36
@sollhui
Copy link
Contributor Author

sollhui commented May 28, 2025

run buildall

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 7d76676 to cb46b2d Compare May 28, 2025 09:42
@sollhui
Copy link
Contributor Author

sollhui commented May 28, 2025

run buildall

@sollhui
Copy link
Contributor Author

sollhui commented May 28, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33988 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d29276456d5af1c698bacf814304fd552c10775e, data reload: false

------ Round 1 ----------------------------------
q1	26473	5063	5149	5063
q2	2069	279	188	188
q3	10409	1253	710	710
q4	10230	1025	543	543
q5	7563	2383	2383	2383
q6	189	169	129	129
q7	880	744	602	602
q8	9314	1342	1075	1075
q9	6768	5104	5173	5104
q10	6824	2349	1907	1907
q11	488	292	261	261
q12	337	351	213	213
q13	17789	3704	3060	3060
q14	231	235	219	219
q15	539	502	485	485
q16	415	437	376	376
q17	618	874	376	376
q18	7839	7347	7193	7193
q19	1486	956	560	560
q20	343	338	223	223
q21	4114	3249	2373	2373
q22	1023	1022	945	945
Total cold run time: 115941 ms
Total hot run time: 33988 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5183	5091	5080	5080
q2	236	327	233	233
q3	2148	2676	2275	2275
q4	1326	1913	1473	1473
q5	4532	4487	4363	4363
q6	206	170	126	126
q7	1955	1923	1702	1702
q8	2567	2548	2485	2485
q9	7121	7105	7313	7105
q10	2967	3208	2750	2750
q11	578	502	491	491
q12	717	760	627	627
q13	3586	3997	3258	3258
q14	275	286	267	267
q15	514	481	484	481
q16	431	471	438	438
q17	1168	1596	1383	1383
q18	7735	7466	7345	7345
q19	804	844	924	844
q20	1967	2081	1825	1825
q21	4773	4273	4223	4223
q22	1045	1029	959	959
Total cold run time: 51834 ms
Total hot run time: 49733 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186159 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d29276456d5af1c698bacf814304fd552c10775e, data reload: false

query1	1017	507	501	501
query2	6559	1845	1842	1842
query3	6750	221	223	221
query4	26602	23330	23589	23330
query5	4341	629	480	480
query6	306	216	198	198
query7	4625	494	298	298
query8	265	233	217	217
query9	8624	2656	2679	2656
query10	488	345	273	273
query11	15282	15038	14724	14724
query12	163	110	104	104
query13	1647	520	446	446
query14	9435	6231	6275	6231
query15	206	188	174	174
query16	7264	627	483	483
query17	1197	708	551	551
query18	1969	384	288	288
query19	189	183	172	172
query20	123	111	117	111
query21	209	122	104	104
query22	4082	4320	4022	4022
query23	34026	33041	33172	33041
query24	8432	2389	2447	2389
query25	522	448	400	400
query26	1229	271	158	158
query27	2767	496	348	348
query28	4370	2135	2111	2111
query29	784	560	436	436
query30	276	214	185	185
query31	917	865	762	762
query32	71	64	63	63
query33	566	366	317	317
query34	803	828	555	555
query35	775	823	739	739
query36	970	975	907	907
query37	111	107	75	75
query38	4167	4194	4077	4077
query39	1512	1435	1413	1413
query40	212	117	107	107
query41	63	57	57	57
query42	122	107	108	107
query43	527	530	475	475
query44	1324	809	813	809
query45	181	178	170	170
query46	841	1024	637	637
query47	1786	1796	1734	1734
query48	397	412	327	327
query49	760	501	395	395
query50	650	680	405	405
query51	4118	4158	4057	4057
query52	113	104	96	96
query53	223	266	186	186
query54	581	582	507	507
query55	87	87	85	85
query56	320	308	298	298
query57	1116	1132	1043	1043
query58	277	277	255	255
query59	2655	2602	2598	2598
query60	320	328	308	308
query61	135	126	149	126
query62	814	729	656	656
query63	225	194	193	193
query64	4383	1142	775	775
query65	4329	4246	4246	4246
query66	1155	429	315	315
query67	15907	15431	15530	15431
query68	7775	888	530	530
query69	484	316	283	283
query70	1225	1082	1100	1082
query71	422	335	298	298
query72	5480	4701	4707	4701
query73	674	609	358	358
query74	8911	9310	8726	8726
query75	3455	3181	2701	2701
query76	3392	1181	765	765
query77	759	378	299	299
query78	10109	10186	9503	9503
query79	2306	865	582	582
query80	663	510	440	440
query81	503	260	223	223
query82	451	123	101	101
query83	257	247	238	238
query84	250	113	95	95
query85	824	360	315	315
query86	380	316	291	291
query87	4344	4368	4275	4275
query88	3980	2348	2292	2292
query89	406	315	282	282
query90	1901	212	204	204
query91	139	142	115	115
query92	71	64	58	58
query93	2153	933	570	570
query94	707	387	299	299
query95	374	290	288	288
query96	496	557	282	282
query97	2753	2794	2695	2695
query98	234	214	209	209
query99	1421	1417	1259	1259
Total cold run time: 274561 ms
Total hot run time: 186159 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d29276456d5af1c698bacf814304fd552c10775e, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.25	0.18	0.19
query4	1.60	0.18	0.19
query5	0.45	0.44	0.46
query6	1.49	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.52
query10	0.57	0.58	0.57
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.60
query14	0.80	0.82	0.80
query15	0.86	0.84	0.86
query16	0.40	0.38	0.41
query17	1.00	1.03	1.02
query18	0.22	0.21	0.21
query19	1.96	1.80	1.85
query20	0.02	0.01	0.01
query21	15.40	0.90	0.52
query22	0.78	1.11	0.88
query23	14.76	1.41	0.68
query24	7.12	2.22	0.61
query25	0.47	0.26	0.05
query26	0.54	0.16	0.13
query27	0.05	0.05	0.06
query28	9.30	0.89	0.46
query29	12.68	4.02	3.33
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.23	0.55	0.47
query33	3.04	3.05	3.05
query34	15.74	5.09	4.48
query35	4.52	4.47	4.49
query36	0.69	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.05	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 103.2 s
Total hot run time: 29.02 s

@sollhui
Copy link
Contributor Author

sollhui commented May 29, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34077 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5d23b2c5bbed95f0284a8b638cc89f797e50c0b7, data reload: false

------ Round 1 ----------------------------------
q1	26310	5080	5040	5040
q2	2005	280	185	185
q3	10361	1249	725	725
q4	10237	1018	520	520
q5	7643	2412	2377	2377
q6	183	161	129	129
q7	919	742	617	617
q8	9318	1306	1131	1131
q9	6820	5197	5088	5088
q10	6890	2320	1931	1931
q11	509	297	292	292
q12	349	353	221	221
q13	17767	3672	3094	3094
q14	232	229	222	222
q15	581	497	496	496
q16	439	432	379	379
q17	627	886	386	386
q18	7735	7237	7095	7095
q19	2119	990	582	582
q20	339	346	231	231
q21	3903	3213	2368	2368
q22	1055	1014	968	968
Total cold run time: 116341 ms
Total hot run time: 34077 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5225	5104	5103	5103
q2	245	325	228	228
q3	2172	2670	2333	2333
q4	1378	1771	1494	1494
q5	4587	4425	4363	4363
q6	219	162	122	122
q7	1967	1932	1781	1781
q8	2607	2561	2617	2561
q9	7196	7084	7174	7084
q10	3006	3175	2795	2795
q11	578	518	494	494
q12	673	786	637	637
q13	3522	3864	3244	3244
q14	281	296	270	270
q15	529	486	483	483
q16	458	477	442	442
q17	1181	1560	1342	1342
q18	7920	7603	7305	7305
q19	859	917	996	917
q20	1979	2045	1925	1925
q21	4883	4304	4302	4302
q22	1089	1039	1001	1001
Total cold run time: 52554 ms
Total hot run time: 50226 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185989 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5d23b2c5bbed95f0284a8b638cc89f797e50c0b7, data reload: false

query1	1024	484	494	484
query2	6573	1805	1797	1797
query3	6762	231	233	231
query4	26500	23347	23811	23347
query5	4292	618	472	472
query6	290	209	194	194
query7	4606	501	286	286
query8	259	222	213	213
query9	8597	2627	2630	2627
query10	506	363	277	277
query11	15305	14975	14924	14924
query12	158	109	107	107
query13	1655	533	427	427
query14	8759	6225	6244	6225
query15	203	198	173	173
query16	7125	622	467	467
query17	910	712	555	555
query18	1965	386	297	297
query19	189	184	158	158
query20	120	114	110	110
query21	225	126	106	106
query22	3973	4164	3993	3993
query23	33875	33032	33208	33032
query24	8601	2394	2373	2373
query25	534	457	394	394
query26	1238	269	148	148
query27	2775	512	344	344
query28	4318	2163	2141	2141
query29	797	560	437	437
query30	287	219	189	189
query31	914	856	759	759
query32	71	61	67	61
query33	555	357	344	344
query34	789	871	526	526
query35	775	803	732	732
query36	959	984	889	889
query37	109	95	77	77
query38	4226	4099	4112	4099
query39	1486	1447	1390	1390
query40	212	115	108	108
query41	63	67	57	57
query42	135	109	108	108
query43	504	485	459	459
query44	1326	839	829	829
query45	180	174	171	171
query46	827	1032	626	626
query47	1755	1785	1753	1753
query48	391	419	318	318
query49	757	489	386	386
query50	652	677	402	402
query51	4140	4287	4134	4134
query52	112	105	96	96
query53	228	261	185	185
query54	584	570	514	514
query55	92	86	83	83
query56	336	292	287	287
query57	1123	1135	1104	1104
query58	260	250	252	250
query59	2508	2670	2421	2421
query60	344	337	320	320
query61	126	128	127	127
query62	783	719	655	655
query63	219	192	197	192
query64	4335	996	710	710
query65	4249	4117	4165	4117
query66	1164	438	329	329
query67	15883	15446	15276	15276
query68	5661	888	539	539
query69	497	317	279	279
query70	1162	1110	1146	1110
query71	408	344	326	326
query72	5686	4846	4902	4846
query73	688	675	363	363
query74	9055	9114	8778	8778
query75	3173	3195	2702	2702
query76	3171	1179	757	757
query77	473	373	297	297
query78	10199	10159	9376	9376
query79	2741	781	583	583
query80	635	512	445	445
query81	497	254	228	228
query82	452	127	96	96
query83	252	254	235	235
query84	288	103	94	94
query85	863	357	320	320
query86	384	296	304	296
query87	4380	4527	4324	4324
query88	3987	2332	2354	2332
query89	392	318	293	293
query90	1884	214	207	207
query91	147	142	114	114
query92	73	64	59	59
query93	2432	936	586	586
query94	718	414	304	304
query95	381	297	290	290
query96	496	578	292	292
query97	2761	2743	2635	2635
query98	229	204	204	204
query99	1308	1404	1257	1257
Total cold run time: 271066 ms
Total hot run time: 185989 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5d23b2c5bbed95f0284a8b638cc89f797e50c0b7, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.24	0.20	0.18
query4	1.59	0.19	0.20
query5	0.45	0.42	0.42
query6	1.17	0.67	0.66
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.57	0.51	0.53
query10	0.56	0.58	0.56
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.62	0.61	0.60
query14	0.78	0.80	0.82
query15	0.86	0.85	0.85
query16	0.38	0.38	0.38
query17	1.03	1.06	1.04
query18	0.23	0.21	0.21
query19	1.93	1.79	1.87
query20	0.02	0.01	0.01
query21	15.41	0.91	0.53
query22	0.75	0.97	0.68
query23	15.12	1.39	0.67
query24	6.35	1.86	0.72
query25	0.53	0.18	0.11
query26	0.53	0.17	0.15
query27	0.05	0.05	0.04
query28	9.84	0.89	0.44
query29	12.55	3.97	3.32
query30	0.25	0.10	0.07
query31	2.81	0.58	0.40
query32	3.23	0.55	0.48
query33	3.10	3.09	3.14
query34	15.77	5.04	4.53
query35	4.48	4.54	4.48
query36	0.69	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 102.92 s
Total hot run time: 29.1 s

@sollhui sollhui force-pushed the quorum_success_write_memtable_on_sink branch from 5d23b2c to 7fb4030 Compare May 30, 2025 14:45
@sollhui
Copy link
Contributor Author

sollhui commented May 30, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33751 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fb4030cb28f3b5d1f45280e3ae0ff536a78aa10, data reload: false

------ Round 1 ----------------------------------
q1	26361	5073	5010	5010
q2	1958	293	195	195
q3	10352	1219	731	731
q4	10234	989	524	524
q5	7610	2423	2318	2318
q6	183	163	133	133
q7	914	736	600	600
q8	9302	1301	1098	1098
q9	6802	5071	5076	5071
q10	6878	2347	1901	1901
q11	481	280	275	275
q12	352	352	214	214
q13	17784	3711	3086	3086
q14	222	233	225	225
q15	568	500	502	500
q16	434	429	377	377
q17	598	867	371	371
q18	7761	7253	7044	7044
q19	1641	941	564	564
q20	336	336	234	234
q21	3830	3137	2317	2317
q22	1049	1002	963	963
Total cold run time: 115650 ms
Total hot run time: 33751 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5188	5095	5093	5093
q2	236	312	222	222
q3	2156	2647	2253	2253
q4	1356	1798	1368	1368
q5	4452	4428	4456	4428
q6	273	175	129	129
q7	2009	1929	1792	1792
q8	2586	2560	2466	2466
q9	7182	7220	6876	6876
q10	3013	3212	2753	2753
q11	568	519	512	512
q12	687	757	622	622
q13	3516	3890	3319	3319
q14	288	297	286	286
q15	519	478	472	472
q16	444	485	462	462
q17	1174	1529	1417	1417
q18	7788	7663	7465	7465
q19	812	845	860	845
q20	2029	2096	1862	1862
q21	4753	4378	4467	4378
q22	1120	1047	996	996
Total cold run time: 52149 ms
Total hot run time: 50016 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191816 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fb4030cb28f3b5d1f45280e3ae0ff536a78aa10, data reload: false

query1	1409	1095	1038	1038
query2	6170	1801	1799	1799
query3	10987	4571	4461	4461
query4	53836	25841	23473	23473
query5	5152	546	461	461
query6	366	209	193	193
query7	4996	489	305	305
query8	289	230	226	226
query9	5985	2625	2640	2625
query10	446	343	271	271
query11	15141	15001	14825	14825
query12	168	116	103	103
query13	1105	520	403	403
query14	10270	6349	6341	6341
query15	194	205	201	201
query16	7156	696	459	459
query17	1086	717	570	570
query18	1550	421	313	313
query19	196	183	158	158
query20	129	132	125	125
query21	212	125	107	107
query22	4367	4288	4287	4287
query23	34379	33658	33474	33474
query24	6536	2398	2428	2398
query25	483	467	410	410
query26	720	277	156	156
query27	2333	522	350	350
query28	2992	2141	2138	2138
query29	580	600	439	439
query30	272	224	190	190
query31	883	846	773	773
query32	75	62	58	58
query33	480	363	286	286
query34	787	875	542	542
query35	817	813	745	745
query36	945	1008	888	888
query37	118	100	73	73
query38	4262	4238	4116	4116
query39	1528	1464	1441	1441
query40	206	124	108	108
query41	63	63	55	55
query42	138	110	117	110
query43	507	519	464	464
query44	1335	838	829	829
query45	186	175	168	168
query46	857	1029	648	648
query47	1828	1866	1764	1764
query48	397	439	323	323
query49	675	493	415	415
query50	685	714	429	429
query51	4255	4262	4215	4215
query52	112	122	114	114
query53	225	261	181	181
query54	605	577	532	532
query55	82	86	80	80
query56	358	314	280	280
query57	1147	1195	1130	1130
query58	265	265	254	254
query59	2702	2757	2713	2713
query60	359	337	320	320
query61	132	131	133	131
query62	752	746	670	670
query63	225	189	183	183
query64	1817	1042	689	689
query65	4258	4138	4158	4138
query66	713	405	304	304
query67	15936	15654	15227	15227
query68	6057	892	523	523
query69	541	304	272	272
query70	1176	1093	1077	1077
query71	455	324	297	297
query72	5765	4764	4687	4687
query73	1206	587	359	359
query74	9056	9069	8630	8630
query75	3231	3201	2706	2706
query76	3884	1193	747	747
query77	552	374	299	299
query78	10056	10212	9436	9436
query79	2493	785	577	577
query80	651	501	435	435
query81	484	276	220	220
query82	184	133	101	101
query83	252	352	239	239
query84	288	109	91	91
query85	800	352	310	310
query86	358	302	294	294
query87	4406	4501	4287	4287
query88	3367	2284	2269	2269
query89	404	311	280	280
query90	1691	216	216	216
query91	144	137	118	118
query92	76	63	55	55
query93	1940	928	558	558
query94	682	420	319	319
query95	365	310	285	285
query96	500	570	284	284
query97	2695	2763	2685	2685
query98	229	205	198	198
query99	1324	1400	1267	1267
Total cold run time: 295532 ms
Total hot run time: 191816 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fb4030cb28f3b5d1f45280e3ae0ff536a78aa10, data reload: false

query1	0.03	0.03	0.03
query2	0.13	0.11	0.10
query3	0.25	0.19	0.19
query4	1.59	0.20	0.19
query5	0.45	0.43	0.44
query6	1.14	0.67	0.65
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.59	0.51	0.52
query10	0.57	0.58	0.56
query11	0.17	0.10	0.11
query12	0.16	0.12	0.12
query13	0.62	0.59	0.60
query14	0.80	0.81	0.82
query15	0.89	0.87	0.85
query16	0.37	0.38	0.39
query17	1.05	1.04	1.03
query18	0.22	0.21	0.20
query19	1.92	1.88	1.83
query20	0.01	0.01	0.02
query21	15.42	0.91	0.54
query22	0.75	1.32	0.68
query23	14.79	1.37	0.65
query24	6.82	1.72	0.78
query25	0.51	0.25	0.13
query26	0.57	0.15	0.14
query27	0.06	0.04	0.05
query28	10.06	0.95	0.45
query29	12.57	4.18	3.40
query30	0.25	0.09	0.07
query31	2.82	0.60	0.39
query32	3.23	0.55	0.48
query33	3.05	3.08	3.05
query34	15.74	5.11	4.46
query35	4.48	4.48	4.49
query36	0.67	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.05	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.04	0.03
Total cold run time: 103.28 s
Total hot run time: 29.17 s

@sollhui sollhui mentioned this pull request Jun 3, 2025
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sollhui @Thearas @doris-robot @hello-stephen