Skip to content

[Fix](compaction) Failed compaction tablets should sleep before being selected again #50672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
May 19, 2025
Merged

[Fix](compaction) Failed compaction tablets should sleep before being selected again #50672

merged 6 commits into from
May 19, 2025

Conversation

Yukang-Lian
Copy link
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Yukang-Lian
Copy link
Collaborator Author

run buildall

luwei16
luwei16 reviewed May 7, 2025
View reviewed changes
Copy link
Contributor

@luwei16 luwei16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add ut

@doris-robot
Copy link

TPC-H: Total hot run time: 34029 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03906a9a576c5efb5913de0af780305231634337, data reload: false

------ Round 1 ----------------------------------
q1	26221	5020	5056	5020
q2	2065	269	178	178
q3	10406	1221	700	700
q4	10230	1012	566	566
q5	7487	2409	2302	2302
q6	184	169	135	135
q7	943	718	615	615
q8	9317	1289	1102	1102
q9	6815	5163	5108	5108
q10	6866	2288	1895	1895
q11	496	289	277	277
q12	351	355	233	233
q13	17766	3685	3098	3098
q14	236	225	212	212
q15	521	479	509	479
q16	409	418	370	370
q17	598	863	363	363
q18	7567	7224	7119	7119
q19	1413	947	564	564
q20	352	332	216	216
q21	4053	3387	2469	2469
q22	1027	1008	1040	1008
Total cold run time: 115323 ms
Total hot run time: 34029 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5210	5117	5149	5117
q2	238	323	227	227
q3	2168	2671	2294	2294
q4	1375	1825	1404	1404
q5	4433	4434	4383	4383
q6	215	167	133	133
q7	1999	1937	1768	1768
q8	2547	2620	2567	2567
q9	7123	7206	6999	6999
q10	3005	3155	2748	2748
q11	562	517	476	476
q12	701	767	626	626
q13	3618	3942	3307	3307
q14	299	315	286	286
q15	505	482	465	465
q16	441	481	446	446
q17	1168	1517	1356	1356
q18	7637	7559	7460	7460
q19	853	877	1025	877
q20	2034	1993	1836	1836
q21	5190	4712	4683	4683
q22	1087	1065	980	980
Total cold run time: 52408 ms
Total hot run time: 50438 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192630 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03906a9a576c5efb5913de0af780305231634337, data reload: false

query1	1432	1097	1097	1097
query2	6310	1829	1801	1801
query3	11042	4447	4523	4447
query4	54624	26240	23394	23394
query5	5240	492	449	449
query6	394	200	199	199
query7	5170	508	284	284
query8	325	258	237	237
query9	6877	2548	2577	2548
query10	430	328	269	269
query11	15030	15169	14757	14757
query12	160	134	100	100
query13	1211	509	372	372
query14	9983	6183	6383	6183
query15	220	195	179	179
query16	7125	644	519	519
query17	1056	719	588	588
query18	1618	435	317	317
query19	195	203	169	169
query20	131	128	130	128
query21	211	127	114	114
query22	4466	4520	4382	4382
query23	34230	33472	33414	33414
query24	6436	2421	2419	2419
query25	492	493	439	439
query26	684	295	157	157
query27	2240	514	341	341
query28	3135	2152	2122	2122
query29	599	580	463	463
query30	286	221	189	189
query31	852	901	807	807
query32	80	71	65	65
query33	484	369	338	338
query34	779	922	542	542
query35	806	816	787	787
query36	930	1016	881	881
query37	114	96	78	78
query38	4175	4209	4127	4127
query39	1471	1416	1439	1416
query40	218	130	122	122
query41	56	53	52	52
query42	125	110	114	110
query43	504	520	488	488
query44	1354	815	814	814
query45	180	175	165	165
query46	832	1012	657	657
query47	1887	1877	1787	1787
query48	379	406	304	304
query49	723	530	437	437
query50	664	686	413	413
query51	4145	4336	4108	4108
query52	117	109	92	92
query53	237	260	194	194
query54	591	591	531	531
query55	85	80	89	80
query56	287	311	298	298
query57	1174	1227	1176	1176
query58	267	266	265	265
query59	2700	2854	2711	2711
query60	331	313	310	310
query61	137	127	161	127
query62	752	755	708	708
query63	231	195	198	195
query64	1774	1041	672	672
query65	4283	4211	4244	4211
query66	727	398	303	303
query67	15824	15585	15475	15475
query68	8231	919	512	512
query69	541	325	261	261
query70	1198	1088	1111	1088
query71	485	331	297	297
query72	5711	4822	4977	4822
query73	1257	693	344	344
query74	8918	9136	8907	8907
query75	3766	3199	2712	2712
query76	4322	1187	759	759
query77	616	360	322	322
query78	10045	10034	9196	9196
query79	2277	804	580	580
query80	636	513	454	454
query81	483	265	220	220
query82	448	128	96	96
query83	262	253	236	236
query84	299	114	90	90
query85	808	361	313	313
query86	419	305	283	283
query87	4485	4427	4280	4280
query88	3613	2232	2236	2232
query89	414	311	282	282
query90	1839	213	208	208
query91	147	137	111	111
query92	78	68	59	59
query93	1822	945	572	572
query94	662	393	336	336
query95	373	297	288	288
query96	479	549	276	276
query97	3169	3190	3100	3100
query98	226	217	197	197
query99	1649	1379	1274	1274
Total cold run time: 301305 ms
Total hot run time: 192630 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03906a9a576c5efb5913de0af780305231634337, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.24	0.20	0.19
query4	1.59	0.20	0.11
query5	0.56	0.54	0.56
query6	1.17	0.71	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.52	0.50
query10	0.57	0.57	0.57
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.60
query14	0.77	0.81	0.80
query15	0.88	0.85	0.86
query16	0.40	0.37	0.38
query17	1.03	1.02	1.00
query18	0.21	0.20	0.20
query19	1.87	1.84	1.83
query20	0.02	0.01	0.02
query21	15.80	0.90	0.57
query22	0.88	1.16	0.68
query23	15.13	1.37	0.62
query24	7.00	0.67	1.08
query25	0.53	0.14	0.16
query26	0.62	0.16	0.14
query27	0.06	0.05	0.04
query28	9.60	0.92	0.44
query29	12.57	3.97	3.22
query30	0.25	0.09	0.07
query31	2.82	0.61	0.38
query32	3.23	0.56	0.46
query33	3.09	3.07	3.11
query34	15.75	5.10	4.48
query35	4.49	4.51	4.45
query36	0.66	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 103.97 s
Total hot run time: 28.9 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.60% (14870/26747)
Line Coverage 44.63% (131208/294006)
Region Coverage 43.35% (66836/154162)
Branch Coverage 37.94% (33730/88902)

@Yukang-Lian
Copy link
Collaborator Author

run cloud_p0

@Yukang-Lian Yukang-Lian requested a review from w41ter as a code owner May 12, 2025 11:19
@Yukang-Lian Yukang-Lian force-pushed the Fail-Compaction-Should-Sleep branch from ab58b73 to 1a046ce Compare May 12, 2025 11:20
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33970 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a046ce0cc16233ccadb36b1f745f4a5b1a9ceff, data reload: false

------ Round 1 ----------------------------------
q1	26630	5249	4986	4986
q2	2081	295	188	188
q3	10439	1289	708	708
q4	10249	1017	534	534
q5	7893	2413	2341	2341
q6	186	162	131	131
q7	928	745	616	616
q8	9332	1305	1014	1014
q9	7496	5212	5077	5077
q10	6837	2312	1901	1901
q11	473	288	285	285
q12	344	349	213	213
q13	17848	3724	3095	3095
q14	245	228	216	216
q15	557	508	505	505
q16	432	433	373	373
q17	615	879	401	401
q18	7559	7189	7163	7163
q19	1228	947	558	558
q20	354	362	237	237
q21	4056	2652	2459	2459
q22	1022	1007	969	969
Total cold run time: 116804 ms
Total hot run time: 33970 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5062	5071	5036	5036
q2	238	329	232	232
q3	2161	2673	2290	2290
q4	1343	1779	1392	1392
q5	4544	4496	4426	4426
q6	211	165	122	122
q7	1961	1897	1733	1733
q8	2603	2520	2428	2428
q9	7262	7138	7162	7138
q10	3063	3189	2741	2741
q11	586	536	478	478
q12	689	759	652	652
q13	3491	3959	3270	3270
q14	288	297	264	264
q15	531	510	497	497
q16	466	482	436	436
q17	1135	1558	1358	1358
q18	7757	7649	7512	7512
q19	829	806	929	806
q20	2048	1986	1812	1812
q21	5164	4505	4564	4505
q22	1060	1042	972	972
Total cold run time: 52492 ms
Total hot run time: 50100 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187248 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a046ce0cc16233ccadb36b1f745f4a5b1a9ceff, data reload: false

query1	998	483	516	483
query2	6576	1808	1790	1790
query3	6760	232	229	229
query4	26366	24183	23031	23031
query5	4321	637	455	455
query6	285	212	204	204
query7	4620	507	305	305
query8	304	272	248	248
query9	8621	2672	2685	2672
query10	469	330	277	277
query11	15324	15162	15106	15106
query12	171	116	113	113
query13	1683	533	409	409
query14	8719	6088	6156	6088
query15	202	193	168	168
query16	7129	649	460	460
query17	1185	711	582	582
query18	1968	391	306	306
query19	189	182	161	161
query20	124	118	117	117
query21	222	126	106	106
query22	4239	4198	4095	4095
query23	34123	33348	33081	33081
query24	8413	2406	2397	2397
query25	520	469	407	407
query26	1248	280	159	159
query27	2744	510	339	339
query28	4355	2146	2119	2119
query29	762	561	433	433
query30	279	206	184	184
query31	936	861	776	776
query32	71	68	62	62
query33	568	373	340	340
query34	788	859	517	517
query35	783	814	741	741
query36	963	978	922	922
query37	104	97	88	88
query38	4096	4259	4107	4107
query39	1465	1429	1417	1417
query40	231	125	115	115
query41	64	60	58	58
query42	135	114	113	113
query43	509	521	473	473
query44	1353	834	827	827
query45	182	179	169	169
query46	863	1035	651	651
query47	1781	1811	1741	1741
query48	386	458	315	315
query49	816	547	463	463
query50	659	684	409	409
query51	4158	4203	4159	4159
query52	107	114	111	111
query53	241	283	194	194
query54	607	591	528	528
query55	90	91	90	90
query56	325	318	317	317
query57	1142	1151	1127	1127
query58	283	275	315	275
query59	2544	2602	2521	2521
query60	320	327	308	308
query61	126	126	128	126
query62	792	733	659	659
query63	235	193	199	193
query64	4309	981	677	677
query65	4334	4261	4240	4240
query66	1135	469	328	328
query67	16023	15507	15319	15319
query68	8887	892	522	522
query69	488	316	280	280
query70	1158	1124	1130	1124
query71	466	378	303	303
query72	5241	4735	4750	4735
query73	709	630	362	362
query74	8907	9045	8824	8824
query75	4254	3268	2736	2736
query76	3625	1211	765	765
query77	874	388	300	300
query78	10215	10178	9403	9403
query79	2538	809	584	584
query80	631	519	452	452
query81	497	258	232	232
query82	510	129	107	107
query83	297	255	243	243
query84	301	113	91	91
query85	792	352	309	309
query86	387	316	290	290
query87	4397	4541	4411	4411
query88	3552	2320	2340	2320
query89	399	314	300	300
query90	1832	237	224	224
query91	157	148	115	115
query92	77	64	61	61
query93	1840	936	580	580
query94	667	424	315	315
query95	381	301	292	292
query96	496	579	293	293
query97	3201	3256	3158	3158
query98	237	210	205	205
query99	1473	1428	1312	1312
Total cold run time: 275929 ms
Total hot run time: 187248 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a046ce0cc16233ccadb36b1f745f4a5b1a9ceff, data reload: false

query1	0.04	0.03	0.04
query2	0.12	0.11	0.10
query3	0.26	0.19	0.19
query4	1.59	0.20	0.10
query5	0.57	0.55	0.58
query6	1.19	0.73	0.70
query7	0.03	0.01	0.02
query8	0.04	0.03	0.04
query9	0.57	0.51	0.51
query10	0.57	0.57	0.58
query11	0.16	0.11	0.12
query12	0.16	0.11	0.12
query13	0.61	0.60	0.60
query14	0.79	0.81	0.82
query15	0.89	0.85	0.89
query16	0.38	0.39	0.40
query17	1.10	1.08	1.04
query18	0.23	0.22	0.22
query19	1.95	1.79	1.89
query20	0.02	0.01	0.01
query21	15.46	0.91	0.54
query22	0.75	1.23	0.79
query23	14.66	1.36	0.63
query24	7.30	0.85	1.08
query25	0.50	0.15	0.17
query26	0.63	0.16	0.15
query27	0.06	0.05	0.05
query28	9.56	0.93	0.44
query29	12.63	3.97	3.29
query30	0.26	0.10	0.07
query31	2.81	0.59	0.40
query32	3.24	0.55	0.46
query33	3.12	3.15	3.05
query34	15.83	5.10	4.51
query35	4.50	4.52	4.49
query36	0.68	0.50	0.49
query37	0.08	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.74 s
Total hot run time: 29.41 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 77.78% (7/9) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.83% (14912/26710)
Line Coverage 44.62% (131904/295646)
Region Coverage 43.68% (66336/151885)
Branch Coverage 38.27% (33994/88818)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 33.33% (3/9) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.37% (20854/26276)
Line Coverage 72.54% (214425/295582)
Region Coverage 70.69% (126100/178385)
Branch Coverage 64.41% (65326/101426)

gavinchou
gavinchou requested changes May 14, 2025
View reviewed changes
@Yukang-Lian Yukang-Lian force-pushed the Fail-Compaction-Should-Sleep branch from 1a046ce to a4ff7c9 Compare May 15, 2025 08:46
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34184 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a4ff7c9663da3be3ed4170207432359a63031b0f, data reload: false

------ Round 1 ----------------------------------
q1	26147	5138	5169	5138
q2	2093	304	190	190
q3	10356	1226	700	700
q4	10232	991	529	529
q5	7517	2425	2426	2425
q6	185	166	131	131
q7	919	775	612	612
q8	9335	1366	1076	1076
q9	6883	5079	5175	5079
q10	6807	2343	1892	1892
q11	491	290	287	287
q12	351	348	210	210
q13	17764	3736	3111	3111
q14	229	234	222	222
q15	562	520	501	501
q16	427	431	372	372
q17	630	894	389	389
q18	7747	7244	7147	7147
q19	1211	960	577	577
q20	343	357	235	235
q21	4036	3190	2389	2389
q22	1055	1045	972	972
Total cold run time: 115320 ms
Total hot run time: 34184 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5130	5131	5069	5069
q2	247	337	230	230
q3	2188	2662	2322	2322
q4	1379	1847	1427	1427
q5	4589	4451	4358	4358
q6	209	168	126	126
q7	1946	1913	1791	1791
q8	2597	2543	2556	2543
q9	7284	7243	7122	7122
q10	3007	3241	2812	2812
q11	577	504	504	504
q12	678	742	619	619
q13	3515	3908	3326	3326
q14	292	297	262	262
q15	542	493	517	493
q16	449	487	450	450
q17	1174	1548	1367	1367
q18	7893	7586	7329	7329
q19	824	822	888	822
q20	1965	2085	1851	1851
q21	4845	4394	4290	4290
q22	1121	1052	1008	1008
Total cold run time: 52451 ms
Total hot run time: 50121 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186023 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a4ff7c9663da3be3ed4170207432359a63031b0f, data reload: false

query1	1012	513	534	513
query2	6591	1813	1775	1775
query3	6745	221	227	221
query4	26039	23733	23488	23488
query5	4310	611	460	460
query6	310	216	211	211
query7	4627	506	282	282
query8	318	264	231	231
query9	8585	2635	2638	2635
query10	488	321	269	269
query11	15708	15073	14837	14837
query12	162	112	106	106
query13	1670	559	423	423
query14	9598	6241	6155	6155
query15	205	204	166	166
query16	7285	652	481	481
query17	1215	747	628	628
query18	1975	401	294	294
query19	188	179	169	169
query20	118	119	118	118
query21	211	120	109	109
query22	3965	4217	4028	4028
query23	34295	33196	33095	33095
query24	8472	2393	2378	2378
query25	545	476	402	402
query26	1243	270	153	153
query27	2734	506	333	333
query28	4323	2104	2088	2088
query29	779	554	426	426
query30	278	215	190	190
query31	957	854	802	802
query32	72	64	67	64
query33	566	365	362	362
query34	785	851	529	529
query35	779	805	712	712
query36	929	976	899	899
query37	108	99	74	74
query38	4134	4160	4049	4049
query39	1491	1419	1391	1391
query40	218	120	110	110
query41	58	56	55	55
query42	124	117	112	112
query43	500	500	478	478
query44	1297	807	811	807
query45	178	175	172	172
query46	829	1041	637	637
query47	1760	1771	1725	1725
query48	392	433	318	318
query49	789	502	409	409
query50	636	697	411	411
query51	4163	4131	4100	4100
query52	112	108	97	97
query53	217	250	190	190
query54	585	555	496	496
query55	87	82	84	82
query56	305	296	289	289
query57	1150	1150	1093	1093
query58	263	256	268	256
query59	2506	2584	2447	2447
query60	323	314	297	297
query61	127	158	131	131
query62	799	727	701	701
query63	228	191	187	187
query64	4311	1023	680	680
query65	4300	4277	4284	4277
query66	1147	418	315	315
query67	16149	15663	15346	15346
query68	8212	889	516	516
query69	463	326	262	262
query70	1208	1157	1075	1075
query71	504	325	307	307
query72	5544	4865	4828	4828
query73	730	662	349	349
query74	8971	9305	8701	8701
query75	3870	3186	2676	2676
query76	3688	1197	805	805
query77	786	382	284	284
query78	10048	10088	9357	9357
query79	2847	813	577	577
query80	630	527	445	445
query81	482	257	227	227
query82	459	126	92	92
query83	280	252	236	236
query84	293	102	87	87
query85	830	355	322	322
query86	337	301	276	276
query87	4461	4392	4379	4379
query88	3140	2224	2287	2224
query89	404	309	277	277
query90	1975	208	206	206
query91	146	143	109	109
query92	76	63	61	61
query93	1931	944	574	574
query94	679	418	306	306
query95	373	291	290	290
query96	488	574	280	280
query97	2752	2781	2642	2642
query98	231	208	207	207
query99	1403	1385	1282	1282
Total cold run time: 275609 ms
Total hot run time: 186023 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a4ff7c9663da3be3ed4170207432359a63031b0f, data reload: false

query1	0.04	0.04	0.03
query2	0.14	0.11	0.11
query3	0.25	0.20	0.19
query4	1.59	0.19	0.11
query5	0.44	0.40	0.43
query6	1.19	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.51
query10	0.56	0.58	0.57
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.61	0.62	0.60
query14	0.78	0.80	0.81
query15	0.88	0.84	0.88
query16	0.39	0.39	0.38
query17	1.02	1.05	1.07
query18	0.22	0.20	0.21
query19	1.86	1.78	1.78
query20	0.02	0.02	0.01
query21	15.41	0.91	0.54
query22	0.76	1.39	0.68
query23	14.75	1.38	0.64
query24	6.81	1.48	0.34
query25	0.31	0.19	0.11
query26	0.55	0.17	0.14
query27	0.06	0.05	0.04
query28	8.99	0.95	0.45
query29	12.57	4.06	3.36
query30	0.25	0.10	0.07
query31	2.85	0.60	0.40
query32	3.22	0.56	0.46
query33	3.02	3.01	3.14
query34	15.73	5.21	4.55
query35	4.52	4.56	4.51
query36	0.67	0.51	0.49
query37	0.08	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 101.86 s
Total hot run time: 28.6 s

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 33.33% (3/9) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.43% (20863/26265)
Line Coverage 72.64% (214786/295694)
Region Coverage 70.83% (126377/178435)
Branch Coverage 64.57% (65486/101418)

luwei16
luwei16 reviewed May 16, 2025
View reviewed changes
Copy link
Contributor

@luwei16 luwei16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use error codes to distinguish failure, no suite version, and delete.

@doris-robot
Copy link

TPC-H: Total hot run time: 34247 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa9659c086396e8c7d96553977aff032de758993, data reload: false

------ Round 1 ----------------------------------
q1	26809	5185	5083	5083
q2	2088	288	194	194
q3	10545	1247	743	743
q4	10243	996	543	543
q5	7825	2341	2364	2341
q6	194	170	132	132
q7	941	766	645	645
q8	9348	1340	1082	1082
q9	6795	5008	5080	5008
q10	6888	2323	1916	1916
q11	497	298	284	284
q12	376	360	224	224
q13	17777	3768	3130	3130
q14	235	243	226	226
q15	549	483	474	474
q16	421	433	392	392
q17	612	920	378	378
q18	7590	7412	7248	7248
q19	1808	974	596	596
q20	347	358	234	234
q21	3872	3331	2402	2402
q22	1013	1007	972	972
Total cold run time: 116773 ms
Total hot run time: 34247 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5225	5113	5085	5085
q2	245	321	227	227
q3	2185	2699	2312	2312
q4	1394	1883	1433	1433
q5	4511	4408	4370	4370
q6	227	178	126	126
q7	2067	1990	1811	1811
q8	2585	2614	2533	2533
q9	7142	7098	7091	7091
q10	3013	3188	2804	2804
q11	577	529	506	506
q12	744	781	629	629
q13	3565	3906	3257	3257
q14	308	318	292	292
q15	553	501	493	493
q16	438	485	461	461
q17	1198	1518	1421	1421
q18	7902	7565	7480	7480
q19	806	862	1133	862
q20	2040	1989	1874	1874
q21	4940	4569	4569	4569
q22	1078	1072	1034	1034
Total cold run time: 52743 ms
Total hot run time: 50670 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194945 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa9659c086396e8c7d96553977aff032de758993, data reload: false

query1	1408	1094	1069	1069
query2	6181	1924	1891	1891
query3	11015	4576	4565	4565
query4	54560	25903	23818	23818
query5	5008	566	463	463
query6	342	215	209	209
query7	4914	529	304	304
query8	321	286	246	246
query9	5731	2695	2702	2695
query10	462	332	277	277
query11	15071	15097	15348	15097
query12	171	121	109	109
query13	1053	544	433	433
query14	10277	6666	6582	6582
query15	206	220	182	182
query16	7091	669	506	506
query17	1116	761	652	652
query18	1600	451	327	327
query19	212	205	187	187
query20	140	134	123	123
query21	211	129	116	116
query22	4420	4440	4216	4216
query23	34539	33654	33577	33577
query24	6704	2468	2485	2468
query25	500	480	436	436
query26	717	278	157	157
query27	2250	533	364	364
query28	3055	2228	2192	2192
query29	608	574	465	465
query30	285	248	192	192
query31	848	852	769	769
query32	72	64	63	63
query33	460	391	328	328
query34	835	890	559	559
query35	806	852	762	762
query36	978	1016	901	901
query37	125	103	82	82
query38	4348	4380	4209	4209
query39	1555	1486	1461	1461
query40	224	135	115	115
query41	60	51	54	51
query42	132	118	116	116
query43	535	544	501	501
query44	1454	906	890	890
query45	186	174	169	169
query46	942	1050	678	678
query47	1825	1860	1784	1784
query48	415	460	352	352
query49	709	530	471	471
query50	712	752	418	418
query51	4239	4308	4219	4219
query52	129	118	113	113
query53	254	294	192	192
query54	603	590	531	531
query55	91	94	91	91
query56	312	337	305	305
query57	1153	1202	1148	1148
query58	284	280	282	280
query59	2789	2877	2844	2844
query60	355	349	336	336
query61	134	123	125	123
query62	794	768	685	685
query63	239	199	195	195
query64	1875	1035	671	671
query65	4306	4226	4297	4226
query66	718	400	303	303
query67	16054	15731	15396	15396
query68	7120	918	538	538
query69	538	319	287	287
query70	1197	1107	1120	1107
query71	512	351	324	324
query72	5962	4822	4950	4822
query73	1462	707	373	373
query74	9245	9122	8682	8682
query75	4019	3224	2683	2683
query76	4284	1240	775	775
query77	665	390	293	293
query78	10249	10225	9436	9436
query79	3254	811	590	590
query80	623	518	459	459
query81	498	268	230	230
query82	468	132	97	97
query83	336	261	233	233
query84	294	105	82	82
query85	790	365	319	319
query86	374	309	307	307
query87	4391	4476	4414	4414
query88	3600	2406	2396	2396
query89	407	325	287	287
query90	1840	217	214	214
query91	138	143	109	109
query92	77	58	60	58
query93	2105	970	594	594
query94	675	417	299	299
query95	374	308	280	280
query96	508	592	296	296
query97	2776	2731	2648	2648
query98	228	214	218	214
query99	1426	1410	1244	1244
Total cold run time: 302153 ms
Total hot run time: 194945 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.65 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa9659c086396e8c7d96553977aff032de758993, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.10
query3	0.26	0.19	0.20
query4	1.60	0.19	0.19
query5	0.46	0.43	0.44
query6	1.16	0.67	0.66
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.60	0.51	0.52
query10	0.59	0.58	0.57
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.62	0.59	0.60
query14	0.78	0.82	0.81
query15	0.89	0.88	0.86
query16	0.38	0.40	0.38
query17	1.01	1.05	1.06
query18	0.22	0.20	0.20
query19	1.88	1.80	1.88
query20	0.02	0.01	0.00
query21	15.39	0.91	0.54
query22	0.75	1.38	0.64
query23	14.76	1.37	0.63
query24	6.72	1.39	0.35
query25	0.35	0.12	0.16
query26	0.76	0.16	0.13
query27	0.05	0.06	0.04
query28	8.86	0.94	0.46
query29	12.54	4.00	3.37
query30	0.26	0.10	0.07
query31	2.81	0.60	0.39
query32	3.23	0.56	0.47
query33	3.09	3.03	3.08
query34	15.69	5.11	4.52
query35	4.48	4.54	4.46
query36	0.66	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 101.9 s
Total hot run time: 28.65 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 26.92% (7/26) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.94% (14903/26642)
Line Coverage 44.69% (132016/295391)
Region Coverage 43.80% (66472/151754)
Branch Coverage 38.40% (34062/88712)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 25.93% (7/27) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.43% (20835/26229)
Line Coverage 72.62% (214508/295402)
Region Coverage 70.75% (126159/178311)
Branch Coverage 64.52% (65390/101344)

gavinchou
gavinchou previously approved these changes May 17, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels May 17, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 19, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 33877 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8148dc77b358e7385983f581b92c5a33de12f693, data reload: false

------ Round 1 ----------------------------------
q1	26362	5049	5202	5049
q2	2103	279	192	192
q3	10570	1250	691	691
q4	10236	1000	527	527
q5	7810	2412	2345	2345
q6	188	167	137	137
q7	901	758	621	621
q8	9313	1280	1058	1058
q9	6814	5082	5172	5082
q10	6830	2300	1891	1891
q11	484	300	268	268
q12	340	365	213	213
q13	17775	3723	3058	3058
q14	225	227	208	208
q15	551	516	512	512
q16	424	438	379	379
q17	586	862	366	366
q18	7620	7226	7152	7152
q19	1693	972	566	566
q20	348	345	230	230
q21	3736	3193	2381	2381
q22	1046	1014	951	951
Total cold run time: 115955 ms
Total hot run time: 33877 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5143	5071	5043	5043
q2	245	327	230	230
q3	2172	2660	2277	2277
q4	1429	1804	1418	1418
q5	4526	4483	4390	4390
q6	213	170	128	128
q7	2005	1910	1789	1789
q8	2585	2583	2574	2574
q9	7046	7024	7109	7024
q10	2997	3178	2712	2712
q11	571	497	494	494
q12	713	765	659	659
q13	3447	3871	3307	3307
q14	279	293	288	288
q15	535	508	494	494
q16	448	479	451	451
q17	1150	1522	1350	1350
q18	7712	7614	7391	7391
q19	805	812	864	812
q20	2010	2033	1896	1896
q21	4863	4442	4268	4268
q22	1054	999	999	999
Total cold run time: 51948 ms
Total hot run time: 49994 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185734 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8148dc77b358e7385983f581b92c5a33de12f693, data reload: false

query1	1010	482	486	482
query2	6555	1919	1860	1860
query3	6764	224	216	216
query4	26443	23825	23462	23462
query5	4323	619	507	507
query6	289	193	192	192
query7	4623	488	282	282
query8	283	243	251	243
query9	8595	2610	2640	2610
query10	481	323	285	285
query11	15816	15051	14760	14760
query12	152	110	105	105
query13	1657	540	423	423
query14	9010	6180	5969	5969
query15	197	184	175	175
query16	7174	618	454	454
query17	1206	720	577	577
query18	1982	402	314	314
query19	198	185	165	165
query20	123	115	121	115
query21	216	127	109	109
query22	4189	4100	4034	4034
query23	34073	33084	33062	33062
query24	8432	2413	2430	2413
query25	540	455	400	400
query26	1226	276	149	149
query27	2741	497	335	335
query28	4317	2109	2083	2083
query29	763	554	433	433
query30	284	220	232	220
query31	898	868	758	758
query32	76	69	59	59
query33	546	367	345	345
query34	824	851	527	527
query35	798	812	776	776
query36	948	982	878	878
query37	110	100	76	76
query38	4203	4095	4032	4032
query39	1501	1399	1501	1399
query40	211	118	103	103
query41	58	52	52	52
query42	124	115	110	110
query43	502	494	484	484
query44	1310	815	817	815
query45	175	175	172	172
query46	839	1025	673	673
query47	1786	1840	1736	1736
query48	386	436	309	309
query49	777	502	439	439
query50	652	676	394	394
query51	4118	4170	4101	4101
query52	108	104	101	101
query53	223	257	179	179
query54	596	571	522	522
query55	88	83	86	83
query56	289	287	326	287
query57	1132	1134	1096	1096
query58	267	257	265	257
query59	2607	2687	2636	2636
query60	323	327	309	309
query61	132	126	128	126
query62	801	737	644	644
query63	234	185	195	185
query64	4347	1037	667	667
query65	4326	4237	4249	4237
query66	1161	406	315	315
query67	15792	15692	15464	15464
query68	8293	886	520	520
query69	475	322	279	279
query70	1236	1160	1124	1124
query71	465	330	295	295
query72	5759	4686	4641	4641
query73	731	592	350	350
query74	9189	9005	8699	8699
query75	3783	3179	2710	2710
query76	3663	1173	735	735
query77	783	399	283	283
query78	10176	10155	9226	9226
query79	2530	806	582	582
query80	620	512	452	452
query81	494	265	222	222
query82	458	125	100	100
query83	289	259	239	239
query84	288	107	85	85
query85	768	352	310	310
query86	387	321	281	281
query87	4493	4525	4323	4323
query88	3651	2273	2271	2271
query89	389	317	286	286
query90	1907	216	211	211
query91	152	136	122	122
query92	78	58	57	57
query93	1865	955	570	570
query94	657	392	305	305
query95	380	284	285	284
query96	487	571	282	282
query97	2769	2720	2631	2631
query98	241	207	205	205
query99	1443	1400	1274	1274
Total cold run time: 275829 ms
Total hot run time: 185734 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.79 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8148dc77b358e7385983f581b92c5a33de12f693, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.25	0.19	0.19
query4	1.59	0.20	0.10
query5	0.43	0.43	0.41
query6	1.17	0.65	0.68
query7	0.03	0.02	0.01
query8	0.04	0.04	0.04
query9	0.57	0.50	0.51
query10	0.56	0.57	0.56
query11	0.16	0.11	0.12
query12	0.16	0.12	0.12
query13	0.62	0.61	0.59
query14	0.78	0.80	0.81
query15	0.87	0.85	0.87
query16	0.38	0.39	0.37
query17	1.03	1.04	1.06
query18	0.23	0.21	0.22
query19	1.93	1.79	1.90
query20	0.01	0.01	0.01
query21	15.40	0.92	0.56
query22	0.74	1.32	0.65
query23	14.82	1.37	0.63
query24	6.71	2.26	0.70
query25	0.47	0.27	0.12
query26	0.54	0.17	0.13
query27	0.05	0.04	0.04
query28	9.78	0.88	0.45
query29	12.55	3.97	3.30
query30	0.26	0.09	0.06
query31	2.81	0.58	0.39
query32	3.23	0.55	0.47
query33	3.10	3.05	3.06
query34	15.93	5.07	4.50
query35	4.49	4.53	4.47
query36	0.66	0.49	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.12
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 103 s
Total hot run time: 28.79 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 26.92% (7/26) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.87% (14902/26671)
Line Coverage 44.64% (132012/295736)
Region Coverage 43.76% (66461/151860)
Branch Coverage 38.36% (34054/88768)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 25.93% (7/27) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.60% (20902/26258)
Line Coverage 72.78% (215241/295747)
Region Coverage 70.92% (126537/178419)
Branch Coverage 64.61% (65516/101404)

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 19, 2025
@gavinchou gavinchou merged commit 1b54ec8 into apache:master May 19, 2025
23 of 25 checks passed
github-actions bot pushed a commit that referenced this pull request May 19, 2025
@Yukang-Lian
[Fix](compaction) Failed compaction tablets should sleep before being…
14bee35 
… selected again (#50672)
@github-actions github-actions bot mentioned this pull request May 19, 2025
Merged
dataroaring pushed a commit that referenced this pull request May 20, 2025
@github-actions @Yukang-Lian
branch-3.0: [Fix](compaction) Failed compaction tablets should sleep …
24c5767 
…before being selected again #50672 (#51030)

Cherry-picked from #50672

Co-authored-by: abmdocrt <lianyukang@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@Yukang-Lian @koarz
[Fix](compaction) Failed compaction tablets should sleep before being…
407391c 
… selected again (apache#50672)
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gavinchou gavinchou gavinchou approved these changes

@luwei16 luwei16 luwei16 approved these changes

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

@w41ter w41ter Awaiting requested review from w41ter w41ter is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.6-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@Yukang-Lian @Thearas @doris-robot @hello-stephen @gavinchou @luwei16 @dataroaring