Skip to content

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented May 8, 2024

BUG

Merge empty rowsets constantly

HOW TO FIX

Merge empty rowset decrease priority

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285
Copy link
Contributor Author

csun5285 commented May 8, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.69% (8983/25171)
Line Coverage: 27.35% (74224/271417)
Region Coverage: 26.59% (38384/144338)
Branch Coverage: 23.41% (19574/83624)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d1130ee5ec02f6307fb378cdd1b37572657dbe7d_d1130ee5ec02f6307fb378cdd1b37572657dbe7d/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41754 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1130ee5ec02f6307fb378cdd1b37572657dbe7d, data reload: false

------ Round 1 ----------------------------------
q1	17607	4305	4204	4204
q2	2006	193	190	190
q3	10476	1196	1195	1195
q4	10212	843	816	816
q5	7478	2737	2692	2692
q6	221	133	130	130
q7	1057	606	604	604
q8	9246	2180	2114	2114
q9	9924	6719	6784	6719
q10	9608	3910	3857	3857
q11	435	240	289	240
q12	437	230	217	217
q13	17235	3149	3218	3149
q14	268	212	218	212
q15	495	480	473	473
q16	505	389	372	372
q17	983	686	730	686
q18	8338	7769	7702	7702
q19	5241	1552	1525	1525
q20	659	338	330	330
q21	5151	4047	4071	4047
q22	361	280	293	280
Total cold run time: 117943 ms
Total hot run time: 41754 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4500	4401	4388	4388
q2	389	257	266	257
q3	3131	2989	2759	2759
q4	1897	1567	1634	1567
q5	5495	5489	5483	5483
q6	218	129	129	129
q7	2337	1947	1959	1947
q8	3226	3370	3438	3370
q9	8575	8706	8612	8612
q10	3870	3816	3839	3816
q11	582	514	499	499
q12	794	636	651	636
q13	15862	3151	3216	3151
q14	293	274	255	255
q15	511	478	482	478
q16	487	432	412	412
q17	1772	1455	1476	1455
q18	7870	7500	7339	7339
q19	2349	1555	1554	1554
q20	2010	1769	1716	1716
q21	14669	4797	4785	4785
q22	593	497	486	486
Total cold run time: 81430 ms
Total hot run time: 55094 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186209 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1130ee5ec02f6307fb378cdd1b37572657dbe7d, data reload: false

query1	902	370	349	349
query2	6423	2452	2331	2331
query3	6641	204	204	204
query4	23458	21248	21468	21248
query5	4092	410	421	410
query6	264	182	185	182
query7	4593	289	291	289
query8	239	194	184	184
query9	8545	2434	2418	2418
query10	422	244	249	244
query11	14709	14086	14130	14086
query12	138	95	96	95
query13	1641	389	368	368
query14	10830	8169	7531	7531
query15	213	175	175	175
query16	7390	268	266	266
query17	1326	554	554	554
query18	1934	287	275	275
query19	207	155	151	151
query20	91	90	87	87
query21	197	127	128	127
query22	5090	4856	4900	4856
query23	33948	33092	33285	33092
query24	6139	2923	2807	2807
query25	490	382	367	367
query26	700	157	160	157
query27	1848	321	318	318
query28	3825	2079	2067	2067
query29	836	609	613	609
query30	237	154	152	152
query31	916	773	718	718
query32	63	54	55	54
query33	491	274	253	253
query34	883	479	480	479
query35	754	655	656	655
query36	1078	914	925	914
query37	103	72	68	68
query38	2824	2723	2739	2723
query39	1606	1513	1526	1513
query40	197	122	131	122
query41	44	41	44	41
query42	105	99	99	99
query43	590	551	559	551
query44	1050	719	737	719
query45	258	259	262	259
query46	1067	704	727	704
query47	1959	1881	1890	1881
query48	393	299	292	292
query49	793	456	388	388
query50	777	377	385	377
query51	6780	6671	6627	6627
query52	96	91	87	87
query53	353	278	278	278
query54	520	434	437	434
query55	73	70	70	70
query56	234	218	227	218
query57	1191	1168	1144	1144
query58	234	201	194	194
query59	3755	3314	3285	3285
query60	262	233	240	233
query61	92	84	84	84
query62	553	454	454	454
query63	303	273	272	272
query64	7527	7523	7192	7192
query65	3090	3047	3053	3047
query66	868	339	338	338
query67	15728	15062	14912	14912
query68	9060	552	565	552
query69	607	303	310	303
query70	1377	1060	1094	1060
query71	499	256	263	256
query72	8314	2522	2341	2341
query73	1604	323	327	323
query74	6525	6292	6181	6181
query75	4639	2628	2585	2585
query76	4860	1037	1009	1009
query77	697	267	260	260
query78	10962	10237	10193	10193
query79	6361	513	511	511
query80	894	451	437	437
query81	512	216	221	216
query82	244	94	98	94
query83	193	162	170	162
query84	264	87	89	87
query85	797	262	264	262
query86	343	329	277	277
query87	3288	3065	3070	3065
query88	4764	2422	2417	2417
query89	530	381	378	378
query90	2186	190	190	190
query91	123	96	96	96
query92	60	51	47	47
query93	5198	523	507	507
query94	1180	182	185	182
query95	390	311	298	298
query96	602	272	266	266
query97	3103	2931	2923	2923
query98	228	214	214	214
query99	1059	839	877	839
Total cold run time: 286441 ms
Total hot run time: 186209 ms

@doris-robot
Copy link

TPC-H: Total hot run time: 41815 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1130ee5ec02f6307fb378cdd1b37572657dbe7d, data reload: false

------ Round 1 ----------------------------------
q1	17617	4450	4230	4230
q2	2019	182	185	182
q3	10553	1208	1222	1208
q4	10228	797	759	759
q5	7490	2730	2715	2715
q6	214	138	133	133
q7	1025	613	568	568
q8	9235	2157	2089	2089
q9	10441	6763	6761	6761
q10	9450	3896	3844	3844
q11	442	250	249	249
q12	452	239	230	230
q13	17454	3178	3228	3178
q14	242	208	210	208
q15	502	464	476	464
q16	507	395	392	392
q17	967	716	666	666
q18	8348	7759	7726	7726
q19	5845	1559	1549	1549
q20	618	318	320	318
q21	5319	4075	4314	4075
q22	361	291	271	271
Total cold run time: 119329 ms
Total hot run time: 41815 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4568	4372	4396	4372
q2	380	275	287	275
q3	3228	3031	2823	2823
q4	1850	1583	1610	1583
q5	5506	5501	5470	5470
q6	215	131	125	125
q7	2323	1947	1996	1947
q8	3207	3403	3423	3403
q9	8672	8708	8612	8612
q10	3949	3887	3847	3847
q11	601	478	480	478
q12	805	641	643	641
q13	16926	3193	3157	3157
q14	280	249	265	249
q15	529	479	462	462
q16	476	410	414	410
q17	1761	1464	1474	1464
q18	7804	7709	7385	7385
q19	1651	1548	1478	1478
q20	1971	1801	1752	1752
q21	8595	4941	4930	4930
q22	575	505	469	469
Total cold run time: 75872 ms
Total hot run time: 55332 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186066 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1130ee5ec02f6307fb378cdd1b37572657dbe7d, data reload: false

query1	916	364	347	347
query2	6431	2430	2375	2375
query3	6638	208	211	208
query4	24034	21307	21114	21114
query5	4117	406	411	406
query6	274	179	170	170
query7	4578	296	286	286
query8	242	195	197	195
query9	8539	2447	2446	2446
query10	450	244	257	244
query11	14752	14097	14236	14097
query12	137	89	88	88
query13	1642	402	382	382
query14	9607	7591	7519	7519
query15	202	175	177	175
query16	7776	265	278	265
query17	1769	574	551	551
query18	1961	288	276	276
query19	197	154	156	154
query20	92	89	91	89
query21	198	133	128	128
query22	4951	4833	4825	4825
query23	33607	33031	33191	33031
query24	6675	2898	2865	2865
query25	520	391	388	388
query26	688	164	162	162
query27	1879	324	334	324
query28	3802	2084	2079	2079
query29	871	619	612	612
query30	232	167	159	159
query31	956	793	726	726
query32	64	55	56	55
query33	490	258	256	256
query34	871	487	486	486
query35	750	716	669	669
query36	1047	924	917	917
query37	108	67	68	67
query38	2858	2730	2724	2724
query39	1585	1535	1527	1527
query40	194	124	125	124
query41	43	41	40	40
query42	103	103	97	97
query43	618	571	561	561
query44	1073	738	751	738
query45	261	258	253	253
query46	1051	712	718	712
query47	1922	1855	1849	1849
query48	452	295	301	295
query49	766	392	406	392
query50	772	385	400	385
query51	6775	6637	6668	6637
query52	109	89	96	89
query53	346	285	278	278
query54	525	435	421	421
query55	72	75	75	75
query56	233	216	221	216
query57	1182	1128	1108	1108
query58	219	210	197	197
query59	3501	3416	3218	3218
query60	259	235	236	235
query61	91	87	92	87
query62	551	455	465	455
query63	307	283	280	280
query64	7613	7225	7230	7225
query65	3070	3058	3043	3043
query66	784	340	367	340
query67	15600	15133	14827	14827
query68	9009	564	563	563
query69	564	309	314	309
query70	1378	1123	1076	1076
query71	497	270	269	269
query72	8068	2540	2348	2348
query73	1586	325	325	325
query74	6539	6088	6105	6088
query75	4829	2607	2606	2606
query76	4821	1026	1035	1026
query77	732	269	269	269
query78	10823	10160	10091	10091
query79	9110	521	530	521
query80	1101	440	440	440
query81	459	223	219	219
query82	780	93	92	92
query83	194	159	165	159
query84	273	84	81	81
query85	868	282	258	258
query86	363	283	303	283
query87	3241	3076	3066	3066
query88	4507	2453	2439	2439
query89	533	381	373	373
query90	1970	188	190	188
query91	133	100	98	98
query92	58	49	49	49
query93	6309	515	506	506
query94	958	186	187	186
query95	395	306	299	299
query96	613	269	272	269
query97	3127	2952	2931	2931
query98	231	219	221	219
query99	1276	856	893	856
Total cold run time: 290343 ms
Total hot run time: 186066 ms

Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 9, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2024

PR approved by anyone and no changes requested.

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang changed the title [fix](compaction) fix time series compaction merge empty rowsets [fix](compaction) fix time series compaction merge empty rowsets priority May 10, 2024
@eldenmoon eldenmoon merged commit 2b68ac6 into apache:master May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.10-merged p0_b reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants