Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix] (inverted index) disable index compaction #41427

Merged
merged 1 commit into from
Oct 8, 2024
Merged

[fix] (inverted index) disable index compaction #41427

merged 1 commit into from
Oct 8, 2024

Conversation

csun5285
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285
Copy link
Contributor Author

run buildall

@csun5285 csun5285 mentioned this pull request Sep 27, 2024
Merged
airborne12
airborne12 approved these changes Sep 27, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

airborne12 pushed a commit that referenced this pull request Sep 27, 2024
@csun5285
[fix] (inverted index) disable index compaction (#41427) (#41430)
90f14f2 
## Proposed changes

pick from master #41427

<!--Describe your changes.-->
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.29% (9625/25811)
Line Coverage: 28.69% (79654/277598)
Region Coverage: 28.12% (41185/146444)
Branch Coverage: 24.74% (20971/84758)
Coverage Report: http://coverage.selectdb-in.cc/coverage/dcf51db527eeca4f292cf54aa2ef1500b3092183_dcf51db527eeca4f292cf54aa2ef1500b3092183/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40579 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dcf51db527eeca4f292cf54aa2ef1500b3092183, data reload: false

------ Round 1 ----------------------------------
q1	17569	7357	7267	7267
q2	2033	277	282	277
q3	12119	1053	1162	1053
q4	10568	688	688	688
q5	7738	2934	2800	2800
q6	237	152	149	149
q7	972	628	605	605
q8	9352	1898	1964	1898
q9	6601	6428	6391	6391
q10	6953	2267	2287	2267
q11	431	241	243	241
q12	407	215	217	215
q13	17794	2974	2980	2974
q14	241	205	217	205
q15	562	514	506	506
q16	645	576	584	576
q17	969	535	531	531
q18	7260	6678	6637	6637
q19	1404	1086	1040	1040
q20	475	198	199	198
q21	4295	3115	3096	3096
q22	1109	965	999	965
Total cold run time: 109734 ms
Total hot run time: 40579 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7206	7208	7185	7185
q2	336	222	222	222
q3	2937	2860	2959	2860
q4	2023	1895	1826	1826
q5	5769	5713	5754	5713
q6	230	146	147	146
q7	2195	1809	1815	1809
q8	3329	3553	3437	3437
q9	8942	8896	8855	8855
q10	3563	3536	3492	3492
q11	568	486	493	486
q12	797	617	647	617
q13	9930	3149	3106	3106
q14	301	280	270	270
q15	568	534	563	534
q16	680	640	637	637
q17	1851	1615	1607	1607
q18	8109	7721	7539	7539
q19	1674	1587	1463	1463
q20	2119	1904	1908	1904
q21	5544	5384	5354	5354
q22	1106	1092	1023	1023
Total cold run time: 69777 ms
Total hot run time: 60085 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191033 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dcf51db527eeca4f292cf54aa2ef1500b3092183, data reload: false

query1	917	391	394	391
query2	6236	2129	2083	2083
query3	8678	194	203	194
query4	33737	23496	23392	23392
query5	3498	462	455	455
query6	267	172	161	161
query7	4196	297	311	297
query8	275	222	212	212
query9	9401	2617	2634	2617
query10	457	275	281	275
query11	17674	15370	15171	15171
query12	154	98	94	94
query13	1526	419	417	417
query14	9378	6555	7321	6555
query15	247	172	179	172
query16	7891	455	438	438
query17	1627	577	617	577
query18	1945	322	323	322
query19	260	161	162	161
query20	124	114	112	112
query21	214	109	106	106
query22	4618	4699	4555	4555
query23	34932	34205	34176	34176
query24	11050	2861	2816	2816
query25	636	409	397	397
query26	1161	164	166	164
query27	2325	297	299	297
query28	7400	2408	2368	2368
query29	839	442	436	436
query30	257	153	153	153
query31	1030	785	790	785
query32	100	51	53	51
query33	764	297	295	295
query34	938	518	507	507
query35	858	742	729	729
query36	1096	920	971	920
query37	156	106	94	94
query38	4044	3872	3943	3872
query39	1464	1417	1419	1417
query40	212	96	98	96
query41	50	46	46	46
query42	118	94	94	94
query43	538	506	474	474
query44	1253	790	784	784
query45	196	162	168	162
query46	1141	720	688	688
query47	1933	1875	1839	1839
query48	459	373	369	369
query49	941	402	410	402
query50	824	413	414	413
query51	6999	7120	6946	6946
query52	97	85	87	85
query53	257	178	184	178
query54	1222	457	467	457
query55	81	80	79	79
query56	270	250	272	250
query57	1217	1102	1074	1074
query58	251	231	238	231
query59	3121	3193	3082	3082
query60	306	272	270	270
query61	127	124	122	122
query62	889	684	683	683
query63	227	187	189	187
query64	4129	640	605	605
query65	3236	3193	3163	3163
query66	837	300	300	300
query67	15886	15552	15509	15509
query68	4711	570	568	568
query69	557	298	296	296
query70	1102	1111	1089	1089
query71	378	261	269	261
query72	7385	4154	4016	4016
query73	767	342	344	342
query74	10170	8959	8946	8946
query75	3447	2734	2652	2652
query76	3352	945	963	945
query77	401	285	284	284
query78	10584	9526	9551	9526
query79	2960	582	602	582
query80	1227	440	452	440
query81	597	242	233	233
query82	670	136	138	136
query83	218	135	133	133
query84	246	81	79	79
query85	1640	296	291	291
query86	436	263	295	263
query87	4437	4259	4389	4259
query88	3682	2364	2303	2303
query89	417	286	274	274
query90	2044	183	182	182
query91	175	138	142	138
query92	62	46	48	46
query93	2447	544	552	544
query94	1059	291	287	287
query95	354	249	254	249
query96	632	285	273	273
query97	3262	3148	3088	3088
query98	215	214	191	191
query99	1532	1309	1295	1295
Total cold run time: 300353 ms
Total hot run time: 191033 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dcf51db527eeca4f292cf54aa2ef1500b3092183, data reload: false

query1	0.04	0.05	0.04
query2	0.06	0.03	0.03
query3	0.24	0.06	0.07
query4	1.65	0.10	0.10
query5	0.58	0.52	0.52
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.49
query10	0.55	0.57	0.54
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.82	2.68	2.83
query15	0.89	0.83	0.82
query16	0.39	0.39	0.38
query17	1.01	1.06	0.97
query18	0.20	0.19	0.19
query19	1.95	1.72	2.01
query20	0.01	0.01	0.01
query21	15.36	0.59	0.58
query22	3.01	2.76	1.74
query23	16.95	0.86	0.87
query24	2.56	0.74	0.87
query25	0.16	0.13	0.06
query26	0.40	0.13	0.14
query27	0.04	0.04	0.04
query28	11.41	1.08	1.06
query29	12.54	3.35	3.31
query30	0.24	0.06	0.05
query31	2.89	0.37	0.38
query32	3.28	0.47	0.47
query33	2.98	3.02	3.05
query34	16.83	4.49	4.50
query35	4.50	4.54	4.48
query36	0.67	0.48	0.50
query37	0.08	0.05	0.06
query38	0.06	0.03	0.04
query39	0.03	0.01	0.03
query40	0.14	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.03	0.02	0.03
Total cold run time: 107.29 s
Total hot run time: 32.28 s

@qidaye qidaye merged commit 48cbc35 into apache:master Oct 8, 2024
23 of 29 checks passed
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 10, 2024
@csun5285 @eldenmoon
[fix] (inverted index) disable index compaction (apache#41427)
3c0ecc3
cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024
@csun5285 @cjj2010
[fix] (inverted index) disable index compaction (apache#41427)
5dccd88
amorynan pushed a commit to amorynan/doris that referenced this pull request Oct 12, 2024
@csun5285 @amorynan
[fix] (inverted index) disable index compaction (apache#41427)
78fc5f2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qidaye qidaye qidaye approved these changes

@airborne12 airborne12 airborne12 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@csun5285 @doris-robot @airborne12 @qidaye @dataroaring