Skip to content

[fix](mow) fix potential mem leak for DeleteBitmap::get_agg #52718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zhannngchen merged 1 commit into from
Jul 3, 2025
Merged

[fix](mow) fix potential mem leak for DeleteBitmap::get_agg #52718

zhannngchen merged 1 commit into from
Jul 3, 2025

Conversation

@zhannngchen
Copy link
Contributor

@zhannngchen zhannngchen commented Jul 3, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jul 3, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhannngchen
Copy link
Contributor Author

zhannngchen commented Jul 3, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 3, 2025

TPC-H: Total hot run time: 33929 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc6266e0619c02f65eb75ce7f89c98125266630f, data reload: false

------ Round 1 ----------------------------------
q1	17584	5174	5046	5046
q2	1958	277	182	182
q3	10318	1282	712	712
q4	10232	1025	526	526
q5	8053	2371	2382	2371
q6	182	164	129	129
q7	894	777	596	596
q8	9327	1303	1123	1123
q9	7352	5296	5120	5120
q10	7002	2411	1990	1990
q11	489	283	260	260
q12	368	347	206	206
q13	17794	3719	3109	3109
q14	223	222	212	212
q15	565	475	485	475
q16	419	423	391	391
q17	586	874	360	360
q18	7434	7266	7042	7042
q19	1267	966	536	536
q20	332	347	227	227
q21	3932	2569	2364	2364
q22	1066	1044	952	952
Total cold run time: 107377 ms
Total hot run time: 33929 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5206	5084	5073	5073
q2	251	323	228	228
q3	2137	2685	2285	2285
q4	1388	1788	1350	1350
q5	4222	4524	4607	4524
q6	221	175	130	130
q7	2059	1960	1772	1772
q8	2650	2673	2545	2545
q9	7423	7323	7335	7323
q10	3125	3328	2898	2898
q11	588	513	494	494
q12	692	830	607	607
q13	3617	4092	3265	3265
q14	283	303	303	303
q15	524	486	485	485
q16	485	521	429	429
q17	1154	1510	1398	1398
q18	8213	7946	7712	7712
q19	817	877	957	877
q20	2041	2069	1909	1909
q21	4877	4339	4347	4339
q22	1041	1073	994	994
Total cold run time: 53014 ms
Total hot run time: 50940 ms

@doris-robot
Copy link

doris-robot commented Jul 3, 2025

TPC-DS: Total hot run time: 185027 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc6266e0619c02f65eb75ce7f89c98125266630f, data reload: false

query1	1029	406	400	400
query2	6527	1671	1672	1671
query3	6737	210	209	209
query4	26369	23843	23297	23297
query5	4345	580	434	434
query6	305	212	193	193
query7	4637	498	290	290
query8	273	217	205	205
query9	8609	2644	2664	2644
query10	463	320	266	266
query11	15376	15082	15570	15082
query12	153	107	99	99
query13	1643	508	391	391
query14	8838	5653	5529	5529
query15	192	186	165	165
query16	7174	628	483	483
query17	1177	679	573	573
query18	1969	398	293	293
query19	193	186	149	149
query20	148	115	114	114
query21	212	119	106	106
query22	3980	4167	4003	4003
query23	33884	32938	33116	32938
query24	8467	2434	2339	2339
query25	572	463	384	384
query26	1226	269	152	152
query27	2759	515	341	341
query28	4338	2146	2117	2117
query29	782	556	423	423
query30	286	225	192	192
query31	892	829	736	736
query32	71	65	60	60
query33	559	348	344	344
query34	796	857	524	524
query35	783	825	730	730
query36	936	986	873	873
query37	113	101	83	83
query38	4164	4094	4135	4094
query39	1489	1419	1421	1419
query40	210	116	110	110
query41	61	58	57	57
query42	125	108	108	108
query43	491	505	468	468
query44	1305	820	830	820
query45	181	173	167	167
query46	838	996	623	623
query47	1733	1836	1714	1714
query48	395	421	314	314
query49	760	497	403	403
query50	620	705	408	408
query51	4109	4148	4163	4148
query52	110	110	99	99
query53	227	252	197	197
query54	576	582	515	515
query55	82	80	81	80
query56	305	324	348	324
query57	1161	1182	1111	1111
query58	259	245	262	245
query59	2595	2717	2591	2591
query60	322	320	312	312
query61	126	119	168	119
query62	770	714	642	642
query63	212	188	183	183
query64	4357	1012	668	668
query65	4245	4194	4189	4189
query66	1138	391	322	322
query67	16042	15702	15502	15502
query68	8447	882	520	520
query69	470	308	260	260
query70	1190	1091	1113	1091
query71	451	325	292	292
query72	5536	4692	4698	4692
query73	699	572	350	350
query74	8844	8984	8918	8918
query75	3932	3167	2695	2695
query76	3637	1123	744	744
query77	794	368	281	281
query78	10075	10206	9290	9290
query79	2862	859	614	614
query80	639	517	465	465
query81	478	255	223	223
query82	470	123	94	94
query83	284	260	233	233
query84	287	105	90	90
query85	811	396	320	320
query86	332	289	290	289
query87	4431	4421	4271	4271
query88	3180	2303	2274	2274
query89	413	318	294	294
query90	1927	208	209	208
query91	141	144	108	108
query92	70	61	56	56
query93	1914	956	590	590
query94	684	382	287	287
query95	368	285	286	285
query96	500	583	282	282
query97	2750	2792	2631	2631
query98	241	206	212	206
query99	1483	1408	1290	1290
Total cold run time: 274161 ms
Total hot run time: 185027 ms

@doris-robot
Copy link

doris-robot commented Jul 3, 2025

ClickBench: Total hot run time: 29.48 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc6266e0619c02f65eb75ce7f89c98125266630f, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.24	0.07	0.08
query4	1.62	0.11	0.11
query5	0.43	0.43	0.42
query6	1.16	0.65	0.66
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.61	0.50	0.53
query10	0.57	0.56	0.58
query11	0.16	0.10	0.11
query12	0.14	0.13	0.11
query13	0.63	0.63	0.60
query14	0.78	0.82	0.79
query15	0.90	0.89	0.88
query16	0.38	0.38	0.40
query17	1.11	1.07	1.09
query18	0.22	0.21	0.21
query19	2.00	1.84	1.82
query20	0.02	0.01	0.01
query21	15.40	0.89	0.55
query22	0.75	1.13	1.02
query23	14.67	1.36	0.66
query24	7.00	1.09	0.38
query25	0.49	0.34	0.09
query26	0.65	0.17	0.14
query27	0.06	0.05	0.06
query28	9.26	0.90	0.44
query29	12.54	4.02	3.30
query30	0.25	0.09	0.08
query31	2.83	0.59	0.39
query32	3.24	0.56	0.48
query33	3.06	3.07	3.27
query34	16.13	5.42	4.78
query35	4.82	4.85	4.84
query36	0.71	0.50	0.48
query37	0.10	0.06	0.07
query38	0.05	0.04	0.03
query39	0.02	0.02	0.03
query40	0.18	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.05	0.03	0.03
Total cold run time: 103.54 s
Total hot run time: 29.48 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 3, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 3, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jul 3, 2025

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.14% (15397/26946)
Line Coverage 46.16% (139628/302504)
Region Coverage 45.47% (70762/155621)
Branch Coverage 40.23% (37342/92822)

@zhannngchen zhannngchen merged commit c2a3c5b into apache:master Jul 3, 2025
25 of 28 checks passed
mymeiyi
mymeiyi reviewed Jul 4, 2025
View reviewed changes
be/src/olap/tablet_meta.cpp
<< ", segment=" << std::get<1>(bmk);
start_version += 1;
}
DeleteBitmapAggCache::instance()->release(handle2);
Copy link
Contributor

@mymeiyi mymeiyi Jul 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should check handle2 is not nullptr?

Copy link
Contributor Author

@zhannngchen zhannngchen Jul 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the release function has already handled nullptr.

Copy link
Contributor Author

@zhannngchen zhannngchen Jul 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mymeiyi LRUCache::release handed nullptr, but ShardedLRUCache::release not, 😑

@zhannngchen zhannngchen mentioned this pull request Jul 4, 2025
Merged
16 tasks
@bobhan1 bobhan1 mentioned this pull request Jul 4, 2025
Merged
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
@koarz
[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (apache#5…
3d9ac02 
…2718)
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
@koarz
[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (apache#5…
ef0cfe5 
…2718)
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 4, 2025
@seawinde
[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (apache#5…
2325045 
…2718)
zhannngchen pushed a commit to zhannngchen/incubator-doris that referenced this pull request Jul 7, 2025
[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (apache#5…
be8443d 
…2718)
@zhannngchen zhannngchen mentioned this pull request Jul 7, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jul 7, 2025
branch-3.1: [fix](mow) fix potential mem leak for DeleteBitmap::get_agg
29d5d99 
#52718,#52756 (#52860)

cherry-pick #52718 #52756
etah000 pushed a commit to etah000/doris that referenced this pull request Jul 7, 2025
@etah000
branch-3.1: [fix](mow) fix potential mem leak for DeleteBitmap::get_agg
cc39d59 
apache#52718,apache#52756 (apache#52860)

cherry-pick apache#52718 apache#52756
zhannngchen pushed a commit to zhannngchen/incubator-doris that referenced this pull request Jul 8, 2025
[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (apache#5…
4bb838c 
…2718)
@zhannngchen zhannngchen mentioned this pull request Jul 8, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jul 9, 2025
branch-3.0:[fix](mow) fix potential mem leak for DeleteBitmap::get_agg (
cfa4f67 
#52718, #52756) (#52931)

cherry-pick #52718, #52756
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mymeiyi mymeiyi mymeiyi left review comments

@gavinchou gavinchou gavinchou approved these changes

+1 more reviewer

@bobhan1 bobhan1 bobhan1 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.x-conflict dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@zhannngchen @Thearas @doris-robot @mymeiyi @gavinchou @bobhan1 @dataroaring @morrySnow