Skip to content

[fix](bloomfilter) Correct bloom_filter_fpp usage in BE#47384

Merged
airborne12 merged 1 commit intoapache:masterfrom
airborne12:fix-fpp
Jan 24, 2025
Merged

[fix](bloomfilter) Correct bloom_filter_fpp usage in BE#47384
airborne12 merged 1 commit intoapache:masterfrom
airborne12:fix-fpp

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Jan 24, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
This commit fixes the issue where bloom_filter_fpp was not actually utilized
in the backend (BE). Ensures that the configured false-positive probability
(bloom_filter_fpp) is properly applied to the bloom filter logic.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

run buildall

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 24, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 32302 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 53f5c9adb2561c81263031c1ee4d682435e40d1d, data reload: false

------ Round 1 ----------------------------------
q1	17573	5473	5362	5362
q2	2065	297	165	165
q3	10442	1260	745	745
q4	10220	969	531	531
q5	7550	2419	2154	2154
q6	190	170	138	138
q7	926	749	610	610
q8	9241	1368	1155	1155
q9	5161	4915	4934	4915
q10	6856	2324	1883	1883
q11	506	281	247	247
q12	344	361	235	235
q13	17757	3641	3092	3092
q14	219	244	219	219
q15	545	478	469	469
q16	639	618	584	584
q17	582	863	324	324
q18	6919	6437	6394	6394
q19	1797	960	531	531
q20	312	316	193	193
q21	3165	2330	2043	2043
q22	369	335	313	313
Total cold run time: 103378 ms
Total hot run time: 32302 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5556	5529	5596	5529
q2	246	330	233	233
q3	2240	2680	2330	2330
q4	1482	1776	1374	1374
q5	4304	4689	4719	4689
q6	169	163	132	132
q7	2047	1938	1783	1783
q8	2620	2790	2812	2790
q9	7323	7229	7346	7229
q10	3034	3314	2816	2816
q11	580	514	493	493
q12	675	721	584	584
q13	3556	3938	3355	3355
q14	287	295	269	269
q15	519	490	478	478
q16	654	697	647	647
q17	1269	1733	1250	1250
q18	7827	7534	7270	7270
q19	829	1167	1122	1122
q20	2011	2048	1955	1955
q21	5834	5309	5029	5029
q22	598	618	539	539
Total cold run time: 53660 ms
Total hot run time: 51896 ms

Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

TPC-DS: Total hot run time: 192818 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 53f5c9adb2561c81263031c1ee4d682435e40d1d, data reload: false

query1	1326	965	915	915
query2	6379	2098	2033	2033
query3	11048	4606	4592	4592
query4	60745	28871	23346	23346
query5	5521	625	475	475
query6	430	207	196	196
query7	5468	510	312	312
query8	334	230	220	220
query9	8174	2607	2584	2584
query10	444	302	276	276
query11	17244	15399	15391	15391
query12	153	117	110	110
query13	1374	564	447	447
query14	11603	7537	7152	7152
query15	209	203	188	188
query16	7221	654	507	507
query17	1107	771	620	620
query18	1870	415	330	330
query19	222	186	171	171
query20	122	117	116	116
query21	211	125	108	108
query22	4514	4747	4368	4368
query23	33945	33443	33284	33284
query24	5676	2386	2414	2386
query25	489	473	406	406
query26	659	288	165	165
query27	1650	493	338	338
query28	3822	2475	2459	2459
query29	588	579	448	448
query30	208	198	156	156
query31	929	863	831	831
query32	65	58	63	58
query33	444	359	323	323
query34	758	858	536	536
query35	842	851	796	796
query36	1025	1078	977	977
query37	123	107	79	79
query38	4325	4346	4255	4255
query39	1500	1452	1445	1445
query40	210	118	103	103
query41	52	49	49	49
query42	121	108	106	106
query43	523	521	496	496
query44	1366	865	834	834
query45	181	185	166	166
query46	921	1077	695	695
query47	1888	1871	1839	1839
query48	413	406	334	334
query49	697	501	410	410
query50	670	678	413	413
query51	4323	4327	4295	4295
query52	109	113	103	103
query53	234	264	188	188
query54	507	518	449	449
query55	85	82	86	82
query56	265	274	285	274
query57	1203	1179	1160	1160
query58	252	233	249	233
query59	3209	3350	3053	3053
query60	272	274	257	257
query61	150	124	119	119
query62	709	704	673	673
query63	234	195	193	193
query64	1279	1060	678	678
query65	3236	3203	3135	3135
query66	716	400	304	304
query67	16128	15763	15406	15406
query68	5029	848	527	527
query69	493	303	265	265
query70	1218	1133	1157	1133
query71	417	301	267	267
query72	6084	3989	3940	3940
query73	808	758	368	368
query74	10172	8820	8806	8806
query75	3211	3182	2682	2682
query76	3780	1190	787	787
query77	483	366	293	293
query78	9943	9805	9294	9294
query79	3517	831	602	602
query80	1708	542	466	466
query81	559	275	236	236
query82	486	152	130	130
query83	277	180	157	157
query84	283	93	70	70
query85	798	440	310	310
query86	462	317	310	310
query87	4537	4461	4277	4277
query88	4608	2205	2188	2188
query89	417	327	290	290
query90	1544	196	194	194
query91	137	143	109	109
query92	71	59	54	54
query93	2918	887	540	540
query94	807	398	292	292
query95	337	272	265	265
query96	488	621	277	277
query97	2858	2889	2779	2779
query98	231	207	198	198
query99	1272	1426	1266	1266
Total cold run time: 312673 ms
Total hot run time: 192818 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 53f5c9adb2561c81263031c1ee4d682435e40d1d, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.40	0.42	0.41
query6	1.14	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.49	0.50
query10	0.55	0.56	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.59	0.58
query14	2.82	2.87	2.72
query15	0.90	0.83	0.83
query16	0.37	0.41	0.39
query17	1.07	0.97	1.06
query18	0.23	0.21	0.21
query19	1.96	1.87	2.04
query20	0.02	0.02	0.01
query21	15.37	0.95	0.57
query22	0.77	0.73	0.64
query23	15.35	1.44	0.51
query24	3.03	1.11	1.26
query25	0.23	0.08	0.06
query26	0.29	0.15	0.15
query27	0.06	0.05	0.04
query28	13.98	1.06	0.45
query29	12.64	4.02	3.31
query30	0.25	0.09	0.06
query31	2.83	0.60	0.38
query32	3.22	0.55	0.46
query33	2.97	3.00	3.07
query34	16.70	5.15	4.52
query35	4.55	4.49	4.58
query36	0.65	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.18	0.13	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.37 s
Total hot run time: 30.7 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.96% (10939/26071)
Line Coverage: 32.20% (92274/286533)
Region Coverage: 31.33% (47267/150880)
Branch Coverage: 27.36% (23908/87374)
Coverage Report: http://coverage.selectdb-in.cc/coverage/53f5c9adb2561c81263031c1ee4d682435e40d1d_53f5c9adb2561c81263031c1ee4d682435e40d1d/report/index.html

Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit fd8673a into apache:master Jan 24, 2025
31 of 33 checks passed
@airborne12 airborne12 deleted the fix-fpp branch January 24, 2025 07:51
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
Problem Summary:
This commit fixes the issue where bloom_filter_fpp was not actually utilized in the backend (BE). 
Ensures that the configured false-positive probability (bloom_filter_fpp) is properly applied to the bloom filter logic.
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Mar 7, 2025
Problem Summary:
This commit fixes the issue where bloom_filter_fpp was not actually utilized in the backend (BE).
Ensures that the configured false-positive probability (bloom_filter_fpp) is properly applied to the bloom filter logic.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
Problem Summary:
This commit fixes the issue where bloom_filter_fpp was not actually utilized in the backend (BE). 
Ensures that the configured false-positive probability (bloom_filter_fpp) is properly applied to the bloom filter logic.
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Jul 7, 2025
Problem Summary:
This commit fixes the issue where bloom_filter_fpp was not actually utilized in the backend (BE).
Ensures that the configured false-positive probability (bloom_filter_fpp) is properly applied to the bloom filter logic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants