Skip to content

Conversation

@yujun777
Copy link
Contributor

@yujun777 yujun777 commented Sep 5, 2025

What problem does this PR solve?

if a sub expression exists multiple time in filter, rule PushDownVirtualColumnsIntoOlapScan will extract it as a virtual slot and push it into olap scan.

but if this sub expression itself contains an unique function which exists multiple times, then it should not push into olap scan, rule AddProjectForUniqueFunction will extract the unique function for it.

But we can't construct a real example SQL satisfy the above case, maybe it's not exists in real SQL, we just still fix it for safe reason. And this PR we don't add a real SQL regression test, we just test it with fe ut.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777
Copy link
Contributor Author

yujun777 commented Sep 5, 2025

run buildall

@yujun777
Copy link
Contributor Author

yujun777 commented Sep 5, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34327 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6b1c278496bf233138f32b67b5cdf3d9089bd75c, data reload: false

------ Round 1 ----------------------------------
q1	17609	5211	5065	5065
q2	1996	348	209	209
q3	10212	1284	719	719
q4	10243	1021	542	542
q5	7573	2501	2394	2394
q6	189	175	139	139
q7	932	767	650	650
q8	9350	1348	1163	1163
q9	6897	5149	5200	5149
q10	6898	2392	1987	1987
q11	500	316	275	275
q12	354	367	246	246
q13	17780	3665	3087	3087
q14	241	237	218	218
q15	579	504	487	487
q16	430	431	389	389
q17	607	881	361	361
q18	7619	7096	7165	7096
q19	1220	969	555	555
q20	352	346	229	229
q21	3843	3216	2394	2394
q22	1074	1033	973	973
Total cold run time: 106498 ms
Total hot run time: 34327 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5159	5089	5115	5089
q2	265	336	239	239
q3	2196	2701	2304	2304
q4	1363	1769	1329	1329
q5	4228	4365	4557	4365
q6	233	179	138	138
q7	2048	2037	1914	1914
q8	2669	2657	2551	2551
q9	7330	7478	7386	7386
q10	3138	3305	2855	2855
q11	574	519	484	484
q12	679	797	620	620
q13	3525	3962	3412	3412
q14	298	318	272	272
q15	524	487	482	482
q16	459	492	446	446
q17	1198	1615	1383	1383
q18	7989	7965	7424	7424
q19	907	906	962	906
q20	1986	2066	1903	1903
q21	4828	4464	4295	4295
q22	1084	1045	999	999
Total cold run time: 52680 ms
Total hot run time: 50796 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186632 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6b1c278496bf233138f32b67b5cdf3d9089bd75c, data reload: false

query1	1069	458	403	403
query2	6552	1752	1710	1710
query3	6759	231	226	226
query4	26448	23581	22791	22791
query5	4449	655	509	509
query6	342	257	238	238
query7	4657	525	303	303
query8	314	262	246	246
query9	8650	2908	2907	2907
query10	496	339	312	312
query11	15938	15105	14737	14737
query12	184	122	124	122
query13	1695	571	435	435
query14	9569	5879	5813	5813
query15	228	196	178	178
query16	7688	675	504	504
query17	1247	775	647	647
query18	2060	455	355	355
query19	209	271	171	171
query20	137	122	119	119
query21	222	137	113	113
query22	4117	4247	4206	4206
query23	34104	32882	32844	32844
query24	8184	2382	2385	2382
query25	552	514	436	436
query26	1249	284	167	167
query27	2663	510	355	355
query28	4375	2295	2260	2260
query29	737	596	485	485
query30	291	255	198	198
query31	910	810	719	719
query32	94	77	78	77
query33	558	383	386	383
query34	793	866	518	518
query35	826	824	775	775
query36	958	1032	923	923
query37	128	113	96	96
query38	4044	4094	4063	4063
query39	1483	1436	1455	1436
query40	227	132	126	126
query41	64	63	59	59
query42	131	117	121	117
query43	514	524	481	481
query44	1356	871	860	860
query45	180	182	172	172
query46	872	1020	663	663
query47	1795	1816	1758	1758
query48	396	432	314	314
query49	759	507	412	412
query50	660	685	403	403
query51	4149	4161	4146	4146
query52	115	120	105	105
query53	260	268	201	201
query54	616	605	567	567
query55	93	92	99	92
query56	341	327	313	313
query57	1191	1209	1131	1131
query58	293	279	279	279
query59	2614	2663	2710	2663
query60	362	359	337	337
query61	189	181	185	181
query62	822	762	664	664
query63	228	195	193	193
query64	4371	1198	848	848
query65	4282	4230	4280	4230
query66	1100	453	357	357
query67	15671	15268	15259	15259
query68	9540	933	586	586
query69	490	331	287	287
query70	1228	1183	1154	1154
query71	594	348	324	324
query72	5553	5025	5128	5025
query73	763	653	373	373
query74	8934	9244	8982	8982
query75	4358	3159	2627	2627
query76	4234	1162	742	742
query77	1018	391	325	325
query78	9644	9666	8819	8819
query79	2799	841	608	608
query80	684	579	526	526
query81	479	259	223	223
query82	267	146	114	114
query83	306	272	247	247
query84	316	112	97	97
query85	835	451	420	420
query86	367	321	318	318
query87	4267	4267	4146	4146
query88	2825	2226	2243	2226
query89	431	331	290	290
query90	2085	232	226	226
query91	168	169	133	133
query92	94	76	72	72
query93	2160	1003	640	640
query94	690	437	324	324
query95	420	340	334	334
query96	496	595	283	283
query97	2644	2753	2552	2552
query98	258	226	222	222
query99	1470	1404	1294	1294
Total cold run time: 279047 ms
Total hot run time: 186632 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6b1c278496bf233138f32b67b5cdf3d9089bd75c, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.45	0.42	0.42
query6	1.18	0.63	0.67
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.60	0.52	0.53
query10	0.59	0.58	0.59
query11	0.16	0.12	0.11
query12	0.15	0.13	0.14
query13	0.63	0.64	0.62
query14	0.80	0.83	0.85
query15	0.86	0.86	0.87
query16	0.38	0.41	0.39
query17	1.04	1.06	1.02
query18	0.22	0.20	0.20
query19	1.92	1.79	1.83
query20	0.02	0.02	0.01
query21	15.39	0.96	0.60
query22	0.78	1.18	0.82
query23	14.77	1.37	0.62
query24	6.95	1.00	0.58
query25	0.56	0.27	0.16
query26	0.51	0.16	0.14
query27	0.06	0.05	0.05
query28	9.85	0.88	0.43
query29	12.66	3.91	3.29
query30	0.29	0.12	0.12
query31	2.82	0.60	0.39
query32	3.24	0.56	0.49
query33	3.01	3.15	3.02
query34	15.90	5.45	4.81
query35	4.94	4.91	4.88
query36	0.68	0.52	0.50
query37	0.11	0.07	0.07
query38	0.07	0.04	0.05
query39	0.04	0.04	0.03
query40	0.18	0.15	0.15
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.09 s
Total hot run time: 29.75 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (5/10) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit 03a5c32 into apache:master Sep 8, 2025
26 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants