Skip to content

branch-3.0: [bugfix](nerids) complete the implementation of the concat method. #51141#51223

Merged
dataroaring merged 1 commit intobranch-3.0from
auto-pick-51141-branch-3.0
May 27, 2025
Merged

branch-3.0: [bugfix](nerids) complete the implementation of the concat method. #51141#51223
dataroaring merged 1 commit intobranch-3.0from
auto-pick-51141-branch-3.0

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented May 26, 2025

Cherry-picked from #51141

@XLPE
[bugfix](nerids) complete the implementation of the concat method. (#…
a2526b9 
…51141)

Problem Summary:
In Doris versions above 2.1.8 and above 3.0.3, , the newly added concat
function can't fold when more than two parameters.
before:
```
mysql> EXPLAIN SELECT concat('a', 'a', 'b', 'c', 'd', 'efddd');
+---------------------------------------------------+
| Explain String(Nereids Planner)                   |
+---------------------------------------------------+
| PLAN FRAGMENT 0                                   |
|   OUTPUT EXPRS:                                   |
|     concat('a', 'a', 'b', 'c', 'd', 'efddd')[#0]  |
|   PARTITION: UNPARTITIONED                        |
|                                                   |
|   HAS_COLO_PLAN_NODE: false                       |
|                                                   |
|   VRESULT SINK                                    |
|      MYSQL_PROTOCAL                               |
|                                                   |
|   0:VUNION(30)                                    |
|      constant exprs:                              |
|          concat('a', 'a', 'b', 'c', 'd', 'efddd') |
+---------------------------------------------------+
```
after:
```
mysql> EXPLAIN SELECT concat('a', 'a', 'b', 'c', 'd', 'efddd');
+----------------------------------+
| Explain String(Nereids Planner)  |
+----------------------------------+
| PLAN FRAGMENT 0                  |
|   OUTPUT EXPRS:                  |
|     'aabcdefddd'[#0]             |
|   PARTITION: UNPARTITIONED       |
|                                  |
|   HAS_COLO_PLAN_NODE: false      |
|                                  |
|   VRESULT SINK                   |
|      MYSQL_PROTOCAL              |
|                                  |
|   0:VUNION(30)                   |
|      constant exprs:             |
|          'aabcdefddd'            |
+----------------------------------+
```


Co-authored-by: weiwh1 <weiwh1@chinatelecom.cn>
@github-actions github-actions bot requested a review from dataroaring as a code owner May 26, 2025 02:03
@hello-stephen
Copy link
Contributor

hello-stephen commented May 26, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this May 26, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented May 26, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 26, 2025

TPC-H: Total hot run time: 39631 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2526b912ff8ad1e2b639bd0dfcc510e33b34bf4, data reload: false

------ Round 1 ----------------------------------
q1	17579	6706	6629	6629
q2	2054	162	160	160
q3	10649	1052	1120	1052
q4	10566	744	719	719
q5	7767	2835	2794	2794
q6	217	134	133	133
q7	968	626	614	614
q8	9376	1991	2005	1991
q9	6566	6358	6345	6345
q10	7039	2278	2258	2258
q11	449	256	256	256
q12	404	212	210	210
q13	17809	2978	2976	2976
q14	241	201	206	201
q15	503	461	462	461
q16	513	390	379	379
q17	956	595	609	595
q18	7189	6629	6619	6619
q19	1436	1107	995	995
q20	462	199	198	198
q21	3949	3077	3287	3077
q22	1079	1000	969	969
Total cold run time: 107771 ms
Total hot run time: 39631 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6571	6533	6528	6528
q2	321	235	234	234
q3	2862	2732	2789	2732
q4	2049	1779	1782	1779
q5	5780	5712	5775	5712
q6	214	125	128	125
q7	2230	1793	1808	1793
q8	3367	3513	3510	3510
q9	8994	8831	8922	8831
q10	3550	3491	3493	3491
q11	591	485	487	485
q12	845	645	613	613
q13	8999	3116	3177	3116
q14	288	268	261	261
q15	505	470	466	466
q16	490	447	436	436
q17	1839	1605	1604	1604
q18	8206	7790	7724	7724
q19	1686	1608	1633	1608
q20	2047	1797	1820	1797
q21	5513	5322	5457	5322
q22	1102	1088	1036	1036
Total cold run time: 68049 ms
Total hot run time: 59203 ms

@doris-robot
Copy link

doris-robot commented May 26, 2025

TPC-DS: Total hot run time: 195610 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2526b912ff8ad1e2b639bd0dfcc510e33b34bf4, data reload: false

query1	1286	896	877	877
query2	6329	1910	1903	1903
query3	10811	4270	4178	4178
query4	61048	34827	23813	23813
query5	5233	442	430	430
query6	314	181	161	161
query7	4907	312	319	312
query8	279	222	216	216
query9	6647	2580	2573	2573
query10	430	266	254	254
query11	16144	15204	15723	15204
query12	160	105	106	105
query13	1059	448	421	421
query14	10464	7332	6523	6523
query15	218	179	178	178
query16	7052	493	494	493
query17	1183	591	583	583
query18	1817	332	319	319
query19	209	174	163	163
query20	119	113	109	109
query21	205	105	103	103
query22	4579	4373	4543	4373
query23	35168	34315	34024	34024
query24	6142	2871	2896	2871
query25	544	447	424	424
query26	658	177	165	165
query27	2081	397	363	363
query28	4171	2172	2118	2118
query29	705	473	447	447
query30	236	160	148	148
query31	977	830	844	830
query32	73	52	55	52
query33	416	296	272	272
query34	912	497	503	497
query35	865	725	718	718
query36	1048	930	930	930
query37	106	71	68	68
query38	4027	3978	3937	3937
query39	1508	1483	1467	1467
query40	218	98	96	96
query41	48	46	49	46
query42	113	99	103	99
query43	535	472	476	472
query44	1148	791	813	791
query45	182	164	168	164
query46	1132	692	706	692
query47	2070	1939	1901	1901
query48	481	388	404	388
query49	721	405	397	397
query50	880	420	425	420
query51	7405	7219	7277	7219
query52	98	90	85	85
query53	252	186	181	181
query54	563	453	474	453
query55	73	80	76	76
query56	259	238	252	238
query57	1253	1129	1165	1129
query58	221	210	235	210
query59	3190	2997	3089	2997
query60	285	251	243	243
query61	111	110	116	110
query62	782	679	680	679
query63	228	192	191	191
query64	1399	705	662	662
query65	3354	3201	3164	3164
query66	679	304	330	304
query67	16027	15737	15484	15484
query68	3782	581	556	556
query69	424	271	262	262
query70	1190	1045	1074	1045
query71	365	262	246	246
query72	6082	4026	4025	4025
query73	753	348	346	346
query74	10490	8947	9145	8947
query75	3352	2633	2635	2633
query76	1731	1000	1081	1000
query77	498	272	260	260
query78	10653	9644	9431	9431
query79	1729	590	578	578
query80	858	436	430	430
query81	504	218	218	218
query82	1247	87	85	85
query83	236	147	142	142
query84	276	82	81	81
query85	891	304	297	297
query86	316	279	302	279
query87	4384	4257	4245	4245
query88	3905	2417	2370	2370
query89	406	287	290	287
query90	1987	183	181	181
query91	184	155	167	155
query92	62	47	52	47
query93	2160	539	543	539
query94	728	295	258	258
query95	355	261	249	249
query96	617	286	286	286
query97	3320	3100	3121	3100
query98	216	198	207	198
query99	1555	1321	1332	1321
Total cold run time: 309093 ms
Total hot run time: 195610 ms

@doris-robot
Copy link

doris-robot commented May 26, 2025

ClickBench: Total hot run time: 30.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2526b912ff8ad1e2b639bd0dfcc510e33b34bf4, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.54	0.49	0.52
query6	1.13	0.73	0.74
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.55	0.50	0.50
query10	0.56	0.54	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.59
query14	0.76	0.78	0.79
query15	0.83	0.82	0.82
query16	0.38	0.38	0.41
query17	1.06	1.06	1.05
query18	0.23	0.21	0.22
query19	1.97	1.89	1.75
query20	0.01	0.01	0.01
query21	15.43	0.59	0.56
query22	2.67	1.74	2.48
query23	16.94	1.01	1.00
query24	3.02	1.71	1.65
query25	0.20	0.17	0.17
query26	0.46	0.14	0.14
query27	0.04	0.03	0.06
query28	9.25	0.50	0.47
query29	12.61	3.24	3.22
query30	0.25	0.06	0.06
query31	2.85	0.39	0.38
query32	3.28	0.47	0.45
query33	2.97	2.97	3.01
query34	17.22	4.47	4.52
query35	4.49	4.56	4.53
query36	0.64	0.47	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.74 s
Total hot run time: 30.95 s

@XLPE
Copy link
Contributor

XLPE commented May 26, 2025

run p0

dataroaring
dataroaring approved these changes May 27, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit ea905d0 into branch-3.0 May 27, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-51141-branch-3.0 branch May 27, 2025 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @XLPE @dataroaring