Skip to content

[Fix](Planner) add date/month/year add function fe constant fold with dateV2/dateTimeV2#34484

Merged
morrySnow merged 5 commits intoapache:branch-2.0from
LiBinfeng-01:fix_planner_2.0_date_add
May 10, 2024
Merged

[Fix](Planner) add date/month/year add function fe constant fold with dateV2/dateTimeV2#34484
morrySnow merged 5 commits intoapache:branch-2.0from
LiBinfeng-01:fix_planner_2.0_date_add

Conversation

@LiBinfeng-01
Copy link
Contributor

Proposed changes

cherry-pick: #34483

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@FEFunction(name = "months_add", argTypes = { "DATEV2", "INT" }, returnType = "DATEV2")
public static DateLiteral monthsAddDateV2(LiteralExpr date, LiteralExpr month) throws AnalysisException {
DateLiteral dateLiteral = (DateLiteral) date;
return dateLiteral.plusYears(month.getLongValue());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return dateLiteral.plusYears(month.getLongValue());
return dateLiteral.plusMonths(month.getLongValue());

@morrySnow
Copy link
Contributor

run buildall

Copy link
Contributor

@gavinchou gavinchou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LiBinfeng-01
Copy link
Contributor Author

run buildall

@LiBinfeng-01
Copy link
Contributor Author

run buildall

@LiBinfeng-01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49364 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20e9e6fdb185a71294229d7d3293d32f71a30756, data reload: false

------ Round 1 ----------------------------------
q1	17794	4371	4329	4329
q2	2028	150	141	141
q3	10378	1870	1924	1870
q4	10350	1218	1299	1218
q5	8608	3864	3911	3864
q6	236	122	122	122
q7	2065	1590	1605	1590
q8	9294	2673	2704	2673
q9	10619	10231	10297	10231
q10	8624	3517	3513	3513
q11	434	242	239	239
q12	475	296	307	296
q13	18352	3954	3999	3954
q14	352	319	318	318
q15	507	452	465	452
q16	664	587	569	569
q17	1131	970	935	935
q18	7371	6788	6783	6783
q19	1675	1560	1526	1526
q20	509	312	304	304
q21	4482	4115	4045	4045
q22	482	392	397	392
Total cold run time: 116430 ms
Total hot run time: 49364 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4284	4268	4317	4268
q2	319	226	224	224
q3	4148	4132	4129	4129
q4	2751	2729	2722	2722
q5	7150	7048	7098	7048
q6	232	119	119	119
q7	3252	2854	2790	2790
q8	4373	4457	4401	4401
q9	16818	16714	16856	16714
q10	4350	4369	4371	4369
q11	762	685	681	681
q12	1058	896	887	887
q13	7642	3767	3758	3758
q14	450	427	418	418
q15	485	464	450	450
q16	739	692	691	691
q17	3882	3864	3897	3864
q18	8836	8846	8677	8677
q19	1696	1695	1658	1658
q20	2358	2127	2144	2127
q21	8402	8447	8455	8447
q22	1035	935	914	914
Total cold run time: 85022 ms
Total hot run time: 79356 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202552 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20e9e6fdb185a71294229d7d3293d32f71a30756, data reload: false

query1	919	395	377	377
query2	6540	2932	2801	2801
query3	6913	198	201	198
query4	21014	18032	18008	18008
query5	19710	6516	6462	6462
query6	281	219	212	212
query7	4154	307	298	298
query8	255	242	234	234
query9	3096	2685	2596	2596
query10	412	285	295	285
query11	11312	10615	10648	10615
query12	121	77	69	69
query13	5576	703	690	690
query14	18096	13279	13286	13279
query15	374	221	222	221
query16	6427	272	252	252
query17	1724	1446	881	881
query18	2304	406	413	406
query19	212	151	148	148
query20	81	74	75	74
query21	191	87	93	87
query22	5229	5064	5134	5064
query23	32457	31810	32111	31810
query24	6924	6485	6518	6485
query25	520	432	413	413
query26	457	165	159	159
query27	1662	288	285	285
query28	6260	2366	2328	2328
query29	2903	2822	2797	2797
query30	237	161	161	161
query31	885	727	729	727
query32	64	61	55	55
query33	404	252	247	247
query34	843	454	467	454
query35	1110	908	900	900
query36	1324	1207	1166	1166
query37	92	64	61	61
query38	3067	2922	2909	2909
query39	1352	1318	1336	1318
query40	203	99	99	99
query41	37	34	34	34
query42	82	85	81	81
query43	799	670	690	670
query44	1112	735	737	735
query45	243	226	229	226
query46	1220	977	947	947
query47	1926	1670	1768	1670
query48	994	738	713	713
query49	625	357	383	357
query50	877	598	642	598
query51	4777	4683	4640	4640
query52	82	69	76	69
query53	448	312	322	312
query54	2627	2472	2441	2441
query55	83	78	80	78
query56	224	186	195	186
query57	1176	1137	1129	1129
query58	218	208	195	195
query59	4581	4003	3790	3790
query60	199	187	206	187
query61	87	81	81	81
query62	855	475	415	415
query63	470	340	334	334
query64	2176	1464	1428	1428
query65	3654	3547	3561	3547
query66	769	361	365	361
query67	15869	15483	15217	15217
query68	10457	671	660	660
query69	568	348	358	348
query70	1964	1346	1433	1346
query71	412	312	310	310
query72	6498	3544	3418	3418
query73	739	325	318	318
query74	6314	5792	5879	5792
query75	5541	3701	3743	3701
query76	6758	1185	1194	1185
query77	1184	261	248	248
query78	12633	11745	11567	11567
query79	7495	630	657	630
query80	1100	393	395	393
query81	484	233	226	226
query82	1249	97	102	97
query83	163	140	130	130
query84	257	71	68	68
query85	793	302	294	294
query86	329	297	289	289
query87	3259	3077	3009	3009
query88	4492	2305	2315	2305
query89	367	297	288	288
query90	1976	197	212	197
query91	168	134	131	131
query92	59	57	53	53
query93	6490	573	573	573
query94	709	208	207	207
query95	1109	1061	1066	1061
query96	645	335	326	326
query97	6432	6265	6343	6265
query98	190	182	176	176
query99	3030	869	951	869
Total cold run time: 315779 ms
Total hot run time: 202552 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20e9e6fdb185a71294229d7d3293d32f71a30756, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.25	0.05	0.05
query4	1.79	0.09	0.07
query5	0.54	0.52	0.53
query6	1.24	0.62	0.62
query7	0.02	0.01	0.01
query8	0.02	0.02	0.02
query9	0.53	0.47	0.48
query10	0.54	0.54	0.53
query11	0.12	0.09	0.08
query12	0.11	0.09	0.09
query13	0.61	0.61	0.62
query14	0.79	0.77	0.80
query15	0.80	0.75	0.78
query16	0.37	0.37	0.36
query17	1.04	1.03	1.00
query18	0.21	0.28	0.23
query19	1.92	1.79	1.87
query20	0.02	0.01	0.01
query21	15.48	0.55	0.52
query22	2.22	2.37	2.04
query23	17.19	0.95	0.83
query24	8.44	0.85	0.73
query25	0.41	0.11	0.05
query26	0.86	0.16	0.16
query27	0.05	0.03	0.04
query28	5.19	0.76	0.79
query29	12.63	2.35	2.21
query30	0.55	0.53	0.53
query31	2.82	0.40	0.37
query32	3.39	0.50	0.50
query33	3.07	3.08	3.09
query34	15.28	4.81	4.83
query35	4.87	4.81	4.85
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 104.88 s
Total hot run time: 30.71 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 20e9e6fdb185a71294229d7d3293d32f71a30756 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels May 9, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2024

PR approved by anyone and no changes requested.

@LiBinfeng-01
Copy link
Contributor Author

run feut

@LiBinfeng-01 LiBinfeng-01 requested a review from gavinchou May 9, 2024 06:25
@morrySnow morrySnow merged commit 9ccacb4 into apache:branch-2.0 May 10, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
… dateV2/dateTimeV2 (apache#34484)

cherry-pick: apache#34483

Problem and Cause:
In original planner, date_add function would choose different data type of datetime or datev2. Which when original planner choose datev2 as constant date type. Fe could not folding date_add function because missing of function signature of folding constant date_add(datev2, int)

Solved:
Add corresponding function signatures of date_add/months_add/years_add in original planner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. kind/test reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants