Skip to content

[opt](Nereids) add truncate statement and update set statement#39312

Merged
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:add_truncate
Aug 14, 2024
Merged

[opt](Nereids) add truncate statement and update set statement#39312
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:add_truncate

Conversation

@morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Aug 13, 2024 •
edited
Loading

  1. add truncate table syntax
  2. update set syntax, let it compatiable with legacy planner
  3. add set and use statement to not fallback if parse failed list

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

morrySnow commented Aug 13, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 13, 2024
@doris-robot
Copy link

doris-robot commented Aug 13, 2024

TPC-H: Total hot run time: 39849 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 593fa2f1fb62f3881536d9e60cdb6f844ef7af61, data reload: false

------ Round 1 ----------------------------------
q1	17763	4458	4408	4408
q2	3018	190	196	190
q3	11168	1201	1080	1080
q4	10577	742	738	738
q5	7919	2874	2882	2874
q6	236	141	150	141
q7	961	589	602	589
q8	9555	2030	2047	2030
q9	8800	6551	6548	6548
q10	7044	2202	2179	2179
q11	452	248	253	248
q12	394	228	227	227
q13	17773	2993	3024	2993
q14	278	236	231	231
q15	511	489	485	485
q16	512	383	392	383
q17	971	654	672	654
q18	8125	7365	7392	7365
q19	7417	1044	986	986
q20	675	350	340	340
q21	5325	4179	4432	4179
q22	1128	1027	981	981
Total cold run time: 120602 ms
Total hot run time: 39849 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4515	4273	4325	4273
q2	374	273	281	273
q3	2878	2637	2600	2600
q4	1937	1592	1643	1592
q5	5353	5385	5336	5336
q6	219	133	133	133
q7	2041	1644	1639	1639
q8	3184	3354	3298	3298
q9	8420	8334	8337	8334
q10	3411	3142	3160	3142
q11	595	504	500	500
q12	771	642	622	622
q13	16405	2985	3014	2985
q14	296	261	272	261
q15	532	488	484	484
q16	470	421	415	415
q17	1763	1494	1463	1463
q18	7751	7498	7390	7390
q19	1701	1548	1467	1467
q20	2059	1798	1787	1787
q21	5179	5015	4991	4991
q22	1093	1003	1032	1003
Total cold run time: 70947 ms
Total hot run time: 53988 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

TPC-DS: Total hot run time: 185047 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 593fa2f1fb62f3881536d9e60cdb6f844ef7af61, data reload: false

query1	913	377	364	364
query2	6464	1942	1870	1870
query3	6644	209	213	209
query4	34152	23153	23126	23126
query5	4215	485	480	480
query6	268	164	169	164
query7	4589	302	292	292
query8	241	204	214	204
query9	8546	2448	2435	2435
query10	427	260	265	260
query11	15451	14913	15011	14913
query12	155	105	102	102
query13	1652	376	377	376
query14	10172	7548	6972	6972
query15	216	170	164	164
query16	7732	490	501	490
query17	1343	587	555	555
query18	1925	295	294	294
query19	205	148	150	148
query20	117	108	107	107
query21	204	100	97	97
query22	4207	4123	4019	4019
query23	34052	33260	33227	33227
query24	12420	2858	2907	2858
query25	678	395	411	395
query26	1802	159	160	159
query27	2956	271	272	271
query28	7791	2050	2044	2044
query29	1133	439	426	426
query30	313	156	163	156
query31	1035	750	761	750
query32	101	55	58	55
query33	771	289	284	284
query34	981	472	474	472
query35	858	721	720	720
query36	1063	929	941	929
query37	291	85	82	82
query38	3902	3771	3731	3731
query39	1456	1387	1389	1387
query40	282	117	118	117
query41	49	48	46	46
query42	112	101	98	98
query43	502	479	455	455
query44	1218	734	729	729
query45	196	169	169	169
query46	1131	719	739	719
query47	1874	1794	1799	1794
query48	369	291	292	291
query49	1224	439	434	434
query50	808	407	421	407
query51	6795	6659	6682	6659
query52	100	95	90	90
query53	259	185	182	182
query54	1006	463	450	450
query55	80	76	78	76
query56	278	257	269	257
query57	1184	1075	1032	1032
query58	251	241	226	226
query59	2910	2748	2653	2653
query60	307	274	270	270
query61	119	117	226	117
query62	851	617	671	617
query63	216	183	185	183
query64	10596	2232	1748	1748
query65	3243	3155	3143	3143
query66	1351	343	363	343
query67	15636	14862	14874	14862
query68	9005	568	575	568
query69	719	406	284	284
query70	1354	1149	1133	1133
query71	560	273	272	272
query72	8101	2200	2058	2058
query73	2195	329	331	329
query74	9048	8679	8753	8679
query75	5560	2698	2634	2634
query76	5604	1035	1003	1003
query77	769	353	322	322
query78	9730	9180	8880	8880
query79	9240	539	531	531
query80	924	486	499	486
query81	585	228	228	228
query82	293	135	136	135
query83	330	148	143	143
query84	267	78	74	74
query85	984	272	271	271
query86	348	295	295	295
query87	4371	4255	4163	4163
query88	4950	2406	2430	2406
query89	488	297	286	286
query90	2079	198	196	196
query91	121	93	93	93
query92	63	49	52	49
query93	6154	532	536	532
query94	1083	281	298	281
query95	359	261	266	261
query96	607	274	274	274
query97	3219	3027	3005	3005
query98	227	205	203	203
query99	1530	1261	1219	1219
Total cold run time: 325255 ms
Total hot run time: 185047 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

ClickBench: Total hot run time: 30.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 593fa2f1fb62f3881536d9e60cdb6f844ef7af61, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.68	0.07	0.10
query5	0.51	0.50	0.49
query6	1.13	0.75	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.54	0.55	0.53
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.60	0.61	0.59
query14	0.76	0.77	0.78
query15	0.84	0.84	0.81
query16	0.37	0.38	0.38
query17	1.00	1.00	0.96
query18	0.24	0.22	0.22
query19	1.86	1.66	1.68
query20	0.01	0.02	0.01
query21	15.39	0.77	0.66
query22	4.57	7.41	1.54
query23	18.32	1.34	1.26
query24	2.17	0.22	0.21
query25	0.15	0.09	0.08
query26	0.29	0.20	0.21
query27	0.45	0.23	0.23
query28	13.27	1.03	1.00
query29	12.60	3.32	3.29
query30	0.23	0.04	0.04
query31	2.90	0.39	0.39
query32	3.28	0.48	0.48
query33	3.00	2.97	2.93
query34	17.02	4.36	4.29
query35	4.42	4.46	4.38
query36	0.64	0.47	0.48
query37	0.19	0.15	0.17
query38	0.15	0.15	0.16
query39	0.05	0.04	0.04
query40	0.15	0.13	0.12
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 110.26 s
Total hot run time: 30.14 s

@morrySnow
Copy link
Contributor Author

morrySnow commented Aug 14, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 14, 2024

TPC-H: Total hot run time: 39950 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a89140df63ca3340ca1cb1202862ff369b4447e, data reload: false

------ Round 1 ----------------------------------
q1	17883	4424	4383	4383
q2	3114	185	186	185
q3	11116	1158	1085	1085
q4	10626	719	723	719
q5	7793	2801	2803	2801
q6	224	145	141	141
q7	967	597	606	597
q8	9654	1979	2035	1979
q9	8691	6512	6527	6512
q10	7043	2256	2206	2206
q11	465	249	246	246
q12	379	225	216	216
q13	18716	2963	2966	2963
q14	281	237	240	237
q15	519	481	467	467
q16	501	377	397	377
q17	955	652	667	652
q18	8066	7438	7448	7438
q19	5020	1125	1053	1053
q20	636	317	345	317
q21	5314	4585	4378	4378
q22	1145	1043	998	998
Total cold run time: 119108 ms
Total hot run time: 39950 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4432	4222	4217	4217
q2	365	279	270	270
q3	2839	2656	2644	2644
q4	1909	1613	1579	1579
q5	5315	5303	5284	5284
q6	219	130	131	130
q7	2049	1643	1674	1643
q8	3161	3293	3313	3293
q9	8416	8316	8376	8316
q10	3378	3148	3147	3147
q11	608	518	493	493
q12	771	630	614	614
q13	16467	2989	2985	2985
q14	306	266	271	266
q15	523	483	469	469
q16	472	412	407	407
q17	1787	1465	1423	1423
q18	7794	7628	7358	7358
q19	1662	1537	1652	1537
q20	2005	1784	1782	1782
q21	5195	5071	5066	5066
q22	1111	1005	996	996
Total cold run time: 70784 ms
Total hot run time: 53919 ms

@doris-robot
Copy link

doris-robot commented Aug 14, 2024

TPC-DS: Total hot run time: 183955 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a89140df63ca3340ca1cb1202862ff369b4447e, data reload: false

query1	913	376	366	366
query2	6466	1907	1856	1856
query3	6644	209	212	209
query4	27822	23220	23175	23175
query5	4228	492	485	485
query6	280	172	153	153
query7	4578	315	290	290
query8	250	202	194	194
query9	8583	2394	2382	2382
query10	441	270	268	268
query11	15398	14835	14905	14835
query12	148	95	97	95
query13	1632	363	352	352
query14	10317	7722	6844	6844
query15	232	166	174	166
query16	7944	462	422	422
query17	1568	562	554	554
query18	2007	275	275	275
query19	184	141	135	135
query20	111	104	113	104
query21	207	98	97	97
query22	4171	3932	3945	3932
query23	33797	33278	33321	33278
query24	11587	2822	2838	2822
query25	621	365	407	365
query26	1309	158	153	153
query27	2801	268	269	268
query28	7613	2007	1988	1988
query29	907	459	399	399
query30	291	143	144	143
query31	963	742	719	719
query32	95	53	54	53
query33	752	278	263	263
query34	998	466	464	464
query35	830	710	718	710
query36	1055	904	904	904
query37	150	81	79	79
query38	3973	3964	3781	3781
query39	1440	1361	1372	1361
query40	269	114	111	111
query41	45	47	45	45
query42	115	100	96	96
query43	503	471	461	461
query44	1182	721	720	720
query45	192	165	163	163
query46	1083	712	739	712
query47	1862	1761	1767	1761
query48	362	297	294	294
query49	1096	409	437	409
query50	785	405	396	396
query51	6748	6815	6629	6629
query52	103	94	91	91
query53	251	186	179	179
query54	1015	431	431	431
query55	75	76	74	74
query56	263	255	246	246
query57	1166	1059	1066	1059
query58	261	215	225	215
query59	3021	2643	2732	2643
query60	301	256	262	256
query61	97	98	96	96
query62	817	625	645	625
query63	216	178	184	178
query64	10824	2223	1774	1774
query65	3258	3135	3155	3135
query66	1366	342	342	342
query67	15175	14754	14802	14754
query68	4595	542	545	542
query69	414	270	271	270
query70	1144	1119	1143	1119
query71	402	271	270	270
query72	7124	2279	2026	2026
query73	744	334	326	326
query74	9358	8948	8711	8711
query75	3435	2682	2716	2682
query76	2894	935	977	935
query77	487	302	295	295
query78	9469	9097	8887	8887
query79	2476	525	518	518
query80	1890	480	481	480
query81	566	215	216	215
query82	910	135	134	134
query83	248	148	144	144
query84	278	77	78	77
query85	2123	274	261	261
query86	520	276	298	276
query87	4408	4145	4249	4145
query88	3765	2383	2405	2383
query89	383	283	281	281
query90	1873	196	188	188
query91	116	97	95	95
query92	66	48	52	48
query93	2583	533	521	521
query94	950	286	299	286
query95	352	255	255	255
query96	587	279	269	269
query97	3243	3045	3055	3045
query98	219	195	196	195
query99	1632	1245	1242	1242
Total cold run time: 295011 ms
Total hot run time: 183955 ms

@doris-robot
Copy link

doris-robot commented Aug 14, 2024

ClickBench: Total hot run time: 30.33 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a89140df63ca3340ca1cb1202862ff369b4447e, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.48	0.47	0.47
query6	1.12	0.72	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.55	0.54	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.60	0.59
query14	0.75	0.81	0.77
query15	0.85	0.82	0.80
query16	0.37	0.37	0.37
query17	0.96	0.98	1.03
query18	0.22	0.23	0.21
query19	1.82	1.83	1.66
query20	0.01	0.01	0.01
query21	15.44	0.75	0.65
query22	4.31	7.31	1.66
query23	18.23	1.36	1.28
query24	2.09	0.22	0.22
query25	0.14	0.08	0.08
query26	0.31	0.21	0.21
query27	0.45	0.22	0.21
query28	13.32	1.02	1.00
query29	12.65	3.31	3.32
query30	0.23	0.06	0.05
query31	2.91	0.39	0.38
query32	3.28	0.49	0.46
query33	2.93	3.01	2.95
query34	16.90	4.38	4.38
query35	4.41	4.40	4.43
query36	0.65	0.46	0.48
query37	0.18	0.15	0.15
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.16	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.67 s
Total hot run time: 30.33 s

@morrySnow
Copy link
Contributor Author

morrySnow commented Aug 14, 2024

run p0

@morrySnow morrySnow changed the title [opt](Nereids) add truncate syntax [opt](Nereids) add truncate statement and update set statement Aug 14, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 14, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 1c1c4e8 into apache:master Aug 14, 2024
@morrySnow morrySnow deleted the add_truncate branch August 14, 2024 08:43
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@morrySnow @dataroaring
[opt](Nereids) add truncate statement and update set statement (#39312)
da074c7 
1. add truncate table syntax
2. update set syntax, let it compatiable with legacy planner
3. add set and use statement to not fallback if parse failed list
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@924060929 924060929 924060929 approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.2-merged doing reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morrySnow @doris-robot @924060929 @starocean999 @dataroaring