Skip to content

[Fix](nereids) report an error when a function that cannot be used for window analysis appears in the over clause#40354

Closed
feiniaofeiafei wants to merge 3 commits intoapache:masterfrom
feiniaofeiafei:report_error_for_func_not_support_window
Closed

[Fix](nereids) report an error when a function that cannot be used for window analysis appears in the over clause#40354
feiniaofeiafei wants to merge 3 commits intoapache:masterfrom
feiniaofeiafei:report_error_for_func_not_support_window

Conversation

@feiniaofeiafei
Copy link
Contributor

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38511 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2ddd8fb2370ac9a5a447b5b775ed56cf00e5b992, data reload: false

------ Round 1 ----------------------------------
q1	18268	5829	4338	4338
q2	2036	188	171	171
q3	11676	991	1160	991
q4	10442	795	725	725
q5	7723	2864	2830	2830
q6	227	141	138	138
q7	949	615	598	598
q8	9326	2087	2123	2087
q9	7348	6568	6587	6568
q10	7020	2269	2205	2205
q11	431	238	236	236
q12	401	219	222	219
q13	17852	3104	3095	3095
q14	287	233	246	233
q15	524	498	503	498
q16	602	497	513	497
q17	991	719	756	719
q18	7444	6840	6845	6840
q19	1399	1055	965	965
q20	686	330	347	330
q21	3883	3182	3200	3182
q22	1110	1048	1046	1046
Total cold run time: 110625 ms
Total hot run time: 38511 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4408	4386	4286	4286
q2	386	263	263	263
q3	2886	2688	2705	2688
q4	1941	1670	1731	1670
q5	5684	5644	5825	5644
q6	227	135	139	135
q7	2229	1848	1859	1848
q8	3283	3479	3521	3479
q9	8848	8849	8888	8849
q10	3637	3366	3329	3329
q11	599	511	528	511
q12	823	656	660	656
q13	13155	3258	3314	3258
q14	328	277	288	277
q15	548	501	496	496
q16	660	596	577	577
q17	1854	1543	1586	1543
q18	8129	7879	7934	7879
q19	1776	1600	1598	1598
q20	2147	1964	1950	1950
q21	5696	5520	5410	5410
q22	1145	1071	1065	1065
Total cold run time: 70389 ms
Total hot run time: 57411 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193876 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2ddd8fb2370ac9a5a447b5b775ed56cf00e5b992, data reload: false

query1	1262	893	866	866
query2	6477	1907	1908	1907
query3	10601	4124	4035	4035
query4	59444	26744	23257	23257
query5	5458	540	507	507
query6	403	163	170	163
query7	5774	312	301	301
query8	307	210	217	210
query9	8977	2512	2503	2503
query10	497	280	259	259
query11	18200	15143	15479	15143
query12	154	101	99	99
query13	1564	397	409	397
query14	11235	7584	7508	7508
query15	265	174	189	174
query16	7572	466	461	461
query17	1103	610	556	556
query18	2091	301	291	291
query19	290	154	154	154
query20	124	115	110	110
query21	208	111	108	108
query22	4717	4605	4649	4605
query23	34354	33411	33281	33281
query24	5981	2859	2834	2834
query25	529	369	406	369
query26	682	154	154	154
query27	1790	290	281	281
query28	3766	2070	2060	2060
query29	723	406	410	406
query30	244	156	155	155
query31	926	766	791	766
query32	85	54	51	51
query33	461	298	283	283
query34	861	481	480	480
query35	818	747	729	729
query36	1071	940	936	936
query37	147	89	85	85
query38	4028	3897	3987	3897
query39	1444	1385	1406	1385
query40	191	117	119	117
query41	47	47	44	44
query42	118	97	96	96
query43	523	492	470	470
query44	1088	741	752	741
query45	199	163	165	163
query46	1080	733	733	733
query47	1932	1801	1809	1801
query48	381	303	301	301
query49	774	454	453	453
query50	803	415	421	415
query51	6994	6930	6834	6834
query52	104	91	90	90
query53	261	185	186	185
query54	590	483	462	462
query55	77	78	80	78
query56	296	270	276	270
query57	1178	1078	1092	1078
query58	232	251	243	243
query59	2990	2825	2810	2810
query60	301	275	291	275
query61	127	125	125	125
query62	752	661	651	651
query63	214	186	188	186
query64	2892	774	756	756
query65	3199	3162	3165	3162
query66	687	357	377	357
query67	15312	15179	15219	15179
query68	3067	584	588	584
query69	428	293	290	290
query70	1171	1115	1006	1006
query71	364	282	287	282
query72	6469	4242	4227	4227
query73	758	338	337	337
query74	9188	8943	9044	8943
query75	3525	2697	2659	2659
query76	1496	997	973	973
query77	550	323	320	320
query78	9759	9068	9080	9068
query79	2123	529	531	529
query80	1276	498	506	498
query81	562	236	235	235
query82	1049	147	150	147
query83	174	149	150	149
query84	264	73	78	73
query85	950	288	283	283
query86	391	290	278	278
query87	4453	4315	4306	4306
query88	3495	2340	2344	2340
query89	390	289	285	285
query90	1901	191	190	190
query91	125	99	97	97
query92	63	49	50	49
query93	2611	537	533	533
query94	845	297	294	294
query95	347	256	252	252
query96	602	267	268	267
query97	3214	3070	3087	3070
query98	224	203	200	200
query99	1558	1271	1273	1271
Total cold run time: 311367 ms
Total hot run time: 193876 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2ddd8fb2370ac9a5a447b5b775ed56cf00e5b992, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.68	0.07	0.07
query5	0.52	0.49	0.49
query6	1.12	0.74	0.72
query7	0.01	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.47	0.49
query10	0.55	0.53	0.53
query11	0.16	0.13	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.59
query14	2.05	2.06	2.06
query15	0.89	0.82	0.85
query16	0.38	0.37	0.36
query17	0.97	0.97	0.96
query18	0.21	0.20	0.20
query19	1.95	1.78	1.84
query20	0.01	0.01	0.01
query21	15.39	0.68	0.67
query22	4.24	7.21	2.07
query23	18.25	1.33	1.26
query24	2.07	0.22	0.21
query25	0.15	0.10	0.07
query26	0.27	0.18	0.18
query27	0.08	0.07	0.08
query28	13.27	1.03	1.00
query29	12.61	3.39	3.38
query30	0.25	0.06	0.05
query31	2.90	0.41	0.41
query32	3.22	0.49	0.47
query33	3.00	2.99	3.01
query34	17.26	4.35	4.44
query35	4.45	4.40	4.49
query36	0.65	0.47	0.48
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.06	0.05
query43	0.05	0.05	0.04
Total cold run time: 111.02 s
Total hot run time: 32.11 s

@feiniaofeiafei feiniaofeiafei force-pushed the report_error_for_func_not_support_window branch from 2c5bbdc to 4fef7b2 Compare September 4, 2024 06:04
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@feiniaofeiafei
Copy link
Contributor Author

run buildall

1 similar comment
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38315 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5e40ab00c75ee5b2ac96c2094fbd849f5f9035f8, data reload: false

------ Round 1 ----------------------------------
q1	17612	4816	4337	4337
q2	2005	180	175	175
q3	11720	964	1178	964
q4	10526	723	706	706
q5	7748	2843	2818	2818
q6	224	137	135	135
q7	955	632	594	594
q8	9320	2047	2078	2047
q9	7220	6519	6505	6505
q10	6968	2211	2167	2167
q11	443	239	236	236
q12	408	229	224	224
q13	19017	3097	3068	3068
q14	269	230	231	230
q15	527	496	487	487
q16	596	508	513	508
q17	974	740	661	661
q18	7350	6824	6867	6824
q19	1391	1055	1105	1055
q20	678	323	327	323
q21	4048	3241	3307	3241
q22	1103	1022	1010	1010
Total cold run time: 111102 ms
Total hot run time: 38315 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4338	4319	4300	4300
q2	399	267	265	265
q3	2867	2620	2653	2620
q4	1951	1657	1706	1657
q5	5686	5692	5764	5692
q6	227	130	129	129
q7	2238	1829	1811	1811
q8	3262	3418	3502	3418
q9	8794	8913	8819	8819
q10	3663	3344	3317	3317
q11	584	507	514	507
q12	857	681	670	670
q13	12794	3254	3261	3254
q14	325	291	277	277
q15	524	477	499	477
q16	646	573	581	573
q17	1826	1517	1567	1517
q18	8134	7818	7846	7818
q19	1761	1464	1535	1464
q20	2134	1900	1962	1900
q21	5742	5446	5371	5371
q22	1097	1019	1020	1019
Total cold run time: 69849 ms
Total hot run time: 56875 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192460 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5e40ab00c75ee5b2ac96c2094fbd849f5f9035f8, data reload: false

query1	1246	879	843	843
query2	6421	2041	1931	1931
query3	10625	3881	3850	3850
query4	59497	27655	23240	23240
query5	5220	500	505	500
query6	399	166	168	166
query7	5767	312	297	297
query8	285	216	209	209
query9	8886	2530	2498	2498
query10	500	285	268	268
query11	18130	15111	15398	15111
query12	151	110	102	102
query13	1588	414	391	391
query14	11131	7491	7297	7297
query15	241	173	184	173
query16	7529	458	461	458
query17	1112	556	574	556
query18	2023	285	290	285
query19	281	144	163	144
query20	130	109	107	107
query21	202	113	106	106
query22	4470	4297	4768	4297
query23	33976	33255	33448	33255
query24	5944	2957	2800	2800
query25	526	387	375	375
query26	685	151	145	145
query27	1794	283	277	277
query28	3877	2069	2050	2050
query29	700	411	413	411
query30	245	156	156	156
query31	939	806	799	799
query32	80	50	53	50
query33	484	292	284	284
query34	871	485	501	485
query35	864	751	741	741
query36	1086	947	946	946
query37	159	95	99	95
query38	4034	3906	3876	3876
query39	1462	1395	1424	1395
query40	196	117	120	117
query41	49	48	47	47
query42	119	95	97	95
query43	521	475	488	475
query44	1138	742	767	742
query45	198	162	165	162
query46	1089	732	719	719
query47	1901	1790	1790	1790
query48	374	299	293	293
query49	753	425	428	425
query50	828	418	415	415
query51	6990	6821	6822	6821
query52	103	84	88	84
query53	253	175	187	175
query54	569	467	456	456
query55	77	77	77	77
query56	285	252	249	249
query57	1201	1051	1081	1051
query58	225	224	223	223
query59	2977	2895	2759	2759
query60	286	270	273	270
query61	101	136	99	99
query62	728	655	629	629
query63	217	186	183	183
query64	2757	687	672	672
query65	3228	3164	3130	3130
query66	669	330	338	330
query67	15447	15228	15359	15228
query68	3035	591	587	587
query69	404	285	287	285
query70	1185	1113	1110	1110
query71	401	281	279	279
query72	5301	3936	4168	3936
query73	759	330	330	330
query74	9285	8810	8912	8810
query75	3324	2684	2709	2684
query76	1373	1004	1040	1004
query77	524	306	331	306
query78	9582	9114	9448	9114
query79	1008	540	532	532
query80	714	528	503	503
query81	466	236	237	236
query82	244	138	150	138
query83	170	148	147	147
query84	264	75	74	74
query85	676	286	280	280
query86	314	297	303	297
query87	4433	4376	4227	4227
query88	3032	2400	2382	2382
query89	382	287	291	287
query90	1994	188	198	188
query91	139	109	107	107
query92	60	51	52	51
query93	1050	535	539	535
query94	726	296	385	296
query95	327	257	254	254
query96	591	269	268	268
query97	3236	3083	3052	3052
query98	226	200	207	200
query99	1761	1280	1276	1276
Total cold run time: 303825 ms
Total hot run time: 192460 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5e40ab00c75ee5b2ac96c2094fbd849f5f9035f8, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.08	0.07
query5	0.51	0.50	0.48
query6	1.12	0.74	0.73
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.49
query10	0.54	0.54	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	2.09	2.04	2.09
query15	0.91	0.82	0.82
query16	0.37	0.38	0.38
query17	1.04	1.06	1.06
query18	0.22	0.20	0.21
query19	1.87	1.86	1.77
query20	0.01	0.01	0.01
query21	15.38	0.67	0.68
query22	4.34	7.78	1.83
query23	18.29	1.33	1.29
query24	2.08	0.23	0.22
query25	0.14	0.09	0.08
query26	0.26	0.18	0.17
query27	0.08	0.07	0.08
query28	13.25	1.01	1.00
query29	12.64	3.40	3.35
query30	0.24	0.06	0.05
query31	2.87	0.40	0.40
query32	3.24	0.49	0.48
query33	3.03	2.98	3.00
query34	16.98	4.38	4.44
query35	4.42	4.45	4.42
query36	0.66	0.47	0.49
query37	0.19	0.15	0.16
query38	0.16	0.14	0.16
query39	0.04	0.04	0.04
query40	0.15	0.13	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.86 s
Total hot run time: 32.05 s

feiniaofeiafei and others added 3 commits September 5, 2024 10:06
…r window analysis appears in the over clause
…r window analysis appears in the over clause
…e/ExtractAndNormalizeWindowExpression.java

Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com>
@feiniaofeiafei feiniaofeiafei force-pushed the report_error_for_func_not_support_window branch from 5e40ab0 to 001163e Compare September 5, 2024 02:07
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38306 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 001163edf36d9d3e02f99e49c4a56f7415ee3e4b, data reload: false

------ Round 1 ----------------------------------
q1	18705	4546	4379	4379
q2	2030	186	191	186
q3	10460	1192	1180	1180
q4	10152	731	768	731
q5	7733	2870	2783	2783
q6	229	140	140	140
q7	968	618	616	616
q8	9387	2072	2073	2072
q9	7211	6529	6541	6529
q10	7020	2271	2182	2182
q11	494	250	253	250
q12	396	228	224	224
q13	18953	3070	3073	3070
q14	289	245	246	245
q15	533	499	499	499
q16	591	508	501	501
q17	977	746	740	740
q18	7468	6840	6816	6816
q19	1386	1078	1059	1059
q20	694	320	347	320
q21	4053	3132	2783	2783
q22	1110	1016	1001	1001
Total cold run time: 110839 ms
Total hot run time: 38306 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4359	4281	4281	4281
q2	387	285	269	269
q3	2902	2662	2685	2662
q4	1971	1689	1678	1678
q5	5415	5409	5394	5394
q6	226	129	133	129
q7	2154	1726	1774	1726
q8	3238	3381	3378	3378
q9	8433	8442	8441	8441
q10	3464	3213	3217	3213
q11	589	511	497	497
q12	780	633	601	601
q13	12276	3096	3105	3096
q14	306	283	263	263
q15	534	499	475	475
q16	590	552	572	552
q17	1792	1503	1487	1487
q18	7695	7382	7603	7382
q19	1649	1571	1528	1528
q20	2087	1835	1849	1835
q21	5483	5347	5294	5294
q22	1120	1010	1031	1010
Total cold run time: 67450 ms
Total hot run time: 55191 ms

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-DS: Total hot run time: 188372 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 001163edf36d9d3e02f99e49c4a56f7415ee3e4b, data reload: false

query1	924	369	387	369
query2	6480	2007	1981	1981
query3	6651	213	214	213
query4	33698	23383	23152	23152
query5	4185	514	488	488
query6	262	173	158	158
query7	4599	296	293	293
query8	283	224	217	217
query9	8548	2511	2488	2488
query10	443	291	275	275
query11	16788	14947	15050	14947
query12	160	101	98	98
query13	1621	389	376	376
query14	9748	6938	6893	6893
query15	276	165	175	165
query16	7896	445	450	445
query17	1594	569	544	544
query18	2083	287	288	287
query19	185	140	139	139
query20	114	109	109	109
query21	209	104	108	104
query22	4533	4336	4120	4120
query23	34069	33503	33870	33503
query24	11186	2841	2945	2841
query25	632	394	391	391
query26	1379	159	152	152
query27	2866	274	280	274
query28	7231	2053	2038	2038
query29	901	417	418	417
query30	308	162	161	161
query31	986	769	796	769
query32	94	53	57	53
query33	755	289	311	289
query34	965	471	484	471
query35	843	749	712	712
query36	1085	965	934	934
query37	170	100	93	93
query38	3973	3931	3890	3890
query39	1469	1399	1423	1399
query40	276	114	112	112
query41	50	47	46	46
query42	113	95	94	94
query43	511	469	485	469
query44	1256	756	741	741
query45	196	167	170	167
query46	1104	732	721	721
query47	1850	1779	1782	1779
query48	395	314	313	313
query49	1108	447	439	439
query50	829	412	415	412
query51	7021	6941	6938	6938
query52	98	89	88	88
query53	253	180	182	180
query54	930	468	473	468
query55	78	74	77	74
query56	283	254	255	254
query57	1205	1085	1092	1085
query58	237	224	236	224
query59	3036	2747	2906	2747
query60	303	268	268	268
query61	104	102	138	102
query62	812	657	672	657
query63	224	180	185	180
query64	5273	675	689	675
query65	3262	3162	3172	3162
query66	1240	349	358	349
query67	15570	15484	15265	15265
query68	3127	587	577	577
query69	392	284	284	284
query70	1154	1159	1019	1019
query71	333	282	281	281
query72	6379	4144	4150	4144
query73	756	328	334	328
query74	9060	8874	8919	8874
query75	3430	2730	2650	2650
query76	1998	989	1035	989
query77	492	333	332	332
query78	9886	9085	9521	9085
query79	2607	535	554	535
query80	1071	534	531	531
query81	591	242	236	236
query82	795	154	149	149
query83	259	160	163	160
query84	244	91	81	81
query85	1294	359	346	346
query86	324	303	312	303
query87	4417	4257	4320	4257
query88	3128	2425	2400	2400
query89	402	291	295	291
query90	1649	203	199	199
query91	141	119	116	116
query92	67	50	49	49
query93	1686	550	538	538
query94	602	313	313	313
query95	369	273	283	273
query96	729	314	269	269
query97	3241	3061	3043	3043
query98	234	197	196	196
query99	1748	1276	1267	1267
Total cold run time: 290414 ms
Total hot run time: 188372 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 001163edf36d9d3e02f99e49c4a56f7415ee3e4b, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.03	0.04
query3	0.23	0.06	0.05
query4	1.65	0.09	0.08
query5	0.51	0.49	0.49
query6	1.12	0.73	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.05
query9	0.54	0.49	0.49
query10	0.55	0.57	0.54
query11	0.15	0.12	0.11
query12	0.16	0.12	0.13
query13	0.60	0.59	0.59
query14	2.05	2.06	2.11
query15	0.84	0.82	0.82
query16	0.39	0.36	0.37
query17	1.07	0.98	0.99
query18	0.23	0.21	0.20
query19	1.81	1.71	1.80
query20	0.01	0.01	0.01
query21	15.42	0.68	0.67
query22	3.52	6.86	2.56
query23	18.24	1.39	1.34
query24	2.11	0.24	0.21
query25	0.14	0.09	0.08
query26	0.27	0.18	0.19
query27	0.08	0.07	0.07
query28	13.24	1.01	1.00
query29	12.63	3.37	3.29
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.24	0.48	0.48
query33	2.98	3.04	2.99
query34	16.97	4.38	4.37
query35	4.41	4.52	4.46
query36	0.66	0.47	0.47
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.06	0.04	0.04
query40	0.16	0.12	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.91 s
Total hot run time: 32.66 s

@morrySnow morrySnow closed this Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants