Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](nereids) when table.rows > topn.limit+topn.offset, do not generate topn-filter #40480

Merged
merged 6 commits into from
Sep 11, 2024
Merged

[opt](nereids) when table.rows > topn.limit+topn.offset, do not generate topn-filter #40480

merged 6 commits into from
Sep 11, 2024

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Sep 6, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

englefly commented Sep 6, 2024

run buildall

BiteTheDDDDt
BiteTheDDDDt previously approved these changes Sep 6, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 6, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 6, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38168 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e1214f14b49229947ca43ad7cafdf80afad56248, data reload: false

------ Round 1 ----------------------------------
q1	17993	4668	4328	4328
q2	2020	191	188	188
q3	11464	1011	1181	1011
q4	10438	701	700	700
q5	7765	2875	2826	2826
q6	225	143	144	143
q7	961	617	632	617
q8	9338	2084	2096	2084
q9	7307	6549	6561	6549
q10	7005	2309	2190	2190
q11	461	240	239	239
q12	399	227	226	226
q13	17991	3088	3078	3078
q14	270	238	251	238
q15	537	490	479	479
q16	526	424	434	424
q17	996	750	704	704
q18	7463	6795	7034	6795
q19	1383	1074	1051	1051
q20	678	333	347	333
q21	3983	3110	2949	2949
q22	1133	1016	1032	1016
Total cold run time: 110336 ms
Total hot run time: 38168 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4335	4254	4850	4254
q2	391	292	284	284
q3	2879	2610	2730	2610
q4	1966	1737	1672	1672
q5	5664	5717	5792	5717
q6	235	133	135	133
q7	2216	1835	1849	1835
q8	3316	3455	3527	3455
q9	8830	8853	8820	8820
q10	3643	3438	3386	3386
q11	611	518	503	503
q12	811	658	676	658
q13	14559	3213	3329	3213
q14	321	289	301	289
q15	543	480	492	480
q16	530	507	519	507
q17	1846	1571	1552	1552
q18	8129	7809	7818	7809
q19	1695	1618	1694	1618
q20	2128	1907	1901	1901
q21	5804	5600	5612	5600
q22	1106	1047	1032	1032
Total cold run time: 71558 ms
Total hot run time: 57328 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192494 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e1214f14b49229947ca43ad7cafdf80afad56248, data reload: false

query1	1266	902	882	882
query2	6279	1931	1949	1931
query3	10699	4038	4171	4038
query4	59947	27336	23211	23211
query5	5462	516	514	514
query6	399	165	165	165
query7	5870	292	291	291
query8	325	232	220	220
query9	9152	2503	2489	2489
query10	495	289	281	281
query11	18202	15037	15519	15037
query12	172	100	102	100
query13	1576	408	384	384
query14	10823	7642	7387	7387
query15	243	171	187	171
query16	7563	460	493	460
query17	1113	556	561	556
query18	2066	303	287	287
query19	276	168	153	153
query20	123	110	105	105
query21	205	107	103	103
query22	4637	4311	4386	4311
query23	34079	33494	33438	33438
query24	5938	2889	2839	2839
query25	529	378	389	378
query26	691	156	156	156
query27	1793	268	274	268
query28	3736	2026	2020	2020
query29	686	410	404	404
query30	239	155	149	149
query31	947	762	785	762
query32	83	54	57	54
query33	443	286	288	286
query34	867	481	487	481
query35	840	713	727	713
query36	1077	935	933	933
query37	142	86	83	83
query38	4004	3833	3983	3833
query39	1511	1387	1417	1387
query40	201	114	116	114
query41	50	47	47	47
query42	114	98	98	98
query43	517	487	488	487
query44	1090	750	750	750
query45	238	163	162	162
query46	1082	736	735	735
query47	1889	1789	1830	1789
query48	365	290	294	290
query49	767	455	463	455
query50	813	421	434	421
query51	7105	6883	6981	6883
query52	99	87	86	86
query53	248	184	184	184
query54	573	451	448	448
query55	78	74	77	74
query56	278	261	277	261
query57	1183	1083	1058	1058
query58	214	262	232	232
query59	3000	2929	2976	2929
query60	301	264	267	264
query61	113	98	98	98
query62	770	663	636	636
query63	215	191	191	191
query64	2793	661	653	653
query65	3208	3111	3162	3111
query66	686	325	339	325
query67	15447	14957	15261	14957
query68	3001	622	581	581
query69	402	292	282	282
query70	1176	1042	1057	1042
query71	353	274	275	274
query72	5391	3971	4051	3971
query73	740	320	322	320
query74	9136	8738	8775	8738
query75	3379	2655	2661	2655
query76	1425	946	981	946
query77	525	347	333	333
query78	10637	9295	9148	9148
query79	1049	568	529	529
query80	688	514	514	514
query81	455	234	230	230
query82	235	134	145	134
query83	171	154	152	152
query84	248	79	76	76
query85	677	290	285	285
query86	301	291	303	291
query87	4510	4210	4307	4210
query88	3422	2329	2454	2329
query89	373	288	293	288
query90	1910	194	193	193
query91	125	96	98	96
query92	58	48	48	48
query93	1045	534	536	534
query94	711	296	301	296
query95	322	254	260	254
query96	582	266	267	266
query97	3175	3017	3088	3017
query98	218	208	203	203
query99	1514	1261	1287	1261
Total cold run time: 305889 ms
Total hot run time: 192494 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.82 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e1214f14b49229947ca43ad7cafdf80afad56248, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.03	0.04
query3	0.23	0.05	0.04
query4	1.67	0.07	0.07
query5	0.52	0.48	0.48
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.06	0.05	0.04
query9	0.54	0.49	0.48
query10	0.55	0.56	0.55
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.59
query14	1.43	1.44	1.39
query15	0.84	0.82	0.81
query16	0.37	0.37	0.37
query17	1.05	0.97	1.05
query18	0.18	0.17	0.18
query19	1.84	1.77	1.70
query20	0.02	0.01	0.01
query21	15.39	0.66	0.65
query22	3.98	6.57	2.47
query23	18.29	1.38	1.35
query24	2.19	0.22	0.21
query25	0.16	0.08	0.07
query26	0.29	0.18	0.18
query27	0.07	0.08	0.08
query28	13.19	1.02	1.00
query29	12.66	3.30	3.29
query30	0.24	0.06	0.05
query31	2.86	0.40	0.40
query32	3.30	0.48	0.50
query33	2.96	3.04	3.00
query34	16.93	4.38	4.44
query35	4.55	4.45	4.53
query36	0.66	0.47	0.49
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.05
query40	0.15	0.13	0.13
query41	0.09	0.05	0.04
query42	0.05	0.06	0.05
query43	0.05	0.03	0.04
Total cold run time: 109.92 s
Total hot run time: 31.82 s

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 9, 2024
@englefly
Copy link
Contributor Author

englefly commented Sep 9, 2024

run buildall

1 similar comment
@englefly
Copy link
Contributor Author

englefly commented Sep 9, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37970 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 315fd0fc3343693b40479222506e7e65fd83a948, data reload: false

------ Round 1 ----------------------------------
q1	17673	4487	4363	4363
q2	2017	202	201	201
q3	11903	953	1104	953
q4	10516	796	761	761
q5	7758	2874	2817	2817
q6	227	143	140	140
q7	982	628	609	609
q8	9334	2076	2103	2076
q9	6973	6576	6590	6576
q10	7013	2218	2191	2191
q11	495	259	252	252
q12	401	230	231	230
q13	17759	3109	3067	3067
q14	302	232	251	232
q15	560	504	491	491
q16	533	429	442	429
q17	989	740	763	740
q18	7509	7050	6931	6931
q19	1461	788	792	788
q20	792	694	702	694
q21	3113	2359	2495	2359
q22	1094	1085	1070	1070
Total cold run time: 109404 ms
Total hot run time: 37970 ms

----- Round 2, with runtime_filter_mode=off -----
q1	3261	3346	3311	3311
q2	320	309	301	301
q3	2186	2217	2211	2211
q4	1341	1362	1390	1362
q5	4405	4371	4752	4371
q6	288	288	294	288
q7	1902	1872	2107	1872
q8	2424	2426	2402	2402
q9	7313	7446	7369	7369
q10	2914	2924	2898	2898
q11	538	525	540	525
q12	712	688	793	688
q13	16725	3580	3619	3580
q14	284	298	273	273
q15	522	526	518	518
q16	526	506	485	485
q17	1225	1185	1169	1169
q18	7057	7055	7183	7055
q19	931	960	949	949
q20	2037	1963	1954	1954
q21	5088	5138	5182	5138
q22	1159	1139	1082	1082
Total cold run time: 63158 ms
Total hot run time: 49801 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198061 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 315fd0fc3343693b40479222506e7e65fd83a948, data reload: false

query1	1262	874	853	853
query2	6193	1848	1951	1848
query3	10643	4030	4044	4030
query4	60970	28760	23115	23115
query5	4967	507	492	492
query6	400	164	175	164
query7	5629	305	310	305
query8	326	216	210	210
query9	7758	2488	2470	2470
query10	404	279	268	268
query11	17106	15089	15336	15089
query12	155	104	98	98
query13	1446	413	423	413
query14	10287	7144	7500	7144
query15	201	184	174	174
query16	6851	479	465	465
query17	1118	566	561	561
query18	1341	305	293	293
query19	207	151	144	144
query20	125	109	111	109
query21	219	108	107	107
query22	4620	4458	4562	4458
query23	34780	33974	33895	33895
query24	6027	2891	2887	2887
query25	515	404	412	404
query26	613	158	161	158
query27	1607	281	291	281
query28	3688	2046	2021	2021
query29	665	422	441	422
query30	201	153	144	144
query31	951	735	789	735
query32	71	61	59	59
query33	440	304	301	301
query34	883	487	483	483
query35	821	733	739	733
query36	1067	918	951	918
query37	141	85	85	85
query38	4018	3895	3886	3886
query39	1474	1469	1404	1404
query40	213	118	124	118
query41	50	48	47	47
query42	116	101	99	99
query43	515	462	467	462
query44	1098	783	753	753
query45	200	167	170	167
query46	1104	734	755	734
query47	1857	1829	1844	1829
query48	385	303	311	303
query49	780	463	454	454
query50	825	429	428	428
query51	7026	6964	6920	6920
query52	98	88	89	88
query53	257	180	197	180
query54	590	474	471	471
query55	76	76	77	76
query56	290	270	270	270
query57	1184	1073	1111	1073
query58	233	249	238	238
query59	2938	2755	2648	2648
query60	306	289	287	287
query61	129	126	130	126
query62	773	670	652	652
query63	225	185	187	185
query64	1488	663	687	663
query65	3240	3156	3233	3156
query66	683	341	341	341
query67	15808	15444	15188	15188
query68	2047	870	851	851
query69	417	328	328	328
query70	1182	1210	1199	1199
query71	364	353	340	340
query72	4912	3537	3658	3537
query73	601	578	588	578
query74	9079	8998	9034	8998
query75	3038	2979	2945	2945
query76	1032	861	868	861
query77	516	402	410	402
query78	9578	9268	9440	9268
query79	925	885	865	865
query80	812	809	821	809
query81	456	262	259	259
query82	267	265	271	265
query83	195	188	187	187
query84	206	108	106	106
query85	511	450	387	387
query86	317	305	330	305
query87	4348	4253	4332	4253
query88	4335	4147	4134	4134
query89	369	375	361	361
query90	1529	322	312	312
query91	125	120	123	120
query92	77	75	77	75
query93	937	939	941	939
query94	561	368	386	368
query95	428	419	408	408
query96	476	478	479	478
query97	3093	3074	3129	3074
query98	236	229	226	226
query99	1363	1241	1293	1241
Total cold run time: 297009 ms
Total hot run time: 198061 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.49 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 315fd0fc3343693b40479222506e7e65fd83a948, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.08	0.07
query5	0.48	0.50	0.49
query6	1.12	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.51	0.48
query10	0.55	0.56	0.55
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	1.37	1.44	1.42
query15	0.84	0.81	0.83
query16	0.38	0.38	0.37
query17	1.05	1.00	0.98
query18	0.19	0.18	0.18
query19	1.85	1.72	1.73
query20	0.01	0.01	0.01
query21	15.41	0.68	0.66
query22	4.18	7.24	1.98
query23	18.29	1.41	1.31
query24	2.09	0.22	0.22
query25	0.17	0.09	0.08
query26	0.27	0.19	0.18
query27	0.08	0.08	0.08
query28	13.30	1.00	1.00
query29	12.63	3.35	3.34
query30	0.24	0.06	0.06
query31	2.86	0.40	0.40
query32	3.26	0.48	0.48
query33	2.95	3.00	2.99
query34	16.99	4.39	4.42
query35	4.45	4.44	4.43
query36	0.66	0.49	0.48
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.13
query41	0.10	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.99 s
Total hot run time: 31.49 s

@englefly
Copy link
Contributor Author

englefly commented Sep 9, 2024

run feut

@englefly
Copy link
Contributor Author

englefly commented Sep 9, 2024

run p0

@englefly
Copy link
Contributor Author

englefly commented Sep 9, 2024

run cloud_p0

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37669 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2535dddf9bed4a2a1eb7e6d00a21caebab5eaa0e, data reload: false

------ Round 1 ----------------------------------
q1	17623	4411	4280	4280
q2	2032	188	183	183
q3	11762	938	1165	938
q4	10507	772	667	667
q5	7752	2784	2833	2784
q6	231	140	139	139
q7	964	612	600	600
q8	9328	2037	2070	2037
q9	7047	6501	6567	6501
q10	6985	2224	2218	2218
q11	445	243	255	243
q12	405	224	220	220
q13	18022	3072	3075	3072
q14	279	236	238	236
q15	538	497	479	479
q16	534	432	439	432
q17	967	717	652	652
q18	7279	6822	6772	6772
q19	1402	1041	1022	1022
q20	687	322	335	322
q21	3942	3020	2865	2865
q22	1125	1014	1007	1007
Total cold run time: 109856 ms
Total hot run time: 37669 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4376	4315	4301	4301
q2	375	281	272	272
q3	2863	2661	2615	2615
q4	1977	1648	1657	1648
q5	5579	5659	5675	5659
q6	225	136	145	136
q7	2209	1808	1835	1808
q8	3289	3437	3424	3424
q9	8863	8804	8803	8803
q10	3561	3379	3361	3361
q11	613	520	505	505
q12	820	655	647	647
q13	15306	3255	3252	3252
q14	321	278	297	278
q15	558	483	502	483
q16	560	511	498	498
q17	1839	1518	1536	1518
q18	8083	7879	7877	7877
q19	1716	1545	1465	1465
q20	2139	1917	1894	1894
q21	5806	5524	5471	5471
q22	1111	1056	1003	1003
Total cold run time: 72189 ms
Total hot run time: 56918 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192528 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2535dddf9bed4a2a1eb7e6d00a21caebab5eaa0e, data reload: false

query1	1236	881	859	859
query2	6331	1958	1935	1935
query3	10633	3956	3949	3949
query4	60276	26128	23312	23312
query5	5529	509	516	509
query6	433	164	162	162
query7	5892	292	287	287
query8	315	217	218	217
query9	9032	2495	2508	2495
query10	519	281	271	271
query11	15901	15022	15288	15022
query12	156	100	109	100
query13	1565	389	388	388
query14	11251	7516	7026	7026
query15	244	181	177	177
query16	7581	501	467	467
query17	1171	598	593	593
query18	2070	320	295	295
query19	302	161	152	152
query20	120	110	108	108
query21	209	109	106	106
query22	4493	4418	4377	4377
query23	34266	33317	33369	33317
query24	5908	2933	2825	2825
query25	551	402	418	402
query26	700	155	158	155
query27	1799	282	275	275
query28	3791	2031	2002	2002
query29	720	414	408	408
query30	242	147	147	147
query31	950	721	753	721
query32	87	54	53	53
query33	466	307	307	307
query34	874	477	478	477
query35	834	737	717	717
query36	1058	930	976	930
query37	146	86	84	84
query38	3977	3878	3862	3862
query39	1440	1415	1376	1376
query40	207	113	115	113
query41	46	48	44	44
query42	117	95	94	94
query43	506	479	459	459
query44	1088	780	742	742
query45	200	163	165	163
query46	1081	728	719	719
query47	1876	1753	1779	1753
query48	378	308	293	293
query49	770	454	451	451
query50	831	414	414	414
query51	7113	6856	6879	6856
query52	97	85	88	85
query53	249	178	179	178
query54	565	450	454	450
query55	75	77	76	76
query56	268	261	261	261
query57	1194	1101	1072	1072
query58	253	218	228	218
query59	3062	2963	2910	2910
query60	311	273	272	272
query61	104	105	97	97
query62	768	646	644	644
query63	227	185	185	185
query64	2831	692	739	692
query65	3204	3115	3136	3115
query66	662	328	336	328
query67	15292	15107	15120	15107
query68	3037	570	593	570
query69	394	287	277	277
query70	1149	1103	1122	1103
query71	364	278	283	278
query72	6318	4247	4005	4005
query73	744	320	331	320
query74	8955	8849	8810	8810
query75	3357	2673	2715	2673
query76	1429	1045	946	946
query77	573	328	321	321
query78	9645	10311	9432	9432
query79	1073	556	532	532
query80	756	508	513	508
query81	544	230	225	225
query82	330	140	138	138
query83	175	156	152	152
query84	258	76	77	76
query85	691	295	286	286
query86	310	291	299	291
query87	4499	4227	4170	4170
query88	2954	2286	2392	2286
query89	389	285	283	283
query90	2033	196	195	195
query91	125	98	99	98
query92	62	48	49	48
query93	1071	535	540	535
query94	745	296	355	296
query95	348	257	252	252
query96	580	271	261	261
query97	3226	3056	3096	3056
query98	225	201	199	199
query99	1554	1326	1250	1250
Total cold run time: 304356 ms
Total hot run time: 192528 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2535dddf9bed4a2a1eb7e6d00a21caebab5eaa0e, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.08	0.07
query5	0.51	0.50	0.48
query6	1.14	0.72	0.73
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.48	0.49
query10	0.55	0.56	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.58	0.58
query14	1.40	1.43	1.39
query15	0.84	0.83	0.84
query16	0.38	0.36	0.36
query17	1.07	1.05	1.01
query18	0.17	0.17	0.18
query19	1.88	1.80	1.79
query20	0.01	0.00	0.01
query21	15.39	0.68	0.67
query22	4.69	7.29	2.01
query23	18.29	1.36	1.28
query24	2.14	0.22	0.21
query25	0.16	0.08	0.09
query26	0.26	0.18	0.18
query27	0.08	0.08	0.07
query28	13.27	1.01	0.99
query29	12.64	3.36	3.34
query30	0.25	0.06	0.06
query31	2.89	0.39	0.38
query32	3.26	0.48	0.47
query33	2.96	3.01	3.00
query34	17.21	4.33	4.39
query35	4.46	4.44	4.45
query36	0.65	0.48	0.47
query37	0.19	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.04
query42	0.05	0.06	0.05
query43	0.04	0.04	0.05
Total cold run time: 110.79 s
Total hot run time: 31.41 s

@englefly
Copy link
Contributor Author

run buildall

@englefly
Copy link
Contributor Author

run cloud_p0

@englefly
Copy link
Contributor Author

run performance

@doris-robot
Copy link

TPC-H: Total hot run time: 38343 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit db6fc9ff2cdc1f4b431545b463a8c9649f48f7f6, data reload: false

------ Round 1 ----------------------------------
q1	18060	4678	4408	4408
q2	2639	197	186	186
q3	11076	1151	1073	1073
q4	10613	745	665	665
q5	7800	2879	2834	2834
q6	231	141	136	136
q7	981	619	612	612
q8	9872	2043	2049	2043
q9	7386	6555	6568	6555
q10	7007	2243	2212	2212
q11	467	247	252	247
q12	398	226	223	223
q13	17855	3085	3071	3071
q14	290	242	239	239
q15	541	496	478	478
q16	518	430	434	430
q17	970	650	653	650
q18	7319	6969	7011	6969
q19	1390	1003	1035	1003
q20	687	330	333	330
q21	4020	2972	3089	2972
q22	1101	1007	1028	1007
Total cold run time: 111221 ms
Total hot run time: 38343 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4375	4287	4306	4287
q2	372	281	260	260
q3	2885	2688	2677	2677
q4	1963	1697	1684	1684
q5	5440	5426	5425	5425
q6	217	129	130	129
q7	2097	1742	1731	1731
q8	3198	3366	3355	3355
q9	8454	8456	8477	8456
q10	3483	3209	3167	3167
q11	599	511	506	506
q12	797	620	609	609
q13	9737	3105	3125	3105
q14	309	276	287	276
q15	529	480	490	480
q16	522	473	464	464
q17	1786	1500	1485	1485
q18	7830	7538	7430	7430
q19	1698	1567	1511	1511
q20	2089	1812	1842	1812
q21	5477	5270	5353	5270
q22	1130	1025	1050	1025
Total cold run time: 64987 ms
Total hot run time: 55144 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192802 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit db6fc9ff2cdc1f4b431545b463a8c9649f48f7f6, data reload: false

query1	914	380	363	363
query2	6456	1900	1849	1849
query3	6643	204	217	204
query4	34111	23258	23180	23180
query5	4214	505	486	486
query6	256	164	164	164
query7	4582	290	297	290
query8	279	210	218	210
query9	8446	2483	2481	2481
query10	435	264	251	251
query11	15955	15067	15186	15067
query12	148	98	95	95
query13	1641	376	364	364
query14	9819	7233	6801	6801
query15	283	171	169	169
query16	8089	438	450	438
query17	1596	576	564	564
query18	2140	309	295	295
query19	226	148	154	148
query20	119	111	111	111
query21	216	106	105	105
query22	4324	4366	4279	4279
query23	34383	33942	33805	33805
query24	11222	2838	2884	2838
query25	633	401	407	401
query26	1424	159	167	159
query27	2873	278	281	278
query28	7644	2054	2045	2045
query29	826	436	427	427
query30	314	162	153	153
query31	986	799	821	799
query32	100	59	57	57
query33	779	293	301	293
query34	978	481	498	481
query35	868	731	729	729
query36	1094	932	943	932
query37	153	89	87	87
query38	3977	3869	3957	3869
query39	1455	1412	1406	1406
query40	268	123	117	117
query41	52	48	49	48
query42	119	97	100	97
query43	510	476	474	474
query44	1173	768	747	747
query45	208	177	171	171
query46	1127	737	747	737
query47	1876	1804	1814	1804
query48	381	292	301	292
query49	1183	455	465	455
query50	827	414	409	409
query51	6965	7021	6908	6908
query52	100	96	88	88
query53	260	190	188	188
query54	944	461	488	461
query55	80	76	77	76
query56	298	266	277	266
query57	1232	1186	1088	1088
query58	262	231	238	231
query59	3001	2769	2797	2769
query60	293	260	265	260
query61	102	111	99	99
query62	842	651	661	651
query63	223	194	187	187
query64	5092	695	653	653
query65	3272	3202	3132	3132
query66	1394	351	341	341
query67	15832	15370	15281	15281
query68	3129	879	838	838
query69	417	316	325	316
query70	1179	1162	1100	1100
query71	348	337	339	337
query72	6158	3559	3479	3479
query73	588	586	580	580
query74	9156	8961	8995	8961
query75	3183	2941	2988	2941
query76	1844	854	855	854
query77	488	412	392	392
query78	9364	9144	9745	9144
query79	904	885	860	860
query80	881	812	821	812
query81	453	261	268	261
query82	266	266	261	261
query83	203	192	193	192
query84	232	108	106	106
query85	647	396	391	391
query86	326	299	321	299
query87	4381	4410	4317	4317
query88	4447	4104	4054	4054
query89	373	366	376	366
query90	1506	318	315	315
query91	128	122	123	122
query92	76	74	75	74
query93	926	934	924	924
query94	577	350	337	337
query95	420	420	404	404
query96	468	472	468	468
query97	3111	3119	3088	3088
query98	238	228	232	228
query99	1395	1282	1278	1278
Total cold run time: 286329 ms
Total hot run time: 192802 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit db6fc9ff2cdc1f4b431545b463a8c9649f48f7f6, data reload: false

query1	0.05	0.05	0.04
query2	0.09	0.04	0.04
query3	0.23	0.04	0.04
query4	1.67	0.07	0.07
query5	0.50	0.50	0.52
query6	1.13	0.73	0.74
query7	0.01	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.49
query10	0.56	0.55	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.60
query14	1.40	1.45	1.42
query15	0.83	0.82	0.83
query16	0.38	0.36	0.36
query17	1.01	0.96	0.98
query18	0.22	0.20	0.21
query19	1.81	1.72	1.79
query20	0.02	0.01	0.01
query21	15.39	0.67	0.66
query22	4.43	8.33	1.02
query23	18.18	1.37	1.24
query24	2.03	0.24	0.23
query25	0.15	0.07	0.08
query26	0.28	0.17	0.18
query27	0.08	0.08	0.07
query28	13.22	1.01	1.00
query29	12.61	3.34	3.35
query30	0.25	0.05	0.06
query31	2.85	0.41	0.41
query32	3.23	0.48	0.47
query33	3.02	2.99	3.03
query34	16.91	4.43	4.41
query35	4.44	4.48	4.51
query36	0.67	0.48	0.48
query37	0.19	0.15	0.14
query38	0.16	0.14	0.14
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.9 s
Total hot run time: 30.5 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 11, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@englefly englefly merged commit 042bfce into apache:master Sep 11, 2024
25 of 28 checks passed
@englefly englefly deleted the limit-topnfilter branch September 11, 2024 08:13
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@englefly @dataroaring
[opt](nereids) when table.rows > topn.limit+topn.offset, do not gener…
0f5b58c 
…ate topn-filter (#40480)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LiBinfeng-01 LiBinfeng-01 LiBinfeng-01 approved these changes

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@englefly @doris-robot @BiteTheDDDDt @LiBinfeng-01 @morrySnow