Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feat](nereids)push Limit to local agg #34853

Merged
merged 4 commits into from
Jul 8, 2024
Merged

[feat](nereids)push Limit to local agg #34853

merged 4 commits into from
Jul 8, 2024

Conversation

englefly
Copy link
Contributor

@englefly englefly commented May 14, 2024 •
edited
Loading

Proposed changes

for a pattern: topn(n)->globalAgg->localAgg
this pr tries to add a filter on global/localAgg which means only the first n tuples are counted, and others could be ignored.

inorder to obtain this benefit, optimizer will change limit node to topn node if the limit number is less than topnOptLimitThreshold.

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@morrySnow
Copy link
Contributor

add description please

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@morrySnow
Copy link
Contributor

add description

@englefly
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 42152 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e9d6017c9dbf124d3af8c1cb73e23ad03eae6a3a, data reload: false

------ Round 1 ----------------------------------
q1	17606	5006	4316	4316
q2	2023	196	195	195
q3	10444	1163	1215	1163
q4	10194	861	835	835
q5	7461	2699	2707	2699
q6	222	134	134	134
q7	1043	602	610	602
q8	9246	2193	2105	2105
q9	9147	6760	6757	6757
q10	9178	3943	3882	3882
q11	459	242	251	242
q12	458	222	229	222
q13	17252	3210	3237	3210
q14	285	234	219	219
q15	528	467	468	467
q16	517	380	398	380
q17	989	669	649	649
q18	8569	7852	7999	7852
q19	6185	1585	1493	1493
q20	644	327	339	327
q21	5252	4185	4128	4128
q22	354	291	275	275
Total cold run time: 118056 ms
Total hot run time: 42152 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4545	4465	4439	4439
q2	380	281	266	266
q3	3174	2927	2755	2755
q4	1868	1607	1688	1607
q5	5536	5535	5498	5498
q6	212	121	128	121
q7	2336	1962	1978	1962
q8	3293	3426	3505	3426
q9	8713	8851	8642	8642
q10	3926	3857	3845	3845
q11	613	494	496	494
q12	812	617	603	603
q13	15955	3115	3197	3115
q14	300	284	270	270
q15	530	497	493	493
q16	507	418	429	418
q17	1787	1515	1466	1466
q18	7769	7650	7633	7633
q19	1667	1586	1515	1515
q20	2024	1789	1776	1776
q21	11576	4809	4951	4809
q22	563	497	460	460
Total cold run time: 78086 ms
Total hot run time: 55613 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 181914 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e9d6017c9dbf124d3af8c1cb73e23ad03eae6a3a, data reload: false

query1	913	394	385	385
query2	6434	2308	2437	2308
query3	6654	205	205	205
query4	23729	21263	21312	21263
query5	4140	420	416	416
query6	273	177	176	176
query7	4580	288	286	286
query8	243	192	195	192
query9	8535	2401	2385	2385
query10	424	249	246	246
query11	14660	14211	14159	14159
query12	133	89	87	87
query13	1637	380	375	375
query14	9670	8529	7722	7722
query15	226	184	180	180
query16	7133	260	256	256
query17	1032	549	548	548
query18	1915	295	269	269
query19	200	153	150	150
query20	93	104	85	85
query21	194	132	127	127
query22	5045	4846	4818	4818
query23	34474	33661	33429	33429
query24	5028	3018	2884	2884
query25	468	402	370	370
query26	692	155	155	155
query27	1845	319	332	319
query28	3844	2043	2044	2043
query29	843	610	592	592
query30	251	183	178	178
query31	920	769	739	739
query32	59	51	60	51
query33	379	249	241	241
query34	865	462	479	462
query35	770	721	665	665
query36	1043	950	899	899
query37	112	71	74	71
query38	2866	2777	2777	2777
query39	1620	1564	1568	1564
query40	198	134	127	127
query41	57	45	48	45
query42	106	95	101	95
query43	601	568	546	546
query44	1063	725	735	725
query45	269	257	244	244
query46	1052	729	734	729
query47	1938	1905	1895	1895
query48	375	309	297	297
query49	780	401	396	396
query50	768	393	375	375
query51	6880	6823	6782	6782
query52	108	90	88	88
query53	354	280	295	280
query54	558	429	432	429
query55	73	70	74	70
query56	235	225	225	225
query57	1231	1138	1147	1138
query58	226	200	200	200
query59	3732	3134	3396	3134
query60	250	236	235	235
query61	93	92	87	87
query62	577	460	483	460
query63	308	283	282	282
query64	2614	1840	1806	1806
query65	3162	3128	3125	3125
query66	688	355	350	350
query67	15248	14660	14844	14660
query68	4550	549	545	545
query69	483	307	310	307
query70	1173	1113	1147	1113
query71	430	274	273	273
query72	7234	2704	2515	2515
query73	723	317	318	317
query74	6647	6257	6204	6204
query75	3342	2644	2605	2605
query76	2334	1053	908	908
query77	449	273	277	273
query78	10735	10249	10287	10249
query79	2301	520	522	520
query80	873	446	445	445
query81	533	240	247	240
query82	809	101	102	101
query83	255	173	177	173
query84	267	93	88	88
query85	1151	326	313	313
query86	464	312	304	304
query87	3312	3129	3176	3129
query88	4067	2335	2333	2333
query89	477	383	387	383
query90	2073	196	191	191
query91	136	107	110	107
query92	66	51	50	50
query93	2045	511	500	500
query94	1133	188	188	188
query95	531	307	335	307
query96	592	263	268	263
query97	3203	2964	2972	2964
query98	243	228	219	219
query99	1225	898	885	885
Total cold run time: 261342 ms
Total hot run time: 181914 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e9d6017c9dbf124d3af8c1cb73e23ad03eae6a3a, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.08
query5	0.50	0.48	0.50
query6	1.12	0.73	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.53	0.50	0.49
query10	0.55	0.57	0.55
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.60	0.59	0.59
query14	0.76	0.77	0.78
query15	0.82	0.81	0.81
query16	0.36	0.37	0.36
query17	0.95	1.04	0.98
query18	0.22	0.25	0.24
query19	1.84	1.72	1.71
query20	0.01	0.01	0.01
query21	15.49	0.68	0.66
query22	4.09	7.55	1.85
query23	18.25	1.38	1.34
query24	1.67	0.28	0.21
query25	0.15	0.09	0.09
query26	0.26	0.17	0.16
query27	0.08	0.08	0.08
query28	13.34	1.02	1.00
query29	12.74	3.30	3.26
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.29	0.47	0.48
query33	2.85	2.94	2.90
query34	17.03	4.43	4.41
query35	4.53	4.53	4.54
query36	0.68	0.49	0.46
query37	0.18	0.16	0.16
query38	0.17	0.14	0.14
query39	0.04	0.04	0.04
query40	0.16	0.14	0.14
query41	0.09	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.93 s
Total hot run time: 30.66 s

@englefly englefly marked this pull request as draft May 22, 2024 06:31
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@englefly
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40308 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8c1ead2da8d951d6396d7c5af18194600ae996f3, data reload: false

------ Round 1 ----------------------------------
q1	17596	4919	4265	4265
q2	2018	188	204	188
q3	10551	1233	1199	1199
q4	10394	838	754	754
q5	7557	2753	2618	2618
q6	222	138	151	138
q7	973	618	631	618
q8	9474	2139	2092	2092
q9	9103	6646	6622	6622
q10	9430	3762	3714	3714
q11	461	252	245	245
q12	483	231	225	225
q13	18829	3012	3041	3012
q14	252	228	229	228
q15	503	472	472	472
q16	547	384	387	384
q17	995	622	749	622
q18	8160	7504	7486	7486
q19	4501	1572	1458	1458
q20	667	317	315	315
q21	5028	3899	3372	3372
q22	348	281	282	281
Total cold run time: 118092 ms
Total hot run time: 40308 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4364	4254	4229	4229
q2	374	263	272	263
q3	2970	2800	2692	2692
q4	1884	1565	1557	1557
q5	5288	5301	5296	5296
q6	215	128	130	128
q7	2168	1691	1783	1691
q8	3222	3322	3329	3322
q9	8400	8312	8328	8312
q10	3873	3699	3684	3684
q11	587	497	504	497
q12	763	599	628	599
q13	17179	3040	3036	3036
q14	281	253	270	253
q15	521	475	482	475
q16	476	422	412	412
q17	1772	1501	1479	1479
q18	7882	7538	7510	7510
q19	1661	1562	1581	1562
q20	1984	1771	1801	1771
q21	4888	4815	4728	4728
q22	571	508	478	478
Total cold run time: 71323 ms
Total hot run time: 53974 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171080 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c1ead2da8d951d6396d7c5af18194600ae996f3, data reload: false

query1	920	387	375	375
query2	6450	2487	2351	2351
query3	6649	211	205	205
query4	20028	17376	17353	17353
query5	4176	423	426	423
query6	252	169	170	169
query7	4601	300	291	291
query8	248	203	206	203
query9	8363	2393	2372	2372
query10	454	296	272	272
query11	10598	10257	10109	10109
query12	139	89	91	89
query13	1645	368	368	368
query14	8528	7307	7755	7307
query15	262	168	170	168
query16	8176	268	266	266
query17	1828	550	539	539
query18	2118	281	277	277
query19	283	165	163	163
query20	93	88	84	84
query21	197	131	132	131
query22	4384	4157	3898	3898
query23	33770	33087	34008	33087
query24	11733	2886	2830	2830
query25	547	351	362	351
query26	1269	151	150	150
query27	2901	293	295	293
query28	7517	2037	2049	2037
query29	858	603	631	603
query30	319	173	176	173
query31	968	751	748	748
query32	87	53	56	53
query33	773	273	259	259
query34	1054	473	481	473
query35	699	569	553	553
query36	1071	928	910	910
query37	212	80	80	80
query38	2910	2793	2802	2793
query39	849	791	792	791
query40	284	121	118	118
query41	48	44	41	41
query42	101	95	100	95
query43	588	564	553	553
query44	1207	738	767	738
query45	184	164	162	162
query46	1074	692	734	692
query47	1841	1772	1738	1738
query48	371	297	290	290
query49	1178	388	390	388
query50	786	416	399	399
query51	6983	6832	6704	6704
query52	110	100	89	89
query53	351	292	284	284
query54	921	434	422	422
query55	75	72	73	72
query56	260	242	242	242
query57	1108	1053	1065	1053
query58	238	209	211	209
query59	3386	3323	3193	3193
query60	301	256	250	250
query61	94	87	90	87
query62	844	684	653	653
query63	311	281	284	281
query64	9751	2234	1807	1807
query65	3197	3108	3134	3108
query66	1274	334	325	325
query67	15245	15041	14835	14835
query68	6406	529	539	529
query69	503	269	276	269
query70	1223	1157	1135	1135
query71	510	269	270	269
query72	7765	5666	2737	2737
query73	821	318	320	318
query74	6130	5612	5664	5612
query75	4024	2599	2632	2599
query76	4521	994	961	961
query77	677	270	272	270
query78	10392	9901	10038	9901
query79	3029	512	520	512
query80	1276	503	434	434
query81	524	246	245	245
query82	763	206	203	203
query83	215	175	171	171
query84	265	86	84	84
query85	1436	286	263	263
query86	463	310	295	295
query87	3357	3094	3079	3079
query88	4393	2315	2328	2315
query89	501	401	388	388
query90	2015	195	192	192
query91	125	95	97	95
query92	62	51	48	48
query93	4047	511	501	501
query94	1135	237	186	186
query95	419	320	317	317
query96	610	266	267	266
query97	3191	2999	2967	2967
query98	239	218	212	212
query99	1663	1283	1282	1282
Total cold run time: 282740 ms
Total hot run time: 171080 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8c1ead2da8d951d6396d7c5af18194600ae996f3, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.06
query4	1.65	0.07	0.08
query5	0.50	0.48	0.50
query6	1.12	0.72	0.71
query7	0.03	0.01	0.01
query8	0.04	0.04	0.04
query9	0.54	0.49	0.49
query10	0.55	0.55	0.53
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.77	0.78
query15	0.83	0.82	0.80
query16	0.37	0.34	0.37
query17	0.96	0.95	0.95
query18	0.21	0.25	0.25
query19	1.82	1.69	1.63
query20	0.02	0.01	0.01
query21	15.74	0.66	0.65
query22	4.00	8.03	1.83
query23	18.25	1.38	1.27
query24	1.64	0.26	0.22
query25	0.14	0.09	0.08
query26	0.25	0.17	0.17
query27	0.08	0.08	0.07
query28	13.48	1.02	1.02
query29	13.05	3.32	3.31
query30	0.24	0.06	0.06
query31	2.84	0.40	0.38
query32	3.27	0.47	0.46
query33	2.92	2.87	2.91
query34	17.26	4.40	4.48
query35	4.52	4.56	4.47
query36	0.65	0.45	0.47
query37	0.17	0.15	0.15
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.16	0.14	0.14
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.74 s
Total hot run time: 30.31 s

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41660 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0736491208510c40800cb07c07a7c2395bfe67bc, data reload: false

------ Round 1 ----------------------------------
q1	17617	4383	4177	4177
q2	2009	190	185	185
q3	10449	1277	1244	1244
q4	10185	918	790	790
q5	7463	2724	2696	2696
q6	216	134	135	134
q7	992	629	627	627
q8	9222	2147	2095	2095
q9	9357	6670	6630	6630
q10	9448	3901	3826	3826
q11	520	235	226	226
q12	415	226	219	219
q13	18198	3283	3222	3222
q14	280	219	240	219
q15	517	483	471	471
q16	499	389	375	375
q17	962	685	696	685
q18	8339	7957	7748	7748
q19	2066	1544	1498	1498
q20	649	298	317	298
q21	5117	4016	4125	4016
q22	347	285	279	279
Total cold run time: 114867 ms
Total hot run time: 41660 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4550	4358	4391	4358
q2	385	270	265	265
q3	3146	2865	2953	2865
q4	2010	1638	1641	1638
q5	5378	5471	5499	5471
q6	217	128	124	124
q7	2192	1841	1817	1817
q8	3205	3393	3355	3355
q9	8680	8632	8619	8619
q10	4043	3728	3830	3728
q11	586	513	491	491
q12	806	613	630	613
q13	16858	3113	3139	3113
q14	303	276	281	276
q15	529	495	482	482
q16	499	435	434	434
q17	1792	1512	1528	1512
q18	7759	7599	7546	7546
q19	1682	1591	1553	1553
q20	2034	1772	1795	1772
q21	4815	4715	4711	4711
q22	589	500	485	485
Total cold run time: 72058 ms
Total hot run time: 55228 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171056 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0736491208510c40800cb07c07a7c2395bfe67bc, data reload: false

query1	922	392	371	371
query2	6429	2240	2346	2240
query3	6655	217	209	209
query4	19493	17393	17407	17393
query5	4113	432	414	414
query6	242	155	151	151
query7	4603	294	289	289
query8	237	191	197	191
query9	8615	2375	2365	2365
query10	460	269	288	269
query11	10393	10198	10051	10051
query12	136	98	87	87
query13	1655	372	368	368
query14	9403	5929	6860	5929
query15	211	171	168	168
query16	7131	286	258	258
query17	1062	528	525	525
query18	1905	272	273	272
query19	202	152	150	150
query20	92	86	84	84
query21	193	127	131	127
query22	4261	3937	3850	3850
query23	33547	33152	32850	32850
query24	6984	2840	2816	2816
query25	566	357	362	357
query26	704	152	146	146
query27	2024	291	288	288
query28	3926	2033	2037	2033
query29	864	604	586	586
query30	246	168	175	168
query31	945	767	740	740
query32	91	87	53	53
query33	501	266	260	260
query34	852	472	469	469
query35	674	589	563	563
query36	1068	928	932	928
query37	116	75	79	75
query38	2865	2776	2744	2744
query39	872	778	778	778
query40	194	120	122	120
query41	46	44	44	44
query42	101	98	98	98
query43	586	550	549	549
query44	1052	721	735	721
query45	184	168	164	164
query46	1060	721	708	708
query47	1829	1748	1773	1748
query48	356	287	289	287
query49	765	381	416	381
query50	770	392	400	392
query51	6898	6713	6759	6713
query52	100	97	91	91
query53	359	287	284	284
query54	540	448	426	426
query55	76	74	73	73
query56	261	251	248	248
query57	1087	1057	1062	1057
query58	230	217	228	217
query59	3380	3157	3045	3045
query60	274	259	250	250
query61	88	90	91	90
query62	770	669	638	638
query63	307	277	280	277
query64	8564	2216	1693	1693
query65	3170	3102	3106	3102
query66	906	333	334	333
query67	15018	14992	14676	14676
query68	4541	538	533	533
query69	447	283	269	269
query70	1177	1144	1178	1144
query71	381	274	272	272
query72	7343	5278	5269	5269
query73	718	319	319	319
query74	5956	5616	5567	5567
query75	3316	2610	2630	2610
query76	2153	944	1016	944
query77	399	270	272	270
query78	10064	9809	9722	9722
query79	1964	513	507	507
query80	989	441	442	441
query81	517	242	243	242
query82	959	197	196	196
query83	262	168	171	168
query84	260	83	88	83
query85	1036	276	267	267
query86	481	314	309	309
query87	3375	3123	3091	3091
query88	3326	2351	2344	2344
query89	472	391	373	373
query90	1995	190	192	190
query91	127	98	93	93
query92	59	48	49	48
query93	1963	505	489	489
query94	1140	185	184	184
query95	399	312	314	312
query96	585	268	265	265
query97	3119	3015	3006	3006
query98	262	220	228	220
query99	1528	1275	1269	1269
Total cold run time: 255542 ms
Total hot run time: 171056 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0736491208510c40800cb07c07a7c2395bfe67bc, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.24	0.06	0.07
query4	1.65	0.10	0.09
query5	0.49	0.51	0.50
query6	1.11	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.51	0.50
query10	0.54	0.55	0.54
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.81	0.77	0.77
query15	0.84	0.82	0.82
query16	0.37	0.37	0.35
query17	1.02	0.94	0.98
query18	0.96	0.94	0.94
query19	1.78	1.71	1.68
query20	0.01	0.01	0.01
query21	15.51	0.69	0.68
query22	3.50	7.70	2.03
query23	18.29	1.30	1.22
query24	1.79	0.26	0.20
query25	0.14	0.08	0.09
query26	0.25	0.16	0.16
query27	0.08	0.07	0.07
query28	13.32	1.01	0.99
query29	13.27	3.32	3.24
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.30	0.46	0.47
query33	2.94	2.86	2.93
query34	17.10	4.46	4.41
query35	4.49	4.48	4.63
query36	0.65	0.46	0.46
query37	0.17	0.15	0.16
query38	0.15	0.14	0.15
query39	0.04	0.03	0.04
query40	0.15	0.14	0.15
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.88 s
Total hot run time: 31.22 s

@doris-robot
Copy link

TPC-DS: Total hot run time: 172443 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e7c599b279f59d903d29985df54cd44ea6bc3d2, data reload: false

query1	905	376	365	365
query2	6456	2400	2287	2287
query3	6658	211	213	211
query4	19070	17470	17158	17158
query5	4186	469	464	464
query6	273	172	172	172
query7	4600	283	279	279
query8	316	284	278	278
query9	8565	2358	2351	2351
query10	625	302	276	276
query11	10512	10009	10043	10009
query12	136	86	83	83
query13	1641	363	361	361
query14	9760	6835	7289	6835
query15	226	186	197	186
query16	7737	264	267	264
query17	1907	537	524	524
query18	1903	267	269	267
query19	189	149	166	149
query20	87	80	80	80
query21	208	127	124	124
query22	4292	4115	4029	4029
query23	33569	33028	33048	33028
query24	12067	2827	2845	2827
query25	651	358	360	358
query26	1797	140	141	140
query27	2980	262	264	262
query28	7629	2041	2027	2027
query29	1082	616	604	604
query30	282	150	151	150
query31	961	727	741	727
query32	95	52	77	52
query33	767	283	280	280
query34	990	464	458	458
query35	703	574	554	554
query36	1090	957	922	922
query37	277	81	78	78
query38	2914	2746	2731	2731
query39	855	783	808	783
query40	281	119	117	117
query41	54	52	55	52
query42	112	98	98	98
query43	609	560	550	550
query44	1243	726	713	713
query45	194	163	157	157
query46	1084	718	710	710
query47	1856	1789	1758	1758
query48	378	295	297	295
query49	1236	405	399	399
query50	780	383	383	383
query51	6809	6789	6741	6741
query52	104	92	92	92
query53	355	283	286	283
query54	1028	446	439	439
query55	75	74	73	73
query56	278	265	273	265
query57	1157	1059	1053	1053
query58	244	237	249	237
query59	3489	3143	3229	3143
query60	305	279	276	276
query61	94	123	90	90
query62	839	646	650	646
query63	315	284	282	282
query64	9902	2228	1718	1718
query65	3171	3107	3057	3057
query66	1377	327	329	327
query67	15535	14964	14824	14824
query68	4493	552	539	539
query69	474	310	311	310
query70	1202	1149	1070	1070
query71	412	319	273	273
query72	7026	5421	5721	5421
query73	740	326	322	322
query74	6002	5468	5460	5460
query75	3452	2682	2658	2658
query76	2926	942	884	884
query77	476	297	290	290
query78	10271	9911	9761	9761
query79	2145	505	495	495
query80	882	464	471	464
query81	591	217	224	217
query82	722	135	130	130
query83	267	167	165	165
query84	241	88	90	88
query85	1832	290	269	269
query86	477	342	309	309
query87	3280	3071	3030	3030
query88	4206	2358	2407	2358
query89	472	377	374	374
query90	1852	188	187	187
query91	128	97	98	97
query92	62	48	49	48
query93	2262	492	489	489
query94	1244	185	188	185
query95	407	315	314	314
query96	604	261	270	261
query97	3163	3030	3037	3030
query98	214	198	192	192
query99	1559	1242	1266	1242
Total cold run time: 275949 ms
Total hot run time: 172443 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e7c599b279f59d903d29985df54cd44ea6bc3d2, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.06	0.07
query5	0.48	0.48	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.50	0.48
query10	0.54	0.54	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.80	0.78	0.80
query15	0.85	0.81	0.82
query16	0.37	0.35	0.37
query17	1.03	0.97	0.97
query18	0.21	0.21	0.19
query19	1.78	1.79	1.79
query20	0.01	0.01	0.01
query21	15.45	0.78	0.67
query22	4.35	6.89	1.84
query23	18.32	1.36	1.35
query24	2.06	0.22	0.22
query25	0.16	0.08	0.09
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.33	1.02	0.99
query29	12.58	3.32	3.26
query30	0.25	0.06	0.05
query31	2.87	0.38	0.38
query32	3.31	0.48	0.46
query33	2.87	2.90	2.92
query34	17.08	4.39	4.38
query35	4.48	4.37	4.48
query36	0.65	0.47	0.47
query37	0.19	0.16	0.15
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.14
query41	0.10	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.59 s
Total hot run time: 30.42 s

@englefly
Copy link
Contributor Author

englefly commented Jul 3, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40197 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 619f6fba56538d562be1f95f2280de03b8da1ce3, data reload: false

------ Round 1 ----------------------------------
q1	18195	4433	4397	4397
q2	2553	199	197	197
q3	12758	1184	1132	1132
q4	10538	824	793	793
q5	7871	2691	2622	2622
q6	224	139	141	139
q7	960	620	604	604
q8	9231	2060	2068	2060
q9	8924	6453	6445	6445
q10	8897	3736	3732	3732
q11	443	243	240	240
q12	399	243	245	243
q13	17839	2968	2972	2968
q14	257	230	220	220
q15	525	483	488	483
q16	506	379	374	374
q17	964	631	626	626
q18	7990	7530	7522	7522
q19	4857	1474	1465	1465
q20	669	328	319	319
q21	4936	3902	3284	3284
q22	401	340	332	332
Total cold run time: 119937 ms
Total hot run time: 40197 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4404	4297	4257	4257
q2	374	273	266	266
q3	3022	2781	2704	2704
q4	1877	1628	1639	1628
q5	5246	5279	5250	5250
q6	222	136	139	136
q7	2109	1714	1728	1714
q8	3227	3334	3312	3312
q9	8279	8292	8307	8292
q10	3893	3665	3684	3665
q11	592	480	497	480
q12	782	631	607	607
q13	17610	2983	2991	2983
q14	294	263	271	263
q15	527	486	480	480
q16	464	426	415	415
q17	1766	1517	1491	1491
q18	7609	7609	7368	7368
q19	1721	1631	1507	1507
q20	1983	1800	1776	1776
q21	5001	4706	4729	4706
q22	612	534	549	534
Total cold run time: 71614 ms
Total hot run time: 53834 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172458 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 619f6fba56538d562be1f95f2280de03b8da1ce3, data reload: false

query1	923	381	374	374
query2	6463	2454	2316	2316
query3	6662	210	214	210
query4	19308	17547	17309	17309
query5	4185	484	498	484
query6	258	189	166	166
query7	4599	303	299	299
query8	317	308	316	308
query9	8448	2387	2368	2368
query10	602	308	281	281
query11	10413	9929	9828	9828
query12	133	86	88	86
query13	1653	376	372	372
query14	9527	7000	7660	7000
query15	244	187	188	187
query16	7796	285	272	272
query17	1871	575	550	550
query18	1928	286	286	286
query19	205	157	162	157
query20	92	82	83	82
query21	220	137	130	130
query22	4330	4282	4134	4134
query23	33617	32957	32940	32940
query24	11882	2892	2767	2767
query25	695	382	392	382
query26	1839	154	155	154
query27	2882	267	278	267
query28	7813	2093	2075	2075
query29	1171	645	613	613
query30	277	156	155	155
query31	957	734	752	734
query32	94	56	55	55
query33	797	307	309	307
query34	909	470	474	470
query35	712	590	593	590
query36	1089	971	971	971
query37	288	80	84	80
query38	2886	2721	2753	2721
query39	912	791	804	791
query40	286	126	121	121
query41	66	56	59	56
query42	122	101	108	101
query43	619	556	564	556
query44	1208	739	741	739
query45	191	170	162	162
query46	1078	780	762	762
query47	1858	1805	1808	1805
query48	372	305	311	305
query49	1224	439	428	428
query50	786	402	412	402
query51	6959	6797	6852	6797
query52	108	94	94	94
query53	368	300	310	300
query54	991	451	453	451
query55	73	78	77	77
query56	304	285	297	285
query57	1149	1065	1054	1054
query58	265	254	372	254
query59	3309	3295	3215	3215
query60	304	292	275	275
query61	95	90	89	89
query62	823	670	655	655
query63	326	298	285	285
query64	9871	2263	1759	1759
query65	3140	3071	3099	3071
query66	1397	355	336	336
query67	15719	14927	15021	14927
query68	7737	566	574	566
query69	698	495	353	353
query70	1187	1121	1051	1051
query71	517	284	274	274
query72	9145	5254	3793	3793
query73	813	336	329	329
query74	5842	5545	5523	5523
query75	4995	2704	2638	2638
query76	5005	994	978	978
query77	765	301	317	301
query78	11280	10211	9800	9800
query79	8727	522	545	522
query80	914	519	483	483
query81	546	216	228	216
query82	639	131	134	131
query83	335	208	171	171
query84	266	85	88	85
query85	713	285	277	277
query86	417	342	324	324
query87	3311	3142	3109	3109
query88	4586	2499	2464	2464
query89	526	397	374	374
query90	2078	206	191	191
query91	133	100	104	100
query92	65	50	49	49
query93	5650	517	526	517
query94	1130	191	191	191
query95	423	320	329	320
query96	608	270	270	270
query97	3199	2977	3015	2977
query98	219	197	191	191
query99	1584	1281	1258	1258
Total cold run time: 296061 ms
Total hot run time: 172458 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 619f6fba56538d562be1f95f2280de03b8da1ce3, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.21	0.06	0.06
query4	1.66	0.10	0.09
query5	0.50	0.49	0.48
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.54	0.48	0.47
query10	0.55	0.54	0.53
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	0.78	0.78	0.77
query15	0.83	0.80	0.81
query16	0.37	0.37	0.37
query17	0.97	1.05	1.00
query18	0.22	0.21	0.20
query19	1.89	1.84	1.76
query20	0.01	0.01	0.01
query21	15.43	0.76	0.66
query22	4.77	7.07	1.97
query23	18.34	1.39	1.28
query24	2.09	0.24	0.22
query25	0.17	0.09	0.09
query26	0.27	0.17	0.18
query27	0.08	0.08	0.08
query28	13.24	1.02	1.00
query29	12.55	3.28	3.24
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.29	0.48	0.47
query33	2.87	2.87	2.89
query34	17.13	4.42	4.39
query35	4.48	4.51	4.52
query36	0.65	0.48	0.49
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.18	0.15	0.15
query41	0.09	0.04	0.05
query42	0.04	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.95 s
Total hot run time: 30.66 s

@englefly
Copy link
Contributor Author

englefly commented Jul 3, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39942 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 774a94b676576d7a81f265c613319dea5f758a9a, data reload: false

------ Round 1 ----------------------------------
q1	17838	6359	4309	4309
q2	2023	194	192	192
q3	10625	1213	1070	1070
q4	10562	710	900	710
q5	7573	2664	2679	2664
q6	230	145	145	145
q7	955	604	625	604
q8	9640	2106	2091	2091
q9	8964	6524	6477	6477
q10	8955	3762	3721	3721
q11	449	241	236	236
q12	468	244	236	236
q13	17772	3040	3026	3026
q14	262	236	222	222
q15	519	476	507	476
q16	510	369	378	369
q17	971	655	704	655
q18	8003	7437	7368	7368
q19	7235	1573	1462	1462
q20	663	326	342	326
q21	4903	3245	3341	3245
q22	392	347	338	338
Total cold run time: 119512 ms
Total hot run time: 39942 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4412	4281	4251	4251
q2	363	266	265	265
q3	2998	2665	2717	2665
q4	1849	1547	1657	1547
q5	5445	5287	5254	5254
q6	246	133	135	133
q7	2099	1727	1720	1720
q8	3203	3325	3302	3302
q9	8362	8338	8329	8329
q10	3900	3665	3690	3665
q11	565	490	484	484
q12	796	634	626	626
q13	17578	3011	3014	3011
q14	293	253	263	253
q15	513	491	477	477
q16	473	409	420	409
q17	1779	1482	1462	1462
q18	7625	7641	7308	7308
q19	1698	1523	1612	1523
q20	1995	1788	1781	1781
q21	4865	4707	4795	4707
q22	621	553	539	539
Total cold run time: 71678 ms
Total hot run time: 53711 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170725 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 774a94b676576d7a81f265c613319dea5f758a9a, data reload: false

query1	942	375	383	375
query2	6470	2299	2219	2219
query3	6660	205	211	205
query4	19780	17431	17154	17154
query5	4164	484	485	484
query6	271	175	164	164
query7	4584	289	285	285
query8	312	317	314	314
query9	8628	2364	2355	2355
query10	617	305	286	286
query11	10320	10005	9925	9925
query12	129	89	85	85
query13	1656	378	365	365
query14	10197	7355	7761	7355
query15	238	192	195	192
query16	7889	264	268	264
query17	1909	559	536	536
query18	1959	279	267	267
query19	199	162	153	153
query20	91	80	92	80
query21	223	143	125	125
query22	4303	4021	4266	4021
query23	33569	33199	32971	32971
query24	11943	2787	2799	2787
query25	680	368	362	362
query26	1777	150	148	148
query27	2902	264	271	264
query28	7741	2061	2053	2053
query29	1141	641	611	611
query30	285	152	156	152
query31	959	726	725	725
query32	95	50	52	50
query33	770	286	293	286
query34	951	467	466	466
query35	694	596	561	561
query36	1079	940	930	930
query37	195	78	80	78
query38	2862	2732	2761	2732
query39	843	809	799	799
query40	282	120	125	120
query41	56	56	59	56
query42	123	102	105	102
query43	587	558	548	548
query44	1269	734	736	734
query45	184	159	163	159
query46	1069	713	702	702
query47	1857	1760	1755	1755
query48	367	311	290	290
query49	1186	409	411	409
query50	784	403	399	399
query51	6892	6867	6745	6745
query52	110	88	95	88
query53	361	304	288	288
query54	1031	451	445	445
query55	76	73	74	73
query56	297	268	264	264
query57	1144	1052	1063	1052
query58	263	252	258	252
query59	3502	3480	3213	3213
query60	319	315	284	284
query61	98	96	91	91
query62	807	635	661	635
query63	321	292	295	292
query64	9856	2262	1778	1778
query65	3174	3101	3102	3101
query66	1387	334	330	330
query67	15535	14877	14919	14877
query68	4630	542	544	542
query69	460	322	317	317
query70	1163	1160	1135	1135
query71	383	274	269	269
query72	7063	2813	2641	2641
query73	736	324	329	324
query74	5907	5550	5530	5530
query75	3376	2683	2671	2671
query76	2572	982	954	954
query77	462	305	309	305
query78	10219	9768	9629	9629
query79	2215	511	508	508
query80	1110	481	469	469
query81	549	220	224	220
query82	723	141	141	141
query83	252	171	175	171
query84	245	85	85	85
query85	1957	344	267	267
query86	492	340	317	317
query87	3294	3051	3109	3051
query88	3877	2463	2449	2449
query89	472	378	377	377
query90	1847	190	192	190
query91	129	99	99	99
query92	69	48	48	48
query93	2418	519	510	510
query94	1281	188	191	188
query95	409	315	315	315
query96	600	279	277	277
query97	3154	3010	3011	3010
query98	213	198	200	198
query99	1696	1252	1272	1252
Total cold run time: 276937 ms
Total hot run time: 170725 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 774a94b676576d7a81f265c613319dea5f758a9a, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.03	0.04
query3	0.22	0.04	0.04
query4	1.69	0.06	0.06
query5	0.50	0.49	0.49
query6	1.13	0.71	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.05
query9	0.55	0.48	0.49
query10	0.55	0.54	0.54
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.58	0.59	0.60
query14	0.78	0.76	0.77
query15	0.85	0.80	0.81
query16	0.36	0.37	0.37
query17	1.03	1.03	1.05
query18	0.22	0.20	0.21
query19	1.79	1.71	1.70
query20	0.01	0.00	0.01
query21	15.46	0.77	0.66
query22	4.23	7.13	1.56
query23	18.32	1.48	1.27
query24	2.08	0.23	0.22
query25	0.16	0.09	0.09
query26	0.26	0.17	0.18
query27	0.09	0.08	0.08
query28	13.33	1.02	1.00
query29	12.59	3.23	3.24
query30	0.25	0.06	0.06
query31	2.87	0.38	0.38
query32	3.30	0.48	0.46
query33	2.87	2.91	2.88
query34	17.11	4.42	4.41
query35	4.46	4.44	4.52
query36	0.65	0.47	0.46
query37	0.18	0.16	0.16
query38	0.16	0.14	0.16
query39	0.04	0.03	0.04
query40	0.17	0.15	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.54 s
Total hot run time: 30.1 s

@englefly
Copy link
Contributor Author

englefly commented Jul 3, 2024

run p0

@englefly
Copy link
Contributor Author

englefly commented Jul 3, 2024

run cloud_p1

@englefly
Copy link
Contributor Author

englefly commented Jul 4, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39917 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 368929706d3861fe09489809e48106e1dc87762a, data reload: false

------ Round 1 ----------------------------------
q1	18524	4531	4378	4378
q2	2972	199	194	194
q3	11211	1158	1138	1138
q4	10689	824	722	722
q5	7635	2741	2664	2664
q6	226	140	143	140
q7	959	607	615	607
q8	9254	2027	2070	2027
q9	8842	6482	6436	6436
q10	8983	3753	3719	3719
q11	442	243	244	243
q12	436	256	236	236
q13	18843	2982	2989	2982
q14	260	236	232	232
q15	525	495	486	486
q16	497	377	375	375
q17	962	619	730	619
q18	7977	7452	7400	7400
q19	7423	1456	1491	1456
q20	665	332	329	329
q21	5086	3199	3342	3199
q22	400	335	345	335
Total cold run time: 122811 ms
Total hot run time: 39917 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4273	4263	4263
q2	371	276	262	262
q3	2955	2725	2696	2696
q4	1907	1653	1653	1653
q5	5279	5309	5262	5262
q6	219	134	134	134
q7	2135	1729	1771	1729
q8	3214	3333	3339	3333
q9	8352	8317	8343	8317
q10	3861	3710	3755	3710
q11	573	482	491	482
q12	797	625	621	621
q13	17510	3035	3032	3032
q14	280	262	251	251
q15	528	480	471	471
q16	469	414	427	414
q17	1788	1477	1476	1476
q18	7694	7444	7331	7331
q19	1736	1498	1455	1455
q20	1998	1766	1774	1766
q21	4948	4880	4784	4784
q22	613	565	514	514
Total cold run time: 71612 ms
Total hot run time: 53956 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172310 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 368929706d3861fe09489809e48106e1dc87762a, data reload: false

query1	924	382	369	369
query2	6458	2377	2264	2264
query3	6668	203	210	203
query4	20133	17548	17170	17170
query5	4183	496	485	485
query6	281	188	173	173
query7	4595	289	279	279
query8	305	291	267	267
query9	8740	2466	2446	2446
query10	596	320	281	281
query11	10513	9932	10051	9932
query12	132	89	81	81
query13	1638	363	367	363
query14	10110	6814	7509	6814
query15	235	184	204	184
query16	7774	321	320	320
query17	1554	583	535	535
query18	1950	282	281	281
query19	195	152	160	152
query20	91	84	85	84
query21	225	134	129	129
query22	4188	3998	3856	3856
query23	33638	33065	33111	33065
query24	12193	2827	2780	2780
query25	687	404	405	404
query26	1835	153	149	149
query27	3030	274	272	272
query28	7760	2128	2114	2114
query29	1163	645	626	626
query30	288	157	153	153
query31	972	765	747	747
query32	98	57	57	57
query33	795	306	299	299
query34	971	469	474	469
query35	693	598	563	563
query36	1097	969	930	930
query37	285	84	94	84
query38	2915	2737	2756	2737
query39	843	795	837	795
query40	278	129	123	123
query41	58	54	55	54
query42	122	187	100	100
query43	610	564	529	529
query44	1301	758	739	739
query45	195	163	163	163
query46	1071	749	740	740
query47	1879	1758	1757	1757
query48	369	291	300	291
query49	1185	411	415	411
query50	773	397	408	397
query51	6862	6622	6774	6622
query52	110	91	97	91
query53	359	302	292	292
query54	1023	462	442	442
query55	74	74	76	74
query56	282	274	268	268
query57	1148	1049	1040	1040
query58	250	259	252	252
query59	3342	3416	3225	3225
query60	318	295	311	295
query61	100	111	95	95
query62	839	666	659	659
query63	317	288	290	288
query64	9859	2235	1794	1794
query65	3165	3109	3101	3101
query66	1382	339	327	327
query67	15491	14847	14972	14847
query68	8889	562	553	553
query69	715	433	334	334
query70	1215	1152	1099	1099
query71	498	282	280	280
query72	7163	5357	5472	5357
query73	833	327	328	327
query74	5814	5496	5472	5472
query75	4199	2702	2684	2684
query76	5122	974	925	925
query77	685	308	303	303
query78	9611	10089	8848	8848
query79	8139	513	519	513
query80	1299	482	482	482
query81	573	217	223	217
query82	761	142	135	135
query83	200	173	168	168
query84	275	86	86	86
query85	1464	291	331	291
query86	422	317	320	317
query87	3326	3140	3098	3098
query88	4433	2372	2395	2372
query89	522	383	382	382
query90	2022	193	192	192
query91	128	99	101	99
query92	62	50	50	50
query93	6253	516	524	516
query94	1276	215	217	215
query95	410	323	319	319
query96	616	270	271	270
query97	3189	3089	3006	3006
query98	214	210	204	204
query99	1617	1288	1268	1268
Total cold run time: 295458 ms
Total hot run time: 172310 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.04 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 368929706d3861fe09489809e48106e1dc87762a, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.06	0.07
query5	0.50	0.47	0.49
query6	1.13	0.73	0.74
query7	0.02	0.01	0.02
query8	0.04	0.05	0.05
query9	0.56	0.49	0.50
query10	0.54	0.53	0.53
query11	0.15	0.11	0.11
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.80	0.81	0.77
query15	0.87	0.82	0.81
query16	0.35	0.35	0.35
query17	0.99	1.01	0.96
query18	0.22	0.21	0.20
query19	1.74	1.70	1.74
query20	0.02	0.01	0.01
query21	15.39	0.76	0.65
query22	4.59	7.67	1.41
query23	18.23	1.30	1.35
query24	2.10	0.23	0.24
query25	0.15	0.09	0.08
query26	0.27	0.18	0.17
query27	0.08	0.09	0.08
query28	13.25	1.02	0.99
query29	12.59	3.25	3.24
query30	0.26	0.05	0.05
query31	2.87	0.39	0.39
query32	3.27	0.48	0.48
query33	2.86	2.92	2.95
query34	17.11	4.47	4.45
query35	4.44	4.48	4.57
query36	0.66	0.46	0.47
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.14	0.14
query41	0.09	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.55 s
Total hot run time: 30.04 s

HappenLee
HappenLee approved these changes Jul 8, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@englefly englefly dismissed morrySnow’s stale review July 8, 2024 09:22

code is refactored

@englefly englefly merged commit 3bb5a8d into apache:master Jul 8, 2024
28 of 31 checks passed
@wm1581066 wm1581066 added the need more test Add more test label Jul 8, 2024
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@englefly @dataroaring
[feat](nereids)push Limit to local agg (#34853)
5c3644d 
## Proposed changes
for a pattern:  topn(n)->globalAgg->localAgg
this pr tries to add a filter on global/localAgg which means only the
first n tuples are counted, and others could be ignored.

inorder to obtain this benefit, optimizer will change limit node to topn
node if the limit number is less than topnOptLimitThreshold.



Issue Number: close #xxx

<!--Describe your changes.-->

## Further comments

If this is a relatively large or complex change, kick off the discussion
at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why
you chose the solution you did and what alternatives you considered,
etc...
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xzj7019 xzj7019 xzj7019 left review comments

@HappenLee HappenLee HappenLee approved these changes

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow left review comments

Assignees
No one assigned
Labels
dev/3.0.1-merged meta-change need more test Add more test not-merge/2.1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@englefly @doris-robot @morrySnow @HappenLee @starocean999 @xzj7019 @wm1581066 @dataroaring