Skip to content

[improve](function) add limit check for lpad/rpad function input big value of length#34810

Merged
HappenLee merged 4 commits intoapache:masterfrom
zhangstar333:lpad_rpad
May 20, 2024
Merged

[improve](function) add limit check for lpad/rpad function input big value of length#34810
HappenLee merged 4 commits intoapache:masterfrom
zhangstar333:lpad_rpad

Conversation

@zhangstar333
Copy link
Contributor

if the user input big value of length, maybe cause some memory problem or
exceeded size limit of thrift, so add this check to report error for user firstly.

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40312 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 38d623b11a871eed01a23810d2318b10e69acac4, data reload: false

------ Round 1 ----------------------------------
q1	17898	4401	4193	4193
q2	2033	191	193	191
q3	10533	1223	1183	1183
q4	10528	772	756	756
q5	7685	2723	2657	2657
q6	218	135	133	133
q7	962	548	529	529
q8	9257	2153	2088	2088
q9	9158	6645	6596	6596
q10	9285	3753	3634	3634
q11	483	247	237	237
q12	422	216	221	216
q13	17784	2959	2976	2959
q14	266	215	220	215
q15	508	473	475	473
q16	525	388	384	384
q17	957	692	749	692
q18	8054	7291	7316	7291
q19	3675	1526	1493	1493
q20	652	306	313	306
q21	4979	3807	3957	3807
q22	353	279	279	279
Total cold run time: 116215 ms
Total hot run time: 40312 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4332	4238	4202	4202
q2	385	270	275	270
q3	2952	2797	2719	2719
q4	1841	1566	1598	1566
q5	5268	5267	5262	5262
q6	210	126	125	125
q7	1715	1366	1371	1366
q8	3226	3342	3314	3314
q9	8396	8306	8343	8306
q10	3875	3658	3614	3614
q11	571	488	490	488
q12	736	578	598	578
q13	17387	2927	2983	2927
q14	286	249	251	249
q15	512	481	470	470
q16	464	415	418	415
q17	1737	1491	1464	1464
q18	7643	7499	7422	7422
q19	1706	1509	1548	1509
q20	1948	1739	1833	1739
q21	4917	4840	4876	4840
q22	551	494	477	477
Total cold run time: 70658 ms
Total hot run time: 53322 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8980/25180)
Line Coverage: 27.32% (74243/271758)
Region Coverage: 26.55% (38354/144455)
Branch Coverage: 23.38% (19568/83704)
Coverage Report: http://coverage.selectdb-in.cc/coverage/38d623b11a871eed01a23810d2318b10e69acac4_38d623b11a871eed01a23810d2318b10e69acac4/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 186804 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 38d623b11a871eed01a23810d2318b10e69acac4, data reload: false

query1	902	360	352	352
query2	6443	2374	2350	2350
query3	6686	208	209	208
query4	24653	21240	21374	21240
query5	4212	418	418	418
query6	265	174	175	174
query7	4587	292	283	283
query8	239	187	197	187
query9	8670	2444	2427	2427
query10	422	252	259	252
query11	14729	14139	14052	14052
query12	133	91	92	91
query13	1677	392	381	381
query14	8791	8024	7563	7563
query15	245	160	169	160
query16	8152	262	268	262
query17	1858	543	538	538
query18	2059	283	263	263
query19	202	149	146	146
query20	93	82	83	82
query21	199	123	125	123
query22	5088	4888	4946	4888
query23	34061	33392	33502	33392
query24	11636	2893	2893	2893
query25	651	367	375	367
query26	1778	158	148	148
query27	3025	311	326	311
query28	7322	2060	2065	2060
query29	985	617	594	594
query30	294	149	146	146
query31	972	762	730	730
query32	94	53	54	53
query33	741	258	248	248
query34	1071	474	491	474
query35	817	692	657	657
query36	1064	920	902	902
query37	169	66	69	66
query38	2866	2775	2777	2775
query39	1601	1556	1565	1556
query40	279	123	124	123
query41	44	38	37	37
query42	105	92	103	92
query43	592	539	543	539
query44	1187	719	737	719
query45	267	257	254	254
query46	1071	714	698	698
query47	1982	1898	1912	1898
query48	388	308	312	308
query49	1186	399	405	399
query50	771	403	387	387
query51	7136	6751	6813	6751
query52	103	92	94	92
query53	349	280	286	280
query54	891	436	422	422
query55	75	72	72	72
query56	242	217	223	217
query57	1225	1151	1157	1151
query58	228	214	204	204
query59	3443	3095	3362	3095
query60	257	234	234	234
query61	90	87	86	86
query62	671	462	497	462
query63	304	288	275	275
query64	9763	7470	7347	7347
query65	3121	3067	3076	3067
query66	1371	346	329	329
query67	15509	14946	15034	14946
query68	4624	524	538	524
query69	468	312	307	307
query70	1194	1048	1177	1048
query71	391	277	284	277
query72	7322	2506	2391	2391
query73	699	328	327	327
query74	6511	6166	6139	6139
query75	3380	2616	2567	2567
query76	2885	940	1058	940
query77	408	262	266	262
query78	10604	10190	10017	10017
query79	1877	519	517	517
query80	1071	435	430	430
query81	501	217	220	217
query82	691	91	91	91
query83	192	159	159	159
query84	264	83	83	83
query85	1389	268	272	268
query86	450	307	308	307
query87	3269	3120	3087	3087
query88	4252	2470	2474	2470
query89	480	379	378	378
query90	1999	195	189	189
query91	188	95	94	94
query92	56	51	47	47
query93	1741	513	495	495
query94	1185	177	179	177
query95	382	316	296	296
query96	579	271	268	268
query97	3198	3007	3019	3007
query98	244	221	221	221
query99	1323	908	920	908
Total cold run time: 285919 ms
Total hot run time: 186804 ms

@doris-robot
Copy link

TPC-H: Total hot run time: 40653 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 38d623b11a871eed01a23810d2318b10e69acac4, data reload: false

------ Round 1 ----------------------------------
q1	17606	4236	4211	4211
q2	2015	182	186	182
q3	10587	1320	1267	1267
q4	10434	829	850	829
q5	7516	2710	2624	2624
q6	221	140	141	140
q7	1033	606	592	592
q8	9687	2102	2097	2097
q9	8799	6502	6497	6497
q10	8873	3739	3729	3729
q11	431	232	237	232
q12	424	221	218	218
q13	17809	2971	2967	2967
q14	260	216	232	216
q15	516	473	473	473
q16	501	396	376	376
q17	962	667	682	667
q18	8111	7440	7365	7365
q19	4481	1546	1479	1479
q20	655	308	297	297
q21	5101	3917	3993	3917
q22	342	281	278	278
Total cold run time: 116364 ms
Total hot run time: 40653 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4341	4197	4186	4186
q2	367	273	281	273
q3	2980	2785	2712	2712
q4	1894	1600	1586	1586
q5	5235	5255	5279	5255
q6	212	125	130	125
q7	2228	1871	1838	1838
q8	3180	3347	3333	3333
q9	8356	8298	8272	8272
q10	3862	3728	3632	3632
q11	600	503	495	495
q12	764	624	591	591
q13	17433	2969	3000	2969
q14	303	250	265	250
q15	518	472	480	472
q16	515	434	431	431
q17	1790	1472	1472	1472
q18	7717	7504	7323	7323
q19	1639	1489	1582	1489
q20	1933	1742	1751	1742
q21	4989	4789	4823	4789
q22	563	489	491	489
Total cold run time: 71419 ms
Total hot run time: 53724 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186868 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 38d623b11a871eed01a23810d2318b10e69acac4, data reload: false

query1	926	359	342	342
query2	6462	2431	2206	2206
query3	6651	212	210	210
query4	23194	21197	21090	21090
query5	4145	425	420	420
query6	267	169	167	167
query7	4576	289	299	289
query8	243	186	198	186
query9	8592	2427	2450	2427
query10	437	256	266	256
query11	14856	14144	14301	14144
query12	139	95	92	92
query13	1660	384	377	377
query14	10331	8403	7690	7690
query15	280	167	169	167
query16	8186	265	262	262
query17	1856	551	541	541
query18	2108	278	268	268
query19	203	145	149	145
query20	92	83	87	83
query21	192	131	129	129
query22	5070	4898	4859	4859
query23	34103	33626	33325	33325
query24	11588	2873	2933	2873
query25	644	370	367	367
query26	1726	161	156	156
query27	2934	320	343	320
query28	7675	2083	2056	2056
query29	1001	610	597	597
query30	306	151	156	151
query31	984	760	739	739
query32	97	55	54	54
query33	768	248	249	248
query34	1040	496	506	496
query35	836	687	660	660
query36	1050	881	905	881
query37	182	75	67	67
query38	2905	2765	2763	2763
query39	1646	1581	1552	1552
query40	274	123	124	123
query41	41	38	38	38
query42	107	98	99	98
query43	587	519	558	519
query44	1208	722	736	722
query45	275	256	254	254
query46	1087	753	727	727
query47	1976	1884	1886	1884
query48	363	304	302	302
query49	1187	397	390	390
query50	762	394	398	394
query51	6907	6802	6725	6725
query52	97	94	89	89
query53	347	285	285	285
query54	914	434	439	434
query55	109	75	71	71
query56	235	224	223	223
query57	1252	1124	1156	1124
query58	213	203	200	200
query59	3480	3255	3352	3255
query60	263	245	240	240
query61	93	88	90	88
query62	674	470	487	470
query63	312	283	284	283
query64	9734	7422	7339	7339
query65	3141	3136	3100	3100
query66	1392	338	343	338
query67	15535	15154	14938	14938
query68	4701	535	529	529
query69	513	308	301	301
query70	1146	1138	1091	1091
query71	419	281	270	270
query72	7950	2536	2340	2340
query73	712	331	329	329
query74	6544	6087	6113	6087
query75	3615	2660	2633	2633
query76	3412	1021	957	957
query77	625	267	258	258
query78	10630	10126	10070	10070
query79	1731	513	518	513
query80	758	436	431	431
query81	507	219	219	219
query82	792	92	90	90
query83	263	160	161	160
query84	231	83	85	83
query85	1538	273	262	262
query86	488	291	316	291
query87	3252	3110	3112	3110
query88	4206	2489	2462	2462
query89	473	380	388	380
query90	1975	187	184	184
query91	132	99	104	99
query92	66	54	49	49
query93	1872	510	492	492
query94	1209	190	199	190
query95	388	303	303	303
query96	607	278	278	278
query97	3170	2987	2985	2985
query98	244	226	228	226
query99	1123	888	938	888
Total cold run time: 287808 ms
Total hot run time: 186868 ms

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8984/25190)
Line Coverage: 27.33% (74275/271794)
Region Coverage: 26.56% (38385/144496)
Branch Coverage: 23.38% (19578/83722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d1d3eb1fe765098d21a03819527a46f8f7aaec6b_d1d3eb1fe765098d21a03819527a46f8f7aaec6b/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41791 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1d3eb1fe765098d21a03819527a46f8f7aaec6b, data reload: false

------ Round 1 ----------------------------------
q1	17607	4358	4271	4271
q2	2023	194	188	188
q3	10455	1134	1148	1134
q4	10203	894	778	778
q5	7491	2724	2748	2724
q6	236	133	136	133
q7	1040	592	609	592
q8	9237	2137	2077	2077
q9	9969	6762	6677	6677
q10	9300	3999	3878	3878
q11	435	247	233	233
q12	448	220	247	220
q13	17193	3207	3187	3187
q14	263	223	212	212
q15	511	473	482	473
q16	485	387	403	387
q17	983	706	690	690
q18	8452	8001	7724	7724
q19	6492	1566	1566	1566
q20	645	328	321	321
q21	5143	4118	4046	4046
q22	366	280	280	280
Total cold run time: 118977 ms
Total hot run time: 41791 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4571	4482	4452	4452
q2	378	279	278	278
q3	3197	2920	2810	2810
q4	1885	1596	1651	1596
q5	5497	5546	5503	5503
q6	214	122	124	122
q7	2348	1963	1951	1951
q8	3513	3382	3440	3382
q9	8636	8778	8625	8625
q10	3888	3844	3835	3835
q11	598	476	499	476
q12	791	638	634	634
q13	17038	3146	3203	3146
q14	315	289	266	266
q15	535	461	473	461
q16	483	421	429	421
q17	1780	1494	1466	1466
q18	7595	7618	7392	7392
q19	1685	1568	1545	1545
q20	1970	1803	1756	1756
q21	9485	4809	4850	4809
q22	573	489	486	486
Total cold run time: 76975 ms
Total hot run time: 55412 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187415 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1d3eb1fe765098d21a03819527a46f8f7aaec6b, data reload: false

query1	921	380	375	375
query2	6430	2451	2374	2374
query3	6643	204	218	204
query4	24310	21490	21298	21298
query5	4119	431	431	431
query6	254	170	173	170
query7	4587	299	288	288
query8	241	191	194	191
query9	8440	2403	2384	2384
query10	437	263	264	263
query11	14762	14291	14250	14250
query12	149	99	91	91
query13	1659	394	376	376
query14	9654	7788	7707	7707
query15	247	173	169	169
query16	8024	258	258	258
query17	1770	557	563	557
query18	2096	274	263	263
query19	201	146	148	146
query20	89	82	88	82
query21	191	125	130	125
query22	5050	4857	4849	4849
query23	34151	33544	33546	33544
query24	10455	2929	2896	2896
query25	556	383	368	368
query26	705	164	153	153
query27	2182	324	336	324
query28	5563	2070	2067	2067
query29	881	611	596	596
query30	268	175	173	173
query31	1014	751	749	749
query32	93	57	57	57
query33	636	250	246	246
query34	896	474	514	474
query35	783	680	671	671
query36	1089	896	881	881
query37	108	72	72	72
query38	2906	2820	2721	2721
query39	1580	1530	1558	1530
query40	199	129	124	124
query41	48	44	45	44
query42	108	98	99	98
query43	575	565	514	514
query44	1081	725	744	725
query45	266	246	256	246
query46	1076	713	735	713
query47	1961	1860	1896	1860
query48	380	294	303	294
query49	847	394	406	394
query50	773	382	385	382
query51	6879	6654	6801	6654
query52	100	94	88	88
query53	348	291	282	282
query54	873	441	445	441
query55	77	75	73	73
query56	242	221	247	221
query57	1233	1143	1146	1143
query58	217	194	201	194
query59	3429	3135	3213	3135
query60	285	234	235	234
query61	89	91	86	86
query62	624	460	467	460
query63	308	278	280	278
query64	8434	7440	7456	7440
query65	3130	3068	3083	3068
query66	779	339	343	339
query67	15411	14960	15031	14960
query68	4523	546	550	546
query69	476	317	309	309
query70	1162	1143	1141	1141
query71	367	275	273	273
query72	7563	2578	2337	2337
query73	713	322	326	322
query74	6581	6109	6135	6109
query75	3330	2612	2620	2612
query76	2278	942	928	928
query77	416	270	269	269
query78	10559	10194	10057	10057
query79	2211	521	512	512
query80	1118	443	497	443
query81	533	241	244	241
query82	641	104	103	103
query83	246	167	170	167
query84	245	89	87	87
query85	1821	276	267	267
query86	511	307	298	298
query87	3293	3120	3116	3116
query88	4273	2350	2362	2350
query89	484	374	392	374
query90	2043	195	208	195
query91	128	99	101	99
query92	62	52	51	51
query93	2100	524	518	518
query94	1281	188	194	188
query95	391	310	315	310
query96	599	274	263	263
query97	3225	2989	2975	2975
query98	240	232	222	222
query99	1170	894	870	870
Total cold run time: 278495 ms
Total hot run time: 187415 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.61 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d1d3eb1fe765098d21a03819527a46f8f7aaec6b, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.08	0.07
query5	0.51	0.50	0.51
query6	1.12	0.73	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.53	0.48	0.50
query10	0.54	0.54	0.55
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.79	0.76	0.79
query15	0.84	0.81	0.81
query16	0.34	0.36	0.37
query17	1.01	1.01	0.95
query18	0.22	0.24	0.23
query19	1.79	1.71	1.71
query20	0.02	0.01	0.01
query21	15.73	0.67	0.65
query22	3.89	7.73	2.01
query23	18.28	1.36	1.27
query24	1.56	0.25	0.20
query25	0.13	0.09	0.09
query26	0.26	0.16	0.17
query27	0.08	0.08	0.08
query28	13.58	1.07	1.08
query29	13.17	3.33	3.31
query30	0.24	0.05	0.06
query31	2.87	0.38	0.39
query32	3.30	0.47	0.47
query33	2.84	2.81	2.87
query34	17.08	4.38	4.43
query35	4.57	4.55	4.52
query36	0.64	0.47	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.14
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.03	0.03
Total cold run time: 109.65 s
Total hot run time: 30.61 s

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8984/25190)
Line Coverage: 27.32% (74266/271791)
Region Coverage: 26.56% (38383/144494)
Branch Coverage: 23.38% (19571/83720)
Coverage Report: http://coverage.selectdb-in.cc/coverage/396b712769c06cd4d0115351625396d14994bf5a_396b712769c06cd4d0115351625396d14994bf5a/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40591 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 396b712769c06cd4d0115351625396d14994bf5a, data reload: false

------ Round 1 ----------------------------------
q1	17648	4365	4271	4271
q2	2019	186	187	186
q3	10470	1172	1158	1158
q4	10144	756	781	756
q5	7473	2681	2656	2656
q6	218	132	134	132
q7	1006	594	579	579
q8	9232	2074	2059	2059
q9	9062	6538	6472	6472
q10	8911	3762	3714	3714
q11	447	238	248	238
q12	441	225	213	213
q13	17777	2967	2961	2961
q14	253	216	222	216
q15	517	485	471	471
q16	507	413	384	384
q17	967	675	737	675
q18	8267	7530	7456	7456
q19	4806	1539	1462	1462
q20	629	301	297	297
q21	5101	3977	3961	3961
q22	344	283	274	274
Total cold run time: 116239 ms
Total hot run time: 40591 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4386	4261	4215	4215
q2	374	266	267	266
q3	2973	2792	2748	2748
q4	1872	1593	1579	1579
q5	5309	5270	5309	5270
q6	211	124	126	124
q7	2250	1885	1886	1885
q8	3199	3349	3321	3321
q9	8424	8383	8363	8363
q10	3869	3731	3690	3690
q11	587	489	493	489
q12	746	593	619	593
q13	16432	3022	2961	2961
q14	298	274	273	273
q15	525	474	469	469
q16	480	411	432	411
q17	1781	1513	1475	1475
q18	7816	7440	7353	7353
q19	4577	1615	1539	1539
q20	1956	1738	1782	1738
q21	5022	4872	4849	4849
q22	577	486	503	486
Total cold run time: 73664 ms
Total hot run time: 54097 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187618 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 396b712769c06cd4d0115351625396d14994bf5a, data reload: false

query1	914	400	366	366
query2	6443	2408	2343	2343
query3	6670	206	213	206
query4	23303	21220	21251	21220
query5	4134	414	415	414
query6	269	169	178	169
query7	4582	288	290	288
query8	241	187	187	187
query9	8445	2462	2458	2458
query10	448	256	260	256
query11	14618	14213	14256	14213
query12	139	87	87	87
query13	1646	372	378	372
query14	10622	8109	7658	7658
query15	263	171	174	171
query16	8146	259	276	259
query17	1841	571	544	544
query18	2097	276	265	265
query19	215	145	148	145
query20	92	83	86	83
query21	198	125	127	125
query22	5054	4830	4804	4804
query23	34444	33724	33696	33696
query24	11464	2943	2931	2931
query25	603	374	379	374
query26	1716	156	150	150
query27	2954	328	330	328
query28	7598	2068	2060	2060
query29	1009	638	636	636
query30	328	170	175	170
query31	979	744	748	744
query32	87	54	56	54
query33	748	248	240	240
query34	1082	476	487	476
query35	812	672	668	668
query36	1078	888	910	888
query37	144	72	72	72
query38	2929	2757	2736	2736
query39	1608	1546	1553	1546
query40	278	127	122	122
query41	48	44	46	44
query42	101	99	97	97
query43	560	533	537	533
query44	1223	716	738	716
query45	275	263	261	261
query46	1087	732	713	713
query47	1932	1860	1897	1860
query48	376	295	310	295
query49	1139	408	420	408
query50	770	400	390	390
query51	6770	6731	6578	6578
query52	102	92	90	90
query53	347	279	282	279
query54	1032	444	454	444
query55	72	73	75	73
query56	244	237	228	228
query57	1249	1147	1155	1147
query58	228	206	211	206
query59	3317	3135	3056	3056
query60	271	241	242	241
query61	108	105	104	104
query62	673	481	484	481
query63	313	288	282	282
query64	9832	7368	7436	7368
query65	3145	3104	3107	3104
query66	1408	340	340	340
query67	15487	15145	15324	15145
query68	5559	536	552	536
query69	544	314	307	307
query70	1207	1118	1111	1111
query71	454	266	265	265
query72	7705	2531	2348	2348
query73	724	331	332	331
query74	6446	6128	6077	6077
query75	3907	2643	2644	2643
query76	3701	1000	992	992
query77	656	305	261	261
query78	10592	10111	10059	10059
query79	2128	512	527	512
query80	1329	428	422	422
query81	505	244	242	242
query82	617	101	96	96
query83	197	161	162	161
query84	265	85	91	85
query85	1361	263	260	260
query86	395	319	306	306
query87	3295	3123	3124	3123
query88	3913	2442	2448	2442
query89	477	371	382	371
query90	2026	188	190	188
query91	126	97	97	97
query92	57	49	49	49
query93	4073	510	510	510
query94	1164	183	183	183
query95	453	305	301	301
query96	595	275	267	267
query97	3177	2958	2992	2958
query98	228	223	213	213
query99	1271	895	905	895
Total cold run time: 291497 ms
Total hot run time: 187618 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 396b712769c06cd4d0115351625396d14994bf5a, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.06
query4	1.65	0.08	0.07
query5	0.48	0.47	0.52
query6	1.12	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.54	0.56	0.54
query11	0.19	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.60
query14	0.77	0.76	0.78
query15	0.82	0.80	0.81
query16	0.35	0.37	0.37
query17	0.97	0.99	1.02
query18	0.24	0.26	0.24
query19	1.80	1.76	1.70
query20	0.01	0.01	0.02
query21	15.51	0.69	0.67
query22	4.03	7.06	2.04
query23	18.26	1.33	1.25
query24	1.93	0.23	0.20
query25	0.13	0.09	0.09
query26	0.26	0.16	0.16
query27	0.08	0.07	0.08
query28	13.32	1.00	1.01
query29	12.69	3.26	3.26
query30	0.26	0.06	0.06
query31	2.87	0.38	0.38
query32	3.25	0.46	0.47
query33	2.79	2.86	2.84
query34	17.31	4.40	4.40
query35	4.48	4.48	4.54
query36	0.65	0.46	0.49
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.15	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.17 s
Total hot run time: 30.57 s

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8984/25194)
Line Coverage: 27.32% (74267/271840)
Region Coverage: 26.55% (38379/144537)
Branch Coverage: 23.37% (19572/83752)
Coverage Report: http://coverage.selectdb-in.cc/coverage/30119366ab7f5812ebe671b70216b3ab75279c48_30119366ab7f5812ebe671b70216b3ab75279c48/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41584 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

------ Round 1 ----------------------------------
q1	17618	4313	4216	4216
q2	2029	193	186	186
q3	10471	1241	1171	1171
q4	10185	805	743	743
q5	7461	2751	2632	2632
q6	217	132	131	131
q7	1067	586	589	586
q8	9236	2131	2071	2071
q9	10005	6737	6706	6706
q10	9652	3859	3937	3859
q11	443	236	228	228
q12	454	214	224	214
q13	18141	3178	3139	3139
q14	251	212	218	212
q15	515	484	466	466
q16	510	410	398	398
q17	982	681	758	681
q18	8414	7730	7693	7693
q19	4340	1533	1549	1533
q20	639	324	316	316
q21	5204	4118	4358	4118
q22	371	285	288	285
Total cold run time: 118205 ms
Total hot run time: 41584 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4491	4333	4450	4333
q2	368	282	264	264
q3	3138	2853	2759	2759
q4	1863	1727	1660	1660
q5	5475	5500	5480	5480
q6	216	125	129	125
q7	2338	2011	1958	1958
q8	3294	3414	3446	3414
q9	8609	8667	8410	8410
q10	3998	3841	3880	3841
q11	611	499	522	499
q12	823	624	625	624
q13	16906	3179	3016	3016
q14	279	259	262	259
q15	524	479	470	470
q16	487	427	419	419
q17	1775	1482	1481	1481
q18	7694	7607	7426	7426
q19	1645	1555	1478	1478
q20	1970	1747	1753	1747
q21	10977	4688	4808	4688
q22	559	503	484	484
Total cold run time: 78040 ms
Total hot run time: 54835 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186604 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

query1	930	393	367	367
query2	6442	2347	2279	2279
query3	6639	209	215	209
query4	25286	21235	21100	21100
query5	4162	415	417	415
query6	263	173	183	173
query7	4579	294	282	282
query8	249	193	201	193
query9	8870	2404	2395	2395
query10	478	253	244	244
query11	14722	14151	14210	14151
query12	138	91	92	91
query13	1665	392	358	358
query14	8711	7797	8246	7797
query15	216	168	167	167
query16	7741	264	253	253
query17	1640	544	548	544
query18	1944	276	277	276
query19	197	150	147	147
query20	94	84	86	84
query21	197	134	125	125
query22	5068	4842	4854	4842
query23	34071	33564	33462	33462
query24	6984	2919	2842	2842
query25	581	370	363	363
query26	707	155	153	153
query27	2045	332	333	332
query28	3928	2087	2055	2055
query29	835	623	604	604
query30	239	175	188	175
query31	975	761	741	741
query32	88	54	53	53
query33	467	244	255	244
query34	851	479	486	479
query35	749	669	669	669
query36	1048	894	926	894
query37	106	71	69	69
query38	2868	2759	2802	2759
query39	1607	1554	1670	1554
query40	207	125	127	125
query41	49	44	43	43
query42	105	93	94	93
query43	557	540	535	535
query44	1035	716	728	716
query45	272	260	253	253
query46	1084	706	727	706
query47	1957	1892	1885	1885
query48	368	290	298	290
query49	847	390	386	386
query50	767	383	386	383
query51	6833	6759	6797	6759
query52	100	97	87	87
query53	350	284	276	276
query54	524	428	422	422
query55	71	72	71	71
query56	232	220	220	220
query57	1236	1159	1142	1142
query58	218	215	207	207
query59	3261	3171	3158	3158
query60	271	260	233	233
query61	93	88	88	88
query62	604	502	464	464
query63	304	275	279	275
query64	8438	7344	7388	7344
query65	3154	3066	3084	3066
query66	797	331	340	331
query67	15404	14918	14704	14704
query68	4544	539	535	535
query69	485	301	298	298
query70	1171	1121	1102	1102
query71	375	268	262	262
query72	7085	2590	2336	2336
query73	706	328	322	322
query74	6432	6165	6161	6161
query75	3285	2588	2646	2588
query76	2325	999	1007	999
query77	385	259	257	257
query78	10449	10144	9973	9973
query79	2006	510	513	510
query80	974	483	427	427
query81	515	244	241	241
query82	949	98	94	94
query83	244	161	157	157
query84	243	87	81	81
query85	1015	285	258	258
query86	453	299	312	299
query87	3271	3130	3160	3130
query88	3161	2398	2408	2398
query89	477	379	371	371
query90	1992	182	189	182
query91	125	101	93	93
query92	55	48	47	47
query93	1850	508	494	494
query94	1087	180	179	179
query95	393	294	308	294
query96	599	267	264	264
query97	3139	2960	3018	2960
query98	238	228	206	206
query99	1167	905	884	884
Total cold run time: 268713 ms
Total hot run time: 186604 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.08	0.07
query5	0.50	0.50	0.52
query6	1.12	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.03	0.05
query9	0.54	0.48	0.50
query10	0.52	0.53	0.54
query11	0.15	0.12	0.11
query12	0.15	0.11	0.11
query13	0.59	0.59	0.60
query14	0.78	0.77	0.78
query15	0.82	0.80	0.82
query16	0.36	0.38	0.35
query17	0.98	0.99	1.03
query18	0.22	0.25	0.25
query19	1.88	1.75	1.80
query20	0.02	0.01	0.00
query21	15.72	0.67	0.66
query22	4.27	7.77	1.43
query23	18.29	1.39	1.30
query24	2.03	0.22	0.19
query25	0.14	0.08	0.08
query26	0.26	0.16	0.16
query27	0.08	0.08	0.08
query28	13.24	1.01	1.00
query29	13.21	3.39	3.25
query30	0.23	0.06	0.06
query31	2.88	0.38	0.38
query32	3.28	0.46	0.46
query33	2.78	2.85	2.87
query34	17.17	4.45	4.45
query35	4.48	4.47	4.48
query36	0.69	0.49	0.48
query37	0.18	0.14	0.14
query38	0.15	0.14	0.15
query39	0.04	0.03	0.04
query40	0.16	0.14	0.15
query41	0.10	0.05	0.05
query42	0.05	0.04	0.04
query43	0.04	0.03	0.03
Total cold run time: 110.19 s
Total hot run time: 30.05 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.65% (8983/25195)
Line Coverage: 27.32% (74279/271868)
Region Coverage: 26.56% (38390/144547)
Branch Coverage: 23.37% (19576/83756)
Coverage Report: http://coverage.selectdb-in.cc/coverage/30119366ab7f5812ebe671b70216b3ab75279c48_30119366ab7f5812ebe671b70216b3ab75279c48/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41738 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

------ Round 1 ----------------------------------
q1	17612	4313	4204	4204
q2	2024	188	185	185
q3	10478	1238	1167	1167
q4	10196	811	746	746
q5	7467	2700	2638	2638
q6	228	135	132	132
q7	1014	591	597	591
q8	9228	2145	2083	2083
q9	10355	6760	6648	6648
q10	9277	3979	3898	3898
q11	444	232	251	232
q12	561	222	227	222
q13	17262	3280	3224	3224
q14	265	219	218	218
q15	512	465	463	463
q16	489	414	409	409
q17	966	641	707	641
q18	8312	7899	7811	7811
q19	4908	1558	1522	1522
q20	659	334	330	330
q21	5492	4093	4245	4093
q22	358	281	286	281
Total cold run time: 118107 ms
Total hot run time: 41738 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4524	4364	4360	4360
q2	370	266	264	264
q3	3130	3005	2731	2731
q4	1867	1617	1651	1617
q5	5485	5500	5509	5500
q6	214	126	119	119
q7	2357	1960	2007	1960
q8	3258	3403	3399	3399
q9	8712	8714	8556	8556
q10	3952	3872	3897	3872
q11	584	485	483	483
q12	792	610	656	610
q13	17119	3182	3101	3101
q14	276	265	250	250
q15	509	465	474	465
q16	469	398	423	398
q17	1751	1453	1444	1444
q18	7642	7667	7458	7458
q19	1643	1535	1509	1509
q20	1952	1754	1791	1754
q21	7738	5096	4920	4920
q22	586	509	503	503
Total cold run time: 74930 ms
Total hot run time: 55273 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187109 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

query1	927	385	375	375
query2	6453	2361	2186	2186
query3	6648	211	217	211
query4	22896	21287	21227	21227
query5	4109	417	423	417
query6	286	175	171	171
query7	4591	299	286	286
query8	240	189	206	189
query9	8527	2419	2413	2413
query10	433	247	265	247
query11	14807	14226	14529	14226
query12	144	91	90	90
query13	1642	396	373	373
query14	9201	7094	7815	7094
query15	217	167	174	167
query16	7740	259	272	259
query17	1736	549	537	537
query18	1964	289	291	289
query19	195	155	147	147
query20	99	85	86	85
query21	202	141	133	133
query22	5091	4931	4840	4840
query23	34099	33580	33701	33580
query24	7295	2968	2923	2923
query25	606	366	365	365
query26	716	158	156	156
query27	2172	321	330	321
query28	5520	2085	2067	2067
query29	854	614	618	614
query30	234	187	183	183
query31	977	738	740	738
query32	92	52	60	52
query33	571	250	245	245
query34	895	484	486	484
query35	785	680	681	680
query36	1043	909	947	909
query37	108	72	71	71
query38	2936	2825	2763	2763
query39	1616	1580	1556	1556
query40	192	123	121	121
query41	47	46	48	46
query42	101	95	92	92
query43	556	510	536	510
query44	1098	714	751	714
query45	276	258	249	249
query46	1084	751	697	697
query47	1980	1882	1878	1878
query48	376	293	293	293
query49	834	382	397	382
query50	764	394	392	392
query51	6902	6752	6840	6752
query52	105	89	94	89
query53	345	283	286	283
query54	554	432	430	430
query55	75	72	72	72
query56	239	230	229	229
query57	1224	1155	1171	1155
query58	210	205	209	205
query59	3326	3134	3025	3025
query60	260	239	243	239
query61	92	88	87	87
query62	633	482	453	453
query63	307	277	281	277
query64	8433	7403	7433	7403
query65	3133	3084	3112	3084
query66	772	344	353	344
query67	15351	15104	15169	15104
query68	4608	542	543	542
query69	479	304	314	304
query70	1171	1139	1121	1121
query71	376	257	264	257
query72	7859	2602	2357	2357
query73	709	329	326	326
query74	6542	6209	6276	6209
query75	3286	2679	2605	2605
query76	2284	1012	1059	1012
query77	392	272	264	264
query78	10604	10184	10171	10171
query79	2237	532	516	516
query80	928	440	430	430
query81	537	246	242	242
query82	945	109	100	100
query83	257	173	171	171
query84	246	87	97	87
query85	1232	269	273	269
query86	443	334	302	302
query87	3351	3088	3130	3088
query88	3826	2456	2424	2424
query89	466	369	381	369
query90	1984	195	192	192
query91	123	94	100	94
query92	59	50	49	49
query93	1788	515	499	499
query94	1261	185	184	184
query95	388	295	295	295
query96	581	279	270	270
query97	3163	2974	2986	2974
query98	236	224	221	221
query99	1198	916	891	891
Total cold run time: 271495 ms
Total hot run time: 187109 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 30119366ab7f5812ebe671b70216b3ab75279c48, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.48	0.51
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.05
query9	0.54	0.50	0.49
query10	0.54	0.54	0.56
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.78	0.79
query15	0.82	0.80	0.80
query16	0.37	0.36	0.37
query17	0.96	1.04	0.95
query18	0.21	0.28	0.23
query19	1.89	1.69	1.69
query20	0.01	0.01	0.02
query21	15.55	0.70	0.66
query22	3.56	6.83	2.62
query23	18.36	1.40	1.19
query24	1.73	0.25	0.25
query25	0.15	0.09	0.09
query26	0.26	0.17	0.16
query27	0.09	0.08	0.08
query28	13.32	1.01	1.00
query29	12.86	3.30	3.24
query30	0.25	0.07	0.06
query31	2.85	0.38	0.38
query32	3.29	0.46	0.47
query33	2.81	2.80	2.88
query34	17.26	4.41	4.44
query35	4.51	4.47	4.56
query36	0.66	0.49	0.50
query37	0.17	0.15	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.17	0.13	0.15
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.03
Total cold run time: 109 s
Total hot run time: 31.05 s

@zhangstar333
Copy link
Contributor Author

run performance

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 20, 2024
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit 460eb0f into apache:master May 20, 2024
yiguolei pushed a commit that referenced this pull request May 21, 2024
dataroaring pushed a commit that referenced this pull request May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants