Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement](function) Support Overlay function #33243

Open
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

koarz
Copy link
Contributor

@koarz koarz commented Apr 3, 2024

Proposed changes

Issue Number: close #33094

Add Function Overlay for doris

mysql> select overlay('Quadratic', 3, 4, 'What');
+------------------------------------+
| overlay('Quadratic', 3, 4, 'What') |
+------------------------------------+
| QuWhattic                          |
+------------------------------------+

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@koarz
Copy link
Contributor Author

koarz commented Apr 3, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/functions/function_string.h Show resolved Hide resolved
be/src/vec/functions/function_string.h Show resolved Hide resolved
@koarz koarz changed the title [Enhancement](doris-future) Support insert function [Enhancement](function) Support insert function Apr 3, 2024
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.65% (8887/24928)
Line Coverage: 27.38% (72959/266444)
Region Coverage: 26.56% (37728/142032)
Branch Coverage: 23.36% (19226/82286)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bba22637a4321ac7e998dc86430925909558deda_bba22637a4321ac7e998dc86430925909558deda/report/index.html

@koarz
Copy link
Contributor Author

koarz commented Apr 3, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.64% (8885/24928)
Line Coverage: 27.37% (72937/266444)
Region Coverage: 26.55% (37705/142032)
Branch Coverage: 23.35% (19217/82286)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2345f0fdb8d42fba6a965b7004f4b717d84706f7_2345f0fdb8d42fba6a965b7004f4b717d84706f7/report/index.html

@koarz
Copy link
Contributor Author

koarz commented Apr 3, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39104 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfa86cd4db427e3333668781dc990cff4c0cdbe2, data reload: false

------ Round 1 ----------------------------------
q1	18517	4260	4223	4223
q2	2335	192	193	192
q3	10785	1244	1363	1244
q4	10832	845	971	845
q5	7847	2996	3014	2996
q6	221	138	139	138
q7	1101	640	623	623
q8	9415	2080	2051	2051
q9	6692	6247	6162	6162
q10	8422	3560	3515	3515
q11	411	241	247	241
q12	384	221	212	212
q13	17768	2961	2949	2949
q14	270	236	245	236
q15	534	482	481	481
q16	486	390	369	369
q17	954	935	898	898
q18	7365	6558	6503	6503
q19	1594	1543	1565	1543
q20	558	317	301	301
q21	3553	3162	3075	3075
q22	370	307	321	307
Total cold run time: 110414 ms
Total hot run time: 39104 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4112	4082	4065	4065
q2	327	223	220	220
q3	2986	2974	2983	2974
q4	1892	1868	1848	1848
q5	5272	5246	5241	5241
q6	210	128	128	128
q7	2250	1833	1810	1810
q8	3212	3300	3285	3285
q9	8544	8460	8536	8460
q10	3779	3836	3831	3831
q11	540	464	454	454
q12	715	531	535	531
q13	16770	2916	2951	2916
q14	287	257	254	254
q15	502	468	477	468
q16	451	407	397	397
q17	1709	1678	1667	1667
q18	7654	7271	7232	7232
q19	1671	1645	1653	1645
q20	1965	1718	1722	1718
q21	4997	4715	4769	4715
q22	505	433	441	433
Total cold run time: 70350 ms
Total hot run time: 54292 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.65% (8886/24928)
Line Coverage: 27.38% (72952/266444)
Region Coverage: 26.55% (37714/142032)
Branch Coverage: 23.36% (19224/82286)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bfa86cd4db427e3333668781dc990cff4c0cdbe2_bfa86cd4db427e3333668781dc990cff4c0cdbe2/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 180714 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfa86cd4db427e3333668781dc990cff4c0cdbe2, data reload: false

query1	1216	1121	1117	1117
query2	6503	1727	1767	1727
query3	6667	211	218	211
query4	23382	21517	21509	21509
query5	4229	400	398	398
query6	285	189	193	189
query7	4611	289	297	289
query8	228	184	181	181
query9	8488	2240	2239	2239
query10	588	253	252	252
query11	14883	14602	14422	14422
query12	141	89	87	87
query13	1643	380	375	375
query14	8709	6720	6782	6720
query15	206	175	174	174
query16	7132	268	257	257
query17	1029	593	590	590
query18	1921	292	282	282
query19	200	157	153	153
query20	94	92	89	89
query21	203	134	131	131
query22	4981	4824	4802	4802
query23	33652	32488	32639	32488
query24	11159	3149	3171	3149
query25	698	388	399	388
query26	1909	161	155	155
query27	3027	321	319	319
query28	6717	1854	1832	1832
query29	1387	578	592	578
query30	314	165	164	164
query31	949	716	720	716
query32	99	56	55	55
query33	674	267	249	249
query34	951	468	491	468
query35	833	706	677	677
query36	995	887	862	862
query37	279	73	73	73
query38	3471	3428	3375	3375
query39	1577	1522	1529	1522
query40	286	128	130	128
query41	52	47	48	47
query42	110	100	100	100
query43	456	402	427	402
query44	1129	716	704	704
query45	272	262	276	262
query46	1074	781	752	752
query47	1867	1778	1781	1778
query48	377	299	301	299
query49	1156	368	363	363
query50	799	388	403	388
query51	6676	6653	6604	6604
query52	104	95	94	94
query53	362	290	280	280
query54	289	230	235	230
query55	83	75	74	74
query56	244	224	222	222
query57	1211	1112	1136	1112
query58	255	216	216	216
query59	2670	2450	2625	2450
query60	262	243	250	243
query61	114	112	112	112
query62	714	444	429	429
query63	312	281	283	281
query64	6375	3182	3250	3182
query65	3097	3021	2977	2977
query66	1438	330	324	324
query67	15563	15213	14904	14904
query68	7867	558	578	558
query69	554	309	307	307
query70	1297	1137	1119	1119
query71	504	277	262	262
query72	6369	2598	2427	2427
query73	816	316	318	316
query74	6755	6468	6346	6346
query75	3686	2311	2301	2301
query76	5788	1210	1237	1210
query77	673	247	250	247
query78	11098	10099	10210	10099
query79	8142	525	531	525
query80	1180	422	423	422
query81	509	236	250	236
query82	748	96	95	95
query83	217	172	171	171
query84	272	94	92	92
query85	1457	290	283	283
query86	469	301	271	271
query87	3709	3507	3479	3479
query88	4240	2296	2251	2251
query89	541	359	361	359
query90	2011	175	175	175
query91	131	105	105	105
query92	57	47	47	47
query93	6800	522	518	518
query94	1110	178	173	173
query95	431	309	318	309
query96	606	265	269	265
query97	2652	2486	2510	2486
query98	230	216	210	210
query99	1317	843	829	829
Total cold run time: 295613 ms
Total hot run time: 180714 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.08 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bfa86cd4db427e3333668781dc990cff4c0cdbe2, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.05	0.04
query4	1.67	0.06	0.06
query5	0.48	0.48	0.50
query6	1.15	0.65	0.67
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.58	0.52	0.51
query10	0.56	0.57	0.59
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.62	0.59	0.59
query14	0.78	0.81	0.79
query15	0.87	0.85	0.85
query16	0.35	0.36	0.35
query17	0.99	0.97	1.01
query18	0.26	0.25	0.26
query19	1.81	1.74	1.83
query20	0.02	0.01	0.01
query21	15.44	0.65	0.65
query22	3.97	7.01	1.75
query23	18.01	1.30	1.21
query24	1.64	0.19	0.19
query25	0.16	0.08	0.08
query26	0.27	0.16	0.17
query27	0.07	0.07	0.08
query28	13.82	0.96	0.95
query29	12.53	3.26	3.23
query30	0.27	0.06	0.07
query31	2.86	0.39	0.38
query32	3.28	0.49	0.47
query33	2.85	2.86	2.86
query34	15.49	4.41	4.33
query35	4.40	4.42	4.40
query36	0.67	0.47	0.50
query37	0.20	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.19	0.15	0.18
query41	0.10	0.05	0.04
query42	0.06	0.05	0.06
query43	0.04	0.04	0.04
Total cold run time: 107.38 s
Total hot run time: 30.08 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit bfa86cd4db427e3333668781dc990cff4c0cdbe2 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       16.1 seconds inserted 10000000 Rows, about 621K ops/s

@koarz
Copy link
Contributor Author

koarz commented Apr 4, 2024

run p0

@koarz
Copy link
Contributor Author

koarz commented Apr 4, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38679 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1c1245bc34b193f1edaf84f41f895b2a272b0181, data reload: false

------ Round 1 ----------------------------------
q1	18076	4259	4259	4259
q2	2360	192	192	192
q3	11420	1170	1341	1170
q4	10520	832	975	832
q5	7809	2973	2943	2943
q6	218	131	131	131
q7	1090	605	605	605
q8	9398	1993	2024	1993
q9	6771	6265	6151	6151
q10	8417	3508	3500	3500
q11	414	229	225	225
q12	370	210	199	199
q13	17773	2863	2914	2863
q14	273	236	243	236
q15	516	479	474	474
q16	492	394	364	364
q17	962	926	900	900
q18	7271	6526	6385	6385
q19	1606	1532	1696	1532
q20	563	310	298	298
q21	3496	3135	3131	3131
q22	356	296	301	296
Total cold run time: 110171 ms
Total hot run time: 38679 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4085	4028	4052	4028
q2	325	216	220	216
q3	2943	2970	2934	2934
q4	1889	1838	1797	1797
q5	5197	5218	5166	5166
q6	207	123	124	123
q7	2231	1803	1791	1791
q8	3220	3275	3296	3275
q9	8440	8372	8432	8372
q10	3781	3785	3806	3785
q11	534	453	459	453
q12	718	518	535	518
q13	16801	2884	2890	2884
q14	302	251	258	251
q15	507	470	465	465
q16	453	393	408	393
q17	1709	1698	1647	1647
q18	7663	7285	7184	7184
q19	1645	1635	1623	1623
q20	1939	1714	1706	1706
q21	5238	4719	4653	4653
q22	499	422	431	422
Total cold run time: 70326 ms
Total hot run time: 53686 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8892/24938)
Line Coverage: 27.39% (73002/266576)
Region Coverage: 26.57% (37755/142116)
Branch Coverage: 23.36% (19238/82348)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1c1245bc34b193f1edaf84f41f895b2a272b0181_1c1245bc34b193f1edaf84f41f895b2a272b0181/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 179631 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1c1245bc34b193f1edaf84f41f895b2a272b0181, data reload: false

query1	886	1122	1108	1108
query2	6473	1886	1813	1813
query3	6650	218	226	218
query4	24440	21308	21464	21308
query5	4207	412	403	403
query6	274	174	177	174
query7	4609	301	299	299
query8	236	173	174	173
query9	8472	2194	2196	2194
query10	589	240	249	240
query11	14898	14467	14373	14373
query12	146	92	87	87
query13	1637	376	386	376
query14	8627	6651	6811	6651
query15	203	168	173	168
query16	7132	277	269	269
query17	1005	586	563	563
query18	1897	284	275	275
query19	205	159	158	158
query20	95	88	87	87
query21	202	136	125	125
query22	5033	4851	4842	4842
query23	33633	32363	32839	32363
query24	10871	3102	3110	3102
query25	666	360	372	360
query26	1907	158	147	147
query27	3020	315	315	315
query28	6719	1805	1778	1778
query29	1342	572	562	562
query30	288	161	164	161
query31	956	699	701	699
query32	99	53	53	53
query33	646	244	238	238
query34	961	475	498	475
query35	817	689	684	684
query36	1015	858	857	857
query37	281	67	70	67
query38	3500	3343	3379	3343
query39	1568	1549	1514	1514
query40	284	120	122	120
query41	48	43	43	43
query42	109	101	98	98
query43	464	426	422	422
query44	1138	713	711	711
query45	284	261	265	261
query46	1072	767	775	767
query47	1881	1772	1798	1772
query48	367	300	294	294
query49	1133	360	343	343
query50	802	384	404	384
query51	6626	6602	6515	6515
query52	99	91	95	91
query53	358	281	275	275
query54	276	237	258	237
query55	85	73	75	73
query56	244	226	224	224
query57	1253	1125	1118	1118
query58	244	211	221	211
query59	2542	2543	2332	2332
query60	254	233	242	233
query61	110	110	105	105
query62	719	445	453	445
query63	307	282	280	280
query64	6343	3129	3226	3129
query65	3065	3013	2972	2972
query66	1466	343	337	337
query67	15245	14982	14833	14833
query68	4680	563	580	563
query69	487	301	298	298
query70	1176	1077	1124	1077
query71	434	291	263	263
query72	6317	2674	2564	2564
query73	716	324	322	322
query74	6704	6321	6422	6321
query75	3040	2310	2252	2252
query76	3155	1094	1204	1094
query77	398	247	246	246
query78	10658	10206	10035	10035
query79	8536	532	529	529
query80	1578	424	405	405
query81	514	238	238	238
query82	944	96	90	90
query83	199	166	160	160
query84	272	87	86	86
query85	1517	279	271	271
query86	464	275	297	275
query87	3679	3489	3483	3483
query88	4349	2339	2344	2339
query89	571	363	358	358
query90	2059	176	180	176
query91	131	101	101	101
query92	59	47	48	47
query93	6904	537	526	526
query94	1148	180	178	178
query95	415	306	312	306
query96	622	272	268	268
query97	2645	2483	2478	2478
query98	234	219	210	210
query99	1347	825	857	825
Total cold run time: 288975 ms
Total hot run time: 179631 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1c1245bc34b193f1edaf84f41f895b2a272b0181, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.65	0.06	0.07
query5	0.48	0.49	0.49
query6	1.12	0.65	0.65
query7	0.02	0.01	0.02
query8	0.05	0.04	0.03
query9	0.57	0.51	0.50
query10	0.56	0.57	0.56
query11	0.15	0.11	0.12
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	0.76	0.78	0.79
query15	0.85	0.84	0.84
query16	0.36	0.35	0.36
query17	0.96	0.97	0.95
query18	0.24	0.23	0.26
query19	1.76	1.72	1.69
query20	0.01	0.01	0.02
query21	15.44	0.65	0.64
query22	4.82	7.51	1.29
query23	17.88	1.24	1.19
query24	1.57	0.20	0.19
query25	0.16	0.08	0.08
query26	0.27	0.17	0.18
query27	0.08	0.07	0.07
query28	13.82	0.97	0.94
query29	12.63	3.20	3.20
query30	0.25	0.06	0.05
query31	2.89	0.39	0.38
query32	3.24	0.48	0.48
query33	2.83	2.84	2.84
query34	15.52	4.33	4.33
query35	4.35	4.34	4.35
query36	0.68	0.48	0.47
query37	0.19	0.15	0.15
query38	0.16	0.13	0.14
query39	0.04	0.04	0.03
query40	0.17	0.16	0.14
query41	0.10	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.79 s
Total hot run time: 29.3 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1c1245bc34b193f1edaf84f41f895b2a272b0181 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       15.4 seconds inserted 10000000 Rows, about 649K ops/s

@koarz
Copy link
Contributor Author

koarz commented Apr 5, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (8892/24938)
Line Coverage: 27.38% (72972/266505)
Region Coverage: 26.57% (37741/142059)
Branch Coverage: 23.37% (19235/82312)
Coverage Report: http://coverage.selectdb-in.cc/coverage/34ee7a3c57640689bf6e4e5229ae1ac954d38f97_34ee7a3c57640689bf6e4e5229ae1ac954d38f97/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38464 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34ee7a3c57640689bf6e4e5229ae1ac954d38f97, data reload: false

------ Round 1 ----------------------------------
q1	17606	4096	4022	4022
q2	2015	186	182	182
q3	10476	1188	1319	1188
q4	10210	796	991	796
q5	7540	2941	2920	2920
q6	218	133	132	132
q7	1086	628	615	615
q8	9393	2016	2004	2004
q9	6743	6201	6158	6158
q10	8455	3505	3490	3490
q11	421	232	235	232
q12	376	210	212	210
q13	17777	2924	2897	2897
q14	273	238	234	234
q15	513	493	482	482
q16	508	387	373	373
q17	944	915	890	890
q18	7201	6422	6513	6422
q19	1575	1532	1526	1526
q20	545	304	330	304
q21	3487	3097	3091	3091
q22	355	296	309	296
Total cold run time: 107717 ms
Total hot run time: 38464 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4101	4018	4005	4005
q2	327	218	229	218
q3	2948	2932	2932	2932
q4	1858	1878	1822	1822
q5	5252	5186	5182	5182
q6	211	126	127	126
q7	2243	1806	1802	1802
q8	3196	3274	3248	3248
q9	8435	8451	8457	8451
q10	3740	3872	4012	3872
q11	562	454	454	454
q12	763	593	544	544
q13	16862	3147	3100	3100
q14	306	295	294	294
q15	549	485	487	485
q16	492	413	439	413
q17	1785	1697	1760	1697
q18	8239	7771	7508	7508
q19	1674	1673	1672	1672
q20	2047	1817	1854	1817
q21	5185	4916	4915	4915
q22	491	450	426	426
Total cold run time: 71266 ms
Total hot run time: 54983 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.44% (8918/25164)
Line Coverage: 27.14% (73290/270057)
Region Coverage: 26.26% (37855/144138)
Branch Coverage: 23.06% (19288/83626)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c89acaecba6f12b33a469f3f4a362d375fe0866f_c89acaecba6f12b33a469f3f4a362d375fe0866f/report/index.html

@zclllyybb
Copy link
Contributor

run buildall

@koarz
Copy link
Contributor Author

koarz commented May 6, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

INT(5),
VARCHAR("9090"),
},
{Null()}},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'TEST' exceeds recommended size/complexity thresholds [readability-function-size]

ypes, data_set);
                          ^
Additional context

be/test/vec/function/function_string_test.cpp:1360: 243 lines including whitespace and comments (threshold 80)

ypes, data_set);
                          ^

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

INT(5),
VARCHAR("9090"),
},
{Null()}},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'TEST' exceeds recommended size/complexity thresholds [readability-function-size]

ypes, data_set);
                          ^
Additional context

be/test/vec/function/function_string_test.cpp:1360: 96 lines including whitespace and comments (threshold 80)

ypes, data_set);
                          ^

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.63% (8971/25181)
Line Coverage: 27.26% (74034/271582)
Region Coverage: 26.48% (38240/144421)
Branch Coverage: 23.23% (19471/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f76d9bdc829fd3668a05b91e3da944cef8b4c8f4_f76d9bdc829fd3668a05b91e3da944cef8b4c8f4/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 187108 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f76d9bdc829fd3668a05b91e3da944cef8b4c8f4, data reload: false

query1	894	357	358	357
query2	7569	2437	2381	2381
query3	6675	216	210	210
query4	24639	21884	21895	21884
query5	3817	415	426	415
query6	262	193	173	173
query7	4550	290	298	290
query8	227	182	183	182
query9	8459	2401	2391	2391
query10	434	261	260	260
query11	15321	14977	14793	14793
query12	120	89	87	87
query13	1674	374	380	374
query14	10698	7506	8262	7506
query15	252	172	180	172
query16	8420	253	301	253
query17	1855	569	558	558
query18	2130	275	265	265
query19	337	143	144	143
query20	91	85	79	79
query21	193	124	118	118
query22	5082	4802	4827	4802
query23	33795	33103	33165	33103
query24	10661	3039	2795	2795
query25	585	356	348	348
query26	1127	148	149	148
query27	2648	310	300	300
query28	7268	2004	2003	2003
query29	841	602	584	584
query30	262	148	152	148
query31	948	708	726	708
query32	92	51	51	51
query33	728	249	235	235
query34	1034	479	465	465
query35	795	648	663	648
query36	1085	939	945	939
query37	128	64	64	64
query38	3149	3026	3018	3018
query39	1576	1540	1540	1540
query40	204	125	126	125
query41	40	38	37	37
query42	101	93	93	93
query43	589	530	552	530
query44	1213	723	733	723
query45	270	256	249	249
query46	1082	750	713	713
query47	1937	1864	1868	1864
query48	369	285	289	285
query49	883	388	379	379
query50	756	373	376	373
query51	6678	6674	6603	6603
query52	100	87	91	87
query53	351	284	276	276
query54	293	223	228	223
query55	76	71	68	68
query56	230	222	212	212
query57	1213	1118	1138	1118
query58	215	191	196	191
query59	3520	3218	3391	3218
query60	244	227	228	227
query61	90	87	87	87
query62	650	469	447	447
query63	324	277	287	277
query64	8504	7199	7221	7199
query65	3126	3123	3079	3079
query66	1359	336	348	336
query67	15322	15013	15047	15013
query68	5100	525	525	525
query69	490	297	307	297
query70	1149	1072	1080	1072
query71	415	266	255	255
query72	7346	2552	2333	2333
query73	694	309	315	309
query74	6553	6064	6162	6064
query75	3388	2652	2637	2637
query76	3182	1034	949	949
query77	464	264	274	264
query78	11013	10286	10189	10189
query79	2908	513	524	513
query80	2077	423	430	423
query81	523	223	216	216
query82	790	98	90	90
query83	319	170	169	169
query84	273	92	84	84
query85	2071	283	267	267
query86	495	333	319	319
query87	3295	3102	3070	3070
query88	4644	2312	2330	2312
query89	482	381	375	375
query90	1995	182	180	180
query91	126	96	127	96
query92	57	45	45	45
query93	4204	501	497	497
query94	1222	181	182	181
query95	398	303	304	303
query96	602	268	274	268
query97	3147	2904	2927	2904
query98	246	223	220	220
query99	1237	870	860	860
Total cold run time: 291065 ms
Total hot run time: 187108 ms

@koarz
Copy link
Contributor Author

koarz commented May 6, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40600 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4523b9cbfe3b0f7f34187eb19c4f5da3605124f4, data reload: false

------ Round 1 ----------------------------------
q1	17881	4289	4262	4262
q2	2029	197	185	185
q3	10521	1264	1239	1239
q4	10253	748	830	748
q5	7553	2724	2691	2691
q6	218	131	132	131
q7	1024	602	600	600
q8	9292	2092	2094	2092
q9	9343	6691	6628	6628
q10	9050	3673	3738	3673
q11	449	243	241	241
q12	473	218	215	215
q13	17762	2969	2948	2948
q14	279	227	214	214
q15	495	483	458	458
q16	485	381	378	378
q17	961	740	728	728
q18	8182	7358	7244	7244
q19	2184	1550	1485	1485
q20	663	309	301	301
q21	5039	3857	3895	3857
q22	369	285	282	282
Total cold run time: 114505 ms
Total hot run time: 40600 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4367	4232	4192	4192
q2	384	267	275	267
q3	2959	2814	2701	2701
q4	1865	1584	1555	1555
q5	5271	5281	5294	5281
q6	208	124	123	123
q7	2233	1882	1871	1871
q8	3211	3325	3330	3325
q9	8397	8427	8417	8417
q10	3851	3749	3659	3659
q11	583	475	482	475
q12	745	585	608	585
q13	16769	2981	2954	2954
q14	278	299	259	259
q15	522	485	468	468
q16	479	420	406	406
q17	1746	1480	1478	1478
q18	7746	7600	7643	7600
q19	1645	1501	1516	1501
q20	1956	1772	1777	1772
q21	5009	4917	4943	4917
q22	569	511	502	502
Total cold run time: 70793 ms
Total hot run time: 54308 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.63% (8971/25181)
Line Coverage: 27.26% (74045/271582)
Region Coverage: 26.48% (38249/144421)
Branch Coverage: 23.24% (19480/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4523b9cbfe3b0f7f34187eb19c4f5da3605124f4_4523b9cbfe3b0f7f34187eb19c4f5da3605124f4/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 185798 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4523b9cbfe3b0f7f34187eb19c4f5da3605124f4, data reload: false

query1	912	351	350	350
query2	6441	2492	2306	2306
query3	6665	203	205	203
query4	23129	21085	21235	21085
query5	4123	416	422	416
query6	269	179	169	169
query7	4587	283	280	280
query8	246	201	193	193
query9	8574	2354	2345	2345
query10	445	253	249	249
query11	14824	14133	14176	14133
query12	143	91	88	88
query13	1671	367	373	367
query14	10598	7444	7754	7444
query15	257	179	178	178
query16	7910	262	269	262
query17	1864	561	545	545
query18	1567	285	282	282
query19	269	155	156	155
query20	97	87	86	86
query21	200	127	134	127
query22	5091	4889	4846	4846
query23	33818	33145	33213	33145
query24	11802	2901	2923	2901
query25	655	386	384	384
query26	1612	156	149	149
query27	2723	322	334	322
query28	6986	2038	2032	2032
query29	965	641	607	607
query30	286	151	151	151
query31	1007	731	729	729
query32	94	57	54	54
query33	759	258	249	249
query34	1082	497	473	473
query35	820	684	657	657
query36	1113	927	896	896
query37	262	70	69	69
query38	3140	3030	2985	2985
query39	1607	1570	1555	1555
query40	287	129	127	127
query41	46	44	39	39
query42	107	98	97	97
query43	596	548	539	539
query44	1229	734	753	734
query45	263	255	250	250
query46	1157	708	704	704
query47	1988	1875	1940	1875
query48	379	290	286	286
query49	1187	401	402	401
query50	794	377	381	377
query51	6755	6644	6649	6644
query52	103	91	89	89
query53	351	280	285	280
query54	331	242	232	232
query55	81	72	72	72
query56	242	220	223	220
query57	1220	1141	1161	1141
query58	226	199	203	199
query59	3417	3213	3281	3213
query60	255	232	239	232
query61	90	88	89	88
query62	673	459	441	441
query63	308	278	278	278
query64	9612	7146	7208	7146
query65	3099	3066	3104	3066
query66	1285	334	340	334
query67	15641	14950	14854	14854
query68	9663	539	542	539
query69	591	315	298	298
query70	1398	1107	1065	1065
query71	507	269	268	268
query72	8619	2543	2338	2338
query73	1641	317	316	316
query74	6760	6056	6084	6056
query75	4518	2650	2660	2650
query76	5890	992	968	968
query77	655	264	267	264
query78	11189	10269	10169	10169
query79	12175	532	521	521
query80	2601	443	448	443
query81	498	219	226	219
query82	260	94	93	93
query83	218	168	172	168
query84	261	84	87	84
query85	1106	269	279	269
query86	341	295	290	290
query87	3311	3079	3142	3079
query88	5213	2327	2329	2327
query89	520	382	382	382
query90	2043	187	190	187
query91	127	100	97	97
query92	61	48	49	48
query93	7273	514	499	499
query94	1368	186	180	180
query95	398	308	304	304
query96	617	264	259	259
query97	3157	2983	2931	2931
query98	243	221	220	220
query99	1251	885	842	842
Total cold run time: 314808 ms
Total hot run time: 185798 ms

@koarz
Copy link
Contributor Author

koarz commented May 10, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

{{
VARCHAR("Test"),
Null(),
INT(5),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'TEST' exceeds recommended size/complexity thresholds [readability-function-size]

  ^
Additional context

be/test/vec/function/function_string_test.cpp:2787: 96 lines including whitespace and comments (threshold 80)

  ^

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.69% (8988/25180)
Line Coverage: 27.36% (74325/271656)
Region Coverage: 26.59% (38411/144472)
Branch Coverage: 23.42% (19597/83684)
Coverage Report: http://coverage.selectdb-in.cc/coverage/375240aba3e7f76b9ef1ed848eadeed5e9ff4b72_375240aba3e7f76b9ef1ed848eadeed5e9ff4b72/report/index.html

@koarz
Copy link
Contributor Author

koarz commented May 10, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.69% (8986/25181)
Line Coverage: 27.35% (74308/271647)
Region Coverage: 26.59% (38409/144454)
Branch Coverage: 23.41% (19588/83678)
Coverage Report: http://coverage.selectdb-in.cc/coverage/375240aba3e7f76b9ef1ed848eadeed5e9ff4b72_375240aba3e7f76b9ef1ed848eadeed5e9ff4b72/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41644 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 375240aba3e7f76b9ef1ed848eadeed5e9ff4b72, data reload: false

------ Round 1 ----------------------------------
q1	17603	4286	4227	4227
q2	2011	189	187	187
q3	10499	1154	1153	1153
q4	10192	918	728	728
q5	7485	2720	2864	2720
q6	215	131	135	131
q7	1025	596	608	596
q8	9239	2184	2118	2118
q9	10245	6720	6750	6720
q10	9241	3916	3760	3760
q11	439	253	244	244
q12	453	227	233	227
q13	18280	3212	3191	3191
q14	251	207	214	207
q15	499	468	465	465
q16	512	412	391	391
q17	969	745	727	727
q18	8355	7801	7752	7752
q19	3785	1536	1555	1536
q20	642	314	318	314
q21	5310	3969	4531	3969
q22	360	297	281	281
Total cold run time: 117610 ms
Total hot run time: 41644 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4607	4380	4348	4348
q2	378	276	275	275
q3	3155	2945	2851	2851
q4	1957	1605	1610	1605
q5	5477	5490	5513	5490
q6	212	126	122	122
q7	2340	2017	1958	1958
q8	3279	3418	3402	3402
q9	8567	8757	8657	8657
q10	3879	3774	3857	3774
q11	586	532	495	495
q12	795	630	634	630
q13	17101	3138	3145	3138
q14	294	277	261	261
q15	538	484	479	479
q16	479	418	423	418
q17	1748	1476	1456	1456
q18	7694	7736	7402	7402
q19	1695	1527	1548	1527
q20	1989	1753	1759	1753
q21	11079	4884	4798	4798
q22	583	524	504	504
Total cold run time: 78432 ms
Total hot run time: 55343 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187509 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 375240aba3e7f76b9ef1ed848eadeed5e9ff4b72, data reload: false

query1	910	368	372	368
query2	6453	2450	2255	2255
query3	6642	217	220	217
query4	23469	21121	21167	21121
query5	4205	420	436	420
query6	257	177	172	172
query7	4580	303	289	289
query8	256	197	192	192
query9	8666	2475	2477	2475
query10	431	279	253	253
query11	14896	14121	14199	14121
query12	145	90	89	89
query13	1645	376	370	370
query14	9179	8559	7639	7639
query15	245	176	183	176
query16	8060	285	277	277
query17	1700	583	563	563
query18	2099	286	276	276
query19	203	155	152	152
query20	93	88	87	87
query21	195	128	132	128
query22	5010	4842	4864	4842
query23	34178	33728	33711	33711
query24	6537	2946	2835	2835
query25	539	388	382	382
query26	694	157	151	151
query27	1859	344	338	338
query28	3791	2100	2109	2100
query29	846	634	616	616
query30	232	159	153	153
query31	991	771	748	748
query32	90	53	62	53
query33	484	258	253	253
query34	882	486	491	486
query35	774	674	671	671
query36	1042	905	905	905
query37	102	68	69	68
query38	2890	2777	2786	2777
query39	1645	1599	1587	1587
query40	198	130	128	128
query41	45	42	42	42
query42	111	97	102	97
query43	593	563	572	563
query44	1036	734	749	734
query45	281	247	259	247
query46	1060	740	719	719
query47	1994	1956	1869	1869
query48	387	305	287	287
query49	783	394	402	394
query50	777	386	392	386
query51	6950	6701	6812	6701
query52	111	90	89	89
query53	347	280	287	280
query54	530	440	421	421
query55	80	73	71	71
query56	236	223	223	223
query57	1251	1164	1159	1159
query58	211	193	197	193
query59	3480	3196	3199	3196
query60	269	235	240	235
query61	89	88	87	87
query62	556	465	473	465
query63	305	288	291	288
query64	8279	7338	7391	7338
query65	3131	3080	3096	3080
query66	774	336	337	336
query67	15817	15962	14922	14922
query68	8930	557	562	557
query69	606	321	312	312
query70	1407	1139	1132	1132
query71	493	273	284	273
query72	8514	2522	2364	2364
query73	1588	328	330	328
query74	6561	6098	6130	6098
query75	4762	2659	2616	2616
query76	4813	989	926	926
query77	688	266	287	266
query78	10909	10231	10109	10109
query79	6626	521	517	517
query80	951	439	431	431
query81	467	220	218	218
query82	233	96	95	95
query83	234	172	173	172
query84	268	87	86	86
query85	754	297	259	259
query86	345	325	310	310
query87	3274	3126	3089	3089
query88	4779	2443	2441	2441
query89	526	395	383	383
query90	2163	190	197	190
query91	129	97	101	97
query92	60	50	51	50
query93	5590	514	510	510
query94	970	184	191	184
query95	411	309	305	305
query96	609	277	272	272
query97	3135	3000	2954	2954
query98	252	239	225	225
query99	1218	889	874	874
Total cold run time: 288861 ms
Total hot run time: 187509 ms

@zclllyybb
Copy link
Contributor

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.68% (8992/25203)
Line Coverage: 27.34% (74342/271937)
Region Coverage: 26.57% (38406/144562)
Branch Coverage: 23.38% (19589/83776)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4c68d889ba5c06b42a24bbbebb0ec75d4d56cb98_4c68d889ba5c06b42a24bbbebb0ec75d4d56cb98/report/index.html

@koarz
Copy link
Contributor Author

koarz commented May 14, 2024

run p0

@zclllyybb
Copy link
Contributor

run performance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement](doris-future) Support insert function
5 participants