[fix](functions) fix be crash for function random_bytes and mark_first/last_n by zclllyybb · Pull Request #35884 · apache/doris

zclllyybb · 2024-06-04T22:03:11Z

Proposed changes

Issue Number: close #xxx

fix crash when:
random_bytes gets negative input
mark_first/last_n gets negative 2nd arg input.

and make mark_first/last_n support literal only for 2nd arg

doris-robot · 2024-06-04T22:03:15Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zclllyybb · 2024-06-04T22:03:28Z

run buildall

github-actions · 2024-06-04T22:10:28Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-06-04T22:25:18Z

TeamCity be ut coverage result:
Function Coverage: 36.39% (8978/24674)
Line Coverage: 27.90% (73303/262752)
Region Coverage: 27.34% (37962/138864)
Branch Coverage: 23.94% (19273/80514)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5fa7530937cea919b4f474b953b97563c28e68d3_5fa7530937cea919b4f474b953b97563c28e68d3/report/index.html

doris-robot · 2024-06-04T22:43:50Z

TPC-H: Total hot run time: 40868 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5fa7530937cea919b4f474b953b97563c28e68d3, data reload: false

------ Round 1 ----------------------------------
q1	17598	4293	4271	4271
q2	2019	198	195	195
q3	10453	1237	1063	1063
q4	10195	812	838	812
q5	7451	2683	2757	2683
q6	226	132	135	132
q7	962	610	613	610
q8	9238	2114	2083	2083
q9	9588	6721	6730	6721
q10	9247	3869	3851	3851
q11	448	238	236	236
q12	458	232	226	226
q13	17236	3213	3229	3213
q14	274	212	226	212
q15	503	472	478	472
q16	467	391	379	379
q17	976	574	723	574
q18	8353	7988	7719	7719
q19	5073	1518	1481	1481
q20	657	348	318	318
q21	5148	3280	4090	3280
q22	407	348	337	337
Total cold run time: 116977 ms
Total hot run time: 40868 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4539	4400	4464	4400
q2	384	268	279	268
q3	3176	2917	2974	2917
q4	1918	1571	1576	1571
q5	5418	5479	5459	5459
q6	224	123	123	123
q7	2158	1766	1807	1766
q8	3195	3350	3366	3350
q9	8682	8568	8698	8568
q10	4156	3880	3795	3795
q11	578	485	479	479
q12	771	597	593	593
q13	17246	3097	3102	3097
q14	298	275	280	275
q15	531	480	472	472
q16	487	428	449	428
q17	1773	1471	1475	1471
q18	7991	7629	7365	7365
q19	1805	1603	1425	1425
q20	3020	1784	1755	1755
q21	10829	4788	4670	4670
q22	699	564	538	538
Total cold run time: 79878 ms
Total hot run time: 54785 ms

doris-robot · 2024-06-04T22:54:45Z

TPC-DS: Total hot run time: 172112 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5fa7530937cea919b4f474b953b97563c28e68d3, data reload: false

query1	918	376	392	376
query2	6442	2354	2482	2354
query3	6645	203	203	203
query4	19681	17347	17281	17281
query5	4116	456	464	456
query6	239	155	158	155
query7	4592	286	279	279
query8	302	288	286	286
query9	8635	2357	2324	2324
query10	446	308	273	273
query11	10381	9996	10087	9996
query12	145	94	90	90
query13	1657	365	375	365
query14	10072	6806	6932	6806
query15	228	189	185	185
query16	7246	263	258	258
query17	1317	514	539	514
query18	1925	277	265	265
query19	204	154	151	151
query20	88	81	82	81
query21	204	128	122	122
query22	4490	4136	3998	3998
query23	33580	33169	33079	33079
query24	11244	2814	2750	2750
query25	628	348	353	348
query26	1424	151	158	151
query27	2999	323	325	323
query28	7547	2010	2040	2010
query29	941	610	587	587
query30	268	151	151	151
query31	983	745	727	727
query32	85	52	52	52
query33	777	276	268	268
query34	961	475	468	468
query35	709	636	594	594
query36	1086	941	935	935
query37	143	69	68	68
query38	2849	2764	2713	2713
query39	853	777	796	777
query40	211	124	123	123
query41	53	51	50	50
query42	120	98	94	94
query43	555	542	560	542
query44	1242	736	759	736
query45	202	162	162	162
query46	1070	713	716	713
query47	1829	1771	1806	1771
query48	384	301	302	301
query49	987	400	451	400
query50	782	378	384	378
query51	6971	6768	6784	6768
query52	104	97	88	88
query53	359	286	297	286
query54	857	446	441	441
query55	74	71	70	70
query56	278	256	263	256
query57	1114	1067	1059	1059
query58	260	248	247	247
query59	3363	3251	3133	3133
query60	284	268	268	268
query61	95	86	85	85
query62	624	461	444	444
query63	324	285	289	285
query64	8960	2201	1697	1697
query65	3145	3092	3106	3092
query66	1046	324	328	324
query67	15355	14781	14943	14781
query68	4931	537	537	537
query69	608	449	367	367
query70	1074	1036	1108	1036
query71	418	282	276	276
query72	7866	5429	5572	5429
query73	759	332	324	324
query74	5896	5537	5514	5514
query75	3451	2620	2659	2620
query76	3057	895	948	895
query77	636	297	297	297
query78	10890	9926	9713	9713
query79	2101	515	518	515
query80	774	470	452	452
query81	553	217	214	214
query82	1175	100	102	100
query83	196	166	162	162
query84	251	87	81	81
query85	1406	278	271	271
query86	496	324	307	307
query87	3232	3060	3071	3060
query88	4088	2458	2451	2451
query89	478	395	390	390
query90	1807	184	190	184
query91	133	107	109	107
query92	63	50	51	50
query93	1830	505	497	497
query94	1225	196	190	190
query95	404	314	312	312
query96	587	277	270	270
query97	3144	3011	3059	3011
query98	250	231	218	218
query99	1093	863	849	849
Total cold run time: 272570 ms
Total hot run time: 172112 ms

doris-robot · 2024-06-04T23:00:14Z

ClickBench: Total hot run time: 31.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5fa7530937cea919b4f474b953b97563c28e68d3, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.22	0.04	0.04
query4	1.69	0.07	0.07
query5	0.50	0.49	0.49
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.55	0.55
query11	0.17	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.62
query14	0.77	0.78	0.78
query15	0.84	0.82	0.80
query16	0.37	0.36	0.36
query17	1.01	1.00	0.95
query18	0.23	0.25	0.24
query19	1.88	1.66	1.71
query20	0.02	0.01	0.01
query21	15.64	0.65	0.64
query22	4.19	7.08	2.61
query23	18.29	1.42	1.33
query24	2.02	0.22	0.21
query25	0.14	0.09	0.08
query26	0.27	0.17	0.17
query27	0.07	0.07	0.07
query28	13.27	1.02	0.99
query29	13.17	3.34	3.28
query30	0.24	0.05	0.06
query31	2.86	0.39	0.38
query32	3.31	0.47	0.47
query33	2.90	2.90	2.88
query34	17.22	4.40	4.47
query35	4.52	4.51	4.47
query36	0.71	0.50	0.50
query37	0.17	0.15	0.15
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.03
Total cold run time: 110.39 s
Total hot run time: 31.2 s

zclllyybb · 2024-06-05T03:10:14Z

run buildall

github-actions · 2024-06-05T03:15:42Z

clang-tidy review says "All clean, LGTM! 👍"

mrhhsg

LGTM

github-actions · 2024-06-05T03:19:58Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-06-05T03:20:00Z

PR approved by anyone and no changes requested.

zclllyybb · 2024-06-05T04:52:39Z

run buildall

github-actions · 2024-06-05T04:58:16Z

clang-tidy review says "All clean, LGTM! 👍"

zclllyybb · 2024-06-05T09:08:02Z

run buildall

github-actions · 2024-06-05T09:14:13Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-06-05T09:27:46Z

TPC-H: Total hot run time: 39622 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 062c72c7795370537adf0bcef4efb031fc60c326, data reload: false

------ Round 1 ----------------------------------
q1	18031	4481	4272	4272
q2	2023	199	188	188
q3	10493	1215	1118	1118
q4	10147	822	770	770
q5	7465	2674	2610	2610
q6	220	132	139	132
q7	958	595	592	592
q8	9215	2091	2065	2065
q9	8640	6441	6492	6441
q10	8830	3726	3692	3692
q11	484	250	244	244
q12	404	241	236	236
q13	17786	2979	3004	2979
q14	269	220	218	218
q15	520	466	465	465
q16	494	383	370	370
q17	963	700	692	692
q18	8103	7566	7317	7317
q19	5401	1469	1371	1371
q20	663	335	333	333
q21	4907	3188	3234	3188
q22	395	334	329	329
Total cold run time: 116411 ms
Total hot run time: 39622 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4394	4260	4292	4260
q2	368	276	269	269
q3	2955	2756	2710	2710
q4	1905	1591	1555	1555
q5	5282	5275	5293	5275
q6	213	128	127	127
q7	2137	1729	1727	1727
q8	3212	3354	3336	3336
q9	8357	8391	8311	8311
q10	3884	3618	3714	3618
q11	581	481	481	481
q12	793	604	608	604
q13	17399	3020	2993	2993
q14	297	273	267	267
q15	505	484	481	481
q16	470	407	421	407
q17	1767	1510	1486	1486
q18	7692	7644	7315	7315
q19	1713	1503	1525	1503
q20	1963	1788	1781	1781
q21	4812	4660	4725	4660
q22	633	535	540	535
Total cold run time: 71332 ms
Total hot run time: 53701 ms

doris-robot · 2024-06-05T09:29:01Z

TeamCity be ut coverage result:
Function Coverage: 36.42% (8986/24676)
Line Coverage: 27.93% (73409/262848)
Region Coverage: 27.36% (38008/138921)
Branch Coverage: 23.95% (19290/80528)
Coverage Report: http://coverage.selectdb-in.cc/coverage/062c72c7795370537adf0bcef4efb031fc60c326_062c72c7795370537adf0bcef4efb031fc60c326/report/index.html

doris-robot · 2024-06-05T09:38:46Z

TPC-DS: Total hot run time: 172952 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 062c72c7795370537adf0bcef4efb031fc60c326, data reload: false

query1	931	370	374	370
query2	6446	2281	2441	2281
query3	6650	206	205	205
query4	19388	17561	17464	17464
query5	4160	444	449	444
query6	248	163	151	151
query7	4589	289	283	283
query8	335	289	293	289
query9	8487	2430	2412	2412
query10	456	288	283	283
query11	10476	10164	10115	10115
query12	134	100	86	86
query13	1651	369	364	364
query14	9646	7341	7345	7341
query15	238	194	182	182
query16	7838	260	262	260
query17	1825	534	502	502
query18	1946	271	275	271
query19	198	148	151	148
query20	92	87	84	84
query21	207	127	130	127
query22	4250	4334	4139	4139
query23	33963	33019	32989	32989
query24	11926	2801	2808	2801
query25	678	374	377	374
query26	1841	163	158	158
query27	2999	322	315	315
query28	7621	2083	2068	2068
query29	1163	629	621	621
query30	295	148	152	148
query31	966	762	751	751
query32	98	57	56	56
query33	788	333	297	297
query34	887	478	467	467
query35	750	647	615	615
query36	1111	915	919	915
query37	197	65	71	65
query38	2914	2770	2749	2749
query39	857	807	788	788
query40	288	124	123	123
query41	47	54	50	50
query42	119	96	95	95
query43	574	539	549	539
query44	1217	731	749	731
query45	201	167	171	167
query46	1083	737	758	737
query47	1816	1786	1803	1786
query48	358	298	298	298
query49	1187	398	415	398
query50	783	385	374	374
query51	6825	6683	6735	6683
query52	107	95	92	92
query53	351	285	283	283
query54	985	450	429	429
query55	75	74	77	74
query56	276	255	256	255
query57	1149	1037	1056	1037
query58	251	264	233	233
query59	3416	3099	3326	3099
query60	285	291	288	288
query61	88	88	87	87
query62	634	454	451	451
query63	322	285	276	276
query64	9809	2196	1711	1711
query65	3176	3104	3122	3104
query66	1374	344	331	331
query67	15590	14992	14929	14929
query68	4538	536	530	530
query69	454	298	300	298
query70	1087	1159	1132	1132
query71	409	275	273	273
query72	7076	5633	5246	5246
query73	732	313	312	312
query74	5999	5544	5418	5418
query75	3561	2670	2629	2629
query76	2892	1016	944	944
query77	466	295	290	290
query78	10251	9863	9631	9631
query79	2358	504	508	504
query80	1231	444	443	443
query81	550	232	216	216
query82	709	104	97	97
query83	238	168	172	168
query84	240	91	83	83
query85	2041	374	298	298
query86	490	330	331	330
query87	3271	3113	3144	3113
query88	4033	2324	2322	2322
query89	490	389	385	385
query90	1820	186	193	186
query91	123	97	94	94
query92	65	48	49	48
query93	2413	505	501	501
query94	1259	183	182	182
query95	400	316	315	315
query96	591	278	260	260
query97	3225	3042	2993	2993
query98	237	222	219	219
query99	1302	859	854	854
Total cold run time: 276182 ms
Total hot run time: 172952 ms

doris-robot · 2024-06-05T09:44:16Z

ClickBench: Total hot run time: 30.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 062c72c7795370537adf0bcef4efb031fc60c326, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.06	0.06
query4	1.65	0.08	0.09
query5	0.50	0.47	0.50
query6	1.13	0.73	0.74
query7	0.02	0.01	0.01
query8	0.04	0.05	0.04
query9	0.54	0.48	0.48
query10	0.53	0.54	0.55
query11	0.15	0.11	0.11
query12	0.16	0.12	0.12
query13	0.61	0.59	0.59
query14	0.82	0.79	0.77
query15	0.84	0.81	0.81
query16	0.36	0.37	0.35
query17	1.07	0.96	1.00
query18	0.21	0.28	0.22
query19	1.76	1.69	1.70
query20	0.02	0.01	0.01
query21	15.72	0.67	0.66
query22	4.63	7.22	1.75
query23	18.26	1.45	1.33
query24	1.29	0.36	0.21
query25	0.14	0.09	0.09
query26	0.26	0.17	0.18
query27	0.08	0.07	0.08
query28	13.53	1.03	1.09
query29	12.84	3.27	3.29
query30	0.25	0.06	0.06
query31	2.85	0.38	0.39
query32	3.30	0.47	0.47
query33	2.87	2.89	2.95
query34	17.18	4.43	4.43
query35	4.44	4.48	4.46
query36	0.65	0.46	0.47
query37	0.17	0.15	0.14
query38	0.14	0.14	0.14
query39	0.04	0.03	0.03
query40	0.17	0.14	0.15
query41	0.09	0.04	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.77 s
Total hot run time: 30.41 s

github-actions · 2024-06-05T11:30:01Z

PR approved by at least one committer and no changes requested.

mrhhsg

LGTM

…t/last_n (apache#35884) fix crash when: random_bytes gets negative input mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg

…rk_first/last_n (#36003) pick #35884

…t/last_n (#35884) fix crash when: random_bytes gets negative input mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg

…t/last_n (apache#35884) fix crash when: random_bytes gets negative input mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg

fix crash when mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg it's a part of #35884

…pache#36357) fix crash when mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg it's a part of apache#35884

…t/last_n (apache#35884) fix crash when: random_bytes gets negative input mark_first/last_n gets negative 2nd arg input. and make mark_first/last_n support literal only for 2nd arg

zclllyybb force-pushed the fix_functions branch from 5fa7530 to dc86358 Compare June 5, 2024 03:09

zclllyybb marked this pull request as ready for review June 5, 2024 03:10

morrySnow reviewed Jun 5, 2024

View reviewed changes

Comment thread ...ore/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/MaskLastN.java Outdated

mrhhsg previously approved these changes Jun 5, 2024

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jun 5, 2024

github-actions Bot added the reviewed label Jun 5, 2024

zclllyybb dismissed mrhhsg’s stale review via 043156e June 5, 2024 04:52

zclllyybb requested review from morrySnow and mrhhsg June 5, 2024 04:52

github-actions Bot removed the approved Indicates a PR has been approved by one committer. label Jun 5, 2024

all

062c72c

zclllyybb force-pushed the fix_functions branch from 043156e to 062c72c Compare June 5, 2024 09:07

morrySnow approved these changes Jun 5, 2024

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jun 5, 2024

mrhhsg approved these changes Jun 6, 2024

View reviewed changes

morrySnow added p0_c dev/2.0.x dev/2.1.x labels Jun 6, 2024

morrySnow merged commit 3e27db2 into apache:master Jun 6, 2024

zclllyybb mentioned this pull request Jun 6, 2024

[branch-2.1](functions) fix be crash for function random_bytes and mark_first/last_n #36003

Merged

morningman pushed a commit that referenced this pull request Jun 7, 2024

[branch-2.1](functions) fix be crash for function random_bytes and ma…

f751ca4

…rk_first/last_n (#36003) pick #35884

morningman added dev/2.1.4-merged and removed dev/2.1.x labels Jun 7, 2024

zclllyybb mentioned this pull request Jun 16, 2024

[branch-2.0](fix) Fix be crash about mark_first/last_n #36357

Merged

xiaokang added dev/2.0.12-merged and removed dev/2.0.x labels Jun 19, 2024

zclllyybb deleted the fix_functions branch June 19, 2024 04:23

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

Conversation

zclllyybb commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Jun 4, 2024

Uh oh!

zclllyybb commented Jun 4, 2024

Uh oh!

github-actions Bot commented Jun 4, 2024

Uh oh!

doris-robot commented Jun 4, 2024

Uh oh!

doris-robot commented Jun 4, 2024

Uh oh!

doris-robot commented Jun 4, 2024

Uh oh!

doris-robot commented Jun 4, 2024

Uh oh!

zclllyybb commented Jun 5, 2024

Uh oh!

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

mrhhsg left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

zclllyybb commented Jun 5, 2024

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

zclllyybb commented Jun 5, 2024

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

doris-robot commented Jun 5, 2024

Uh oh!

doris-robot commented Jun 5, 2024

Uh oh!

doris-robot commented Jun 5, 2024

Uh oh!

doris-robot commented Jun 5, 2024

Uh oh!

github-actions Bot commented Jun 5, 2024

Uh oh!

mrhhsg left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zclllyybb commented Jun 4, 2024 •

edited

Loading