Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf #38149

Open
wants to merge 3 commits into
base: branch-2.0
Choose a base branch
from

Conversation

amorynan
Copy link
Contributor

Proposed changes

backport: #37936
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@amorynan
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466)
Line Coverage: 29.48% (66538/225696)
Region Coverage: 28.95% (34290/118446)
Branch Coverage: 24.83% (17617/70952)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3ee21bdb667057909fc92fe70819a50164d1cee4_3ee21bdb667057909fc92fe70819a50164d1cee4/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 50008 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3ee21bdb667057909fc92fe70819a50164d1cee4, data reload: false

------ Round 1 ----------------------------------
q1	17851	4491	4329	4329
q2	2081	159	155	155
q3	10461	1890	1981	1890
q4	10322	1248	1332	1248
q5	8458	3945	3864	3864
q6	252	142	123	123
q7	2006	1631	1613	1613
q8	9544	2749	2699	2699
q9	14036	10406	10437	10406
q10	8663	3497	3520	3497
q11	406	240	258	240
q12	464	300	313	300
q13	18334	3956	4024	3956
q14	358	323	324	323
q15	509	461	461	461
q16	668	577	572	572
q17	1138	936	969	936
q18	7277	6915	6955	6915
q19	1787	1658	1722	1658
q20	541	297	296	296
q21	4408	4095	4105	4095
q22	524	437	432	432
Total cold run time: 120088 ms
Total hot run time: 50008 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4347	4308	4313	4308
q2	319	238	219	219
q3	4179	4137	4108	4108
q4	2752	2734	2748	2734
q5	7175	7189	7114	7114
q6	239	121	121	121
q7	3264	2819	2806	2806
q8	4297	4441	4483	4441
q9	17405	17239	17067	17067
q10	4209	4256	4290	4256
q11	785	662	687	662
q12	1011	888	853	853
q13	6831	3790	3738	3738
q14	452	424	431	424
q15	514	457	453	453
q16	745	700	692	692
q17	3746	3794	3747	3747
q18	8654	8708	8749	8708
q19	1745	1688	1657	1657
q20	2340	2160	2110	2110
q21	8588	8405	8467	8405
q22	1086	1008	974	974
Total cold run time: 84683 ms
Total hot run time: 79597 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203702 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3ee21bdb667057909fc92fe70819a50164d1cee4, data reload: false

query1	936	423	379	379
query2	6531	2812	2565	2565
query3	6914	211	202	202
query4	21274	18058	17946	17946
query5	19731	6519	6499	6499
query6	294	222	233	222
query7	4150	302	310	302
query8	386	389	400	389
query9	3099	2657	2584	2584
query10	409	315	312	312
query11	11362	10654	10606	10606
query12	121	76	75	75
query13	5599	696	688	688
query14	17989	13206	13104	13104
query15	361	243	256	243
query16	6450	283	259	259
query17	1711	1450	877	877
query18	2314	414	409	409
query19	205	150	148	148
query20	83	78	76	76
query21	186	96	97	96
query22	5325	4961	4762	4762
query23	32446	32003	32024	32003
query24	6951	6600	6527	6527
query25	540	441	429	429
query26	533	164	167	164
query27	1802	300	299	299
query28	6137	2369	2319	2319
query29	2869	2781	2847	2781
query30	246	169	168	168
query31	921	731	747	731
query32	69	64	64	64
query33	411	260	258	258
query34	849	471	490	471
query35	1146	936	976	936
query36	1313	972	1114	972
query37	95	61	61	61
query38	3037	2904	2915	2904
query39	1366	1348	1351	1348
query40	207	93	99	93
query41	46	43	43	43
query42	80	88	86	86
query43	786	782	628	628
query44	1111	714	713	713
query45	245	232	233	232
query46	1212	988	966	966
query47	2052	1614	1628	1614
query48	1019	715	700	700
query49	620	375	387	375
query50	864	619	633	619
query51	4756	4723	4746	4723
query52	83	83	84	83
query53	452	331	314	314
query54	2661	2472	2443	2443
query55	80	81	76	76
query56	246	225	209	209
query57	1281	1194	1092	1092
query58	218	205	206	205
query59	4064	3939	3807	3807
query60	212	201	198	198
query61	99	95	100	95
query62	781	504	479	479
query63	487	342	345	342
query64	2404	1533	1527	1527
query65	3713	3582	3584	3582
query66	811	377	385	377
query67	15627	16658	15876	15876
query68	8463	653	651	651
query69	586	365	335	335
query70	1675	1338	1449	1338
query71	396	303	313	303
query72	6507	3526	3532	3526
query73	747	322	320	320
query74	6227	5772	5855	5772
query75	4621	3756	3783	3756
query76	4802	1136	1174	1136
query77	654	269	259	259
query78	12741	12185	11855	11855
query79	9163	639	652	639
query80	1317	403	416	403
query81	498	241	232	232
query82	1669	100	99	99
query83	184	131	132	131
query84	261	71	70	70
query85	894	342	349	342
query86	344	298	286	286
query87	3199	3057	3087	3057
query88	4983	2293	2274	2274
query89	443	286	279	279
query90	1854	213	223	213
query91	178	140	140	140
query92	60	53	55	53
query93	6323	568	601	568
query94	630	212	214	212
query95	1119	1076	1059	1059
query96	629	332	327	327
query97	6617	6419	6396	6396
query98	191	170	176	170
query99	2995	878	826	826
Total cold run time: 313348 ms
Total hot run time: 203702 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3ee21bdb667057909fc92fe70819a50164d1cee4, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.79	0.07	0.07
query5	0.54	0.52	0.52
query6	1.23	0.61	0.61
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.52	0.51	0.48
query10	0.53	0.52	0.53
query11	0.12	0.08	0.08
query12	0.12	0.09	0.09
query13	0.63	0.62	0.61
query14	0.79	0.78	0.81
query15	0.77	0.75	0.76
query16	0.38	0.37	0.37
query17	1.04	1.01	1.00
query18	0.23	0.25	0.23
query19	1.84	1.84	1.81
query20	0.01	0.01	0.01
query21	15.47	0.57	0.55
query22	2.07	2.76	2.25
query23	17.22	1.12	0.85
query24	6.71	0.65	0.49
query25	0.37	0.11	0.06
query26	0.64	0.15	0.17
query27	0.04	0.03	0.04
query28	7.25	0.76	0.70
query29	12.63	2.35	2.17
query30	0.54	0.53	0.52
query31	2.82	0.40	0.37
query32	3.38	0.49	0.49
query33	3.05	3.03	3.14
query34	15.26	4.82	4.80
query35	4.88	4.86	4.84
query36	1.05	1.02	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.15	0.14	0.15
query41	0.07	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.68 s
Total hot run time: 30.63 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3ee21bdb667057909fc92fe70819a50164d1cee4 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@amorynan
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466)
Line Coverage: 29.48% (66546/225741)
Region Coverage: 28.95% (34299/118466)
Branch Coverage: 24.83% (17622/70960)
Coverage Report: http://coverage.selectdb-in.cc/coverage/dcf18ba45f3f802fc50dba752dffe83fb6227428_dcf18ba45f3f802fc50dba752dffe83fb6227428/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49757 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false

------ Round 1 ----------------------------------
q1	17667	4383	4306	4306
q2	2073	155	144	144
q3	10271	1911	2181	1911
q4	10126	1261	1313	1261
q5	8487	3958	3939	3939
q6	232	146	123	123
q7	2051	1583	1593	1583
q8	9311	2758	2704	2704
q9	10582	10484	10228	10228
q10	8658	3534	3494	3494
q11	427	234	251	234
q12	471	304	295	295
q13	18323	3932	4021	3932
q14	351	318	318	318
q15	509	466	466	466
q16	677	578	563	563
q17	1126	973	974	973
q18	7278	6979	6828	6828
q19	1793	1639	1611	1611
q20	530	307	314	307
q21	4370	4102	4094	4094
q22	531	443	456	443
Total cold run time: 115844 ms
Total hot run time: 49757 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4296	4330	4258	4258
q2	322	230	221	221
q3	4146	4137	4110	4110
q4	2759	2734	2733	2733
q5	7142	7125	7080	7080
q6	236	121	119	119
q7	3239	2826	2848	2826
q8	4377	4495	4512	4495
q9	16805	16708	16692	16692
q10	4220	4255	4247	4247
q11	796	681	697	681
q12	1027	859	837	837
q13	6948	3725	3704	3704
q14	460	433	432	432
q15	500	452	460	452
q16	744	682	665	665
q17	3871	3912	3799	3799
q18	8775	8691	8806	8691
q19	1714	1699	1696	1696
q20	2364	2090	2113	2090
q21	8641	8431	8500	8431
q22	1075	980	1013	980
Total cold run time: 84457 ms
Total hot run time: 79239 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203935 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false

query1	930	424	382	382
query2	6528	2772	2639	2639
query3	6922	210	206	206
query4	19794	18058	17930	17930
query5	19714	6494	6479	6479
query6	284	220	226	220
query7	4162	300	303	300
query8	439	478	427	427
query9	3135	2681	2614	2614
query10	409	317	307	307
query11	11311	10719	10775	10719
query12	121	81	76	76
query13	5604	698	712	698
query14	17701	13041	13298	13041
query15	364	236	245	236
query16	6486	295	271	271
query17	1691	1463	897	897
query18	2289	416	406	406
query19	215	149	151	149
query20	80	77	80	77
query21	187	100	96	96
query22	5352	5075	5029	5029
query23	32416	31909	31903	31903
query24	7139	6456	6494	6456
query25	517	449	430	430
query26	528	164	155	155
query27	1905	298	298	298
query28	6176	2374	2341	2341
query29	3012	2861	2635	2635
query30	248	167	172	167
query31	887	731	720	720
query32	71	62	62	62
query33	410	250	264	250
query34	851	467	482	467
query35	1103	904	901	901
query36	1222	1197	1137	1137
query37	89	60	62	60
query38	3069	2955	2975	2955
query39	1374	1332	1320	1320
query40	206	106	90	90
query41	46	45	45	45
query42	83	82	80	80
query43	812	630	785	630
query44	1113	713	703	703
query45	248	235	231	231
query46	1208	938	973	938
query47	1849	1723	1920	1723
query48	1001	707	711	707
query49	613	371	372	371
query50	862	615	649	615
query51	4675	4702	4670	4670
query52	103	81	98	81
query53	442	346	316	316
query54	2655	2434	2413	2413
query55	97	86	89	86
query56	225	237	201	201
query57	1242	1139	1156	1139
query58	219	193	210	193
query59	4275	3855	4073	3855
query60	209	205	214	205
query61	97	105	93	93
query62	784	449	486	449
query63	484	344	337	337
query64	2554	1549	1477	1477
query65	3650	3523	3572	3523
query66	781	382	368	368
query67	15598	16732	16067	16067
query68	8288	657	662	657
query69	554	359	346	346
query70	1553	1535	1379	1379
query71	374	304	327	304
query72	6462	3479	3474	3474
query73	733	333	325	325
query74	6300	5802	5851	5802
query75	4532	3666	3705	3666
query76	4638	1181	1190	1181
query77	593	268	257	257
query78	12615	11661	12033	11661
query79	9203	658	659	658
query80	1919	400	409	400
query81	504	236	232	232
query82	1689	101	96	96
query83	173	133	138	133
query84	261	71	69	69
query85	1118	324	336	324
query86	324	297	298	297
query87	3263	3060	3049	3049
query88	5229	2338	2346	2338
query89	387	290	285	285
query90	1721	206	210	206
query91	171	154	138	138
query92	59	54	56	54
query93	5298	542	613	542
query94	673	209	211	209
query95	1092	1067	1047	1047
query96	636	338	333	333
query97	6515	6370	6282	6282
query98	193	173	177	173
query99	2922	956	974	956
Total cold run time: 310858 ms
Total hot run time: 203935 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.02
query3	0.25	0.05	0.04
query4	1.77	0.07	0.07
query5	0.54	0.53	0.52
query6	1.34	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.52	0.48	0.49
query10	0.53	0.53	0.54
query11	0.12	0.09	0.09
query12	0.11	0.08	0.09
query13	0.62	0.61	0.61
query14	0.77	0.78	0.81
query15	0.79	0.76	0.76
query16	0.36	0.36	0.37
query17	1.02	1.01	0.98
query18	0.23	0.25	0.25
query19	1.88	1.85	1.84
query20	0.02	0.01	0.01
query21	15.46	0.57	0.56
query22	2.00	2.13	1.40
query23	17.27	1.03	0.79
query24	4.24	2.98	1.07
query25	0.37	0.10	0.06
query26	0.64	0.15	0.16
query27	0.04	0.04	0.03
query28	6.88	0.77	0.73
query29	12.60	2.28	2.34
query30	0.61	0.53	0.53
query31	2.83	0.39	0.38
query32	3.35	0.49	0.50
query33	3.11	3.06	3.06
query34	15.23	4.78	4.78
query35	4.84	4.84	4.82
query36	1.05	1.01	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.13	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.01	0.01
Total cold run time: 101.92 s
Total hot run time: 30.46 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@amorynan
Copy link
Contributor Author

run p0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants