[opt](parquet-reader) Opt parquet fixed length decimal type reading. #29825

kaka11chen · 2024-01-10T18:03:34Z

Proposed changes

[opt] (parquet-reader) Opt parquet fixed length decimal type reading.

select count(l_quantity) from lineitem;

For tpch-500-parquet(one node): 5.81 s -> 2.70 s

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

kaka11chen · 2024-01-10T18:06:23Z

run buildall

github-actions · 2024-01-10T18:11:03Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-01-10T18:47:17Z

TPC-H: Total hot run time: 38478 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c39293ffc96a91ecd0f776ca49cf7c0191ef2bb0, data reload: false

------ Round 1 ----------------------------------
q1	17975	4954	4941	4941
q2	2042	140	132	132
q3	10686	1106	1189	1106
q4	10353	785	856	785
q5	7901	3089	3073	3073
q6	203	124	121	121
q7	879	501	482	482
q8	9274	2002	1997	1997
q9	6654	6426	6435	6426
q10	8241	3055	3062	3055
q11	412	205	209	205
q12	361	192	202	192
q13	18109	3419	3405	3405
q14	231	215	210	210
q15	555	504	513	504
q16	423	378	377	377
q17	929	513	515	513
q18	7245	6677	6657	6657
q19	1586	1394	1406	1394
q20	582	292	285	285
q21	2726	2300	2374	2300
q22	363	318	321	318
Total cold run time: 107730 ms
Total hot run time: 38478 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4923	4942	5010	4942
q2	309	196	197	196
q3	3322	3280	3280	3280
q4	2219	2254	2229	2229
q5	5783	5783	5751	5751
q6	189	118	117	117
q7	2320	1849	1847	1847
q8	3412	3530	3567	3530
q9	8862	8819	8794	8794
q10	3706	3797	3776	3776
q11	559	437	424	424
q12	783	614	594	594
q13	7310	3228	3226	3226
q14	279	263	271	263
q15	572	513	510	510
q16	514	451	466	451
q17	2028	2010	2009	2009
q18	8758	8349	8264	8264
q19	1609	1603	1602	1602
q20	2152	1925	1918	1918
q21	6008	5667	5651	5651
q22	575	489	502	489
Total cold run time: 66192 ms
Total hot run time: 59863 ms

doris-robot · 2024-01-10T18:53:09Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.04 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
storage size: 17183857666 Bytes

doris-robot · 2024-01-10T18:58:51Z

TPC-DS: Total hot run time: 178366 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c39293ffc96a91ecd0f776ca49cf7c0191ef2bb0, data reload: false

query1	935	336	327	327
query2	6724	1952	1818	1818
query3	6720	200	203	200
query4	26774	22243	22038	22038
query5	4899	490	480	480
query6	274	175	179	175
query7	4623	273	282	273
query8	245	210	206	206
query9	9169	2744	2728	2728
query10	571	253	246	246
query11	16154	15375	15289	15289
query12	128	78	77	77
query13	1624	336	347	336
query14	11933	7230	7253	7230
query15	232	204	200	200
query16	6610	272	263	263
query17	1934	491	474	474
query18	1973	273	261	261
query19	299	137	141	137
query20	84	85	80	80
query21	185	92	91	91
query22	5124	4821	4844	4821
query23	32183	31143	31242	31143
query24	12437	2841	2779	2779
query25	574	337	339	337
query26	1766	153	152	152
query27	2887	271	286	271
query28	7178	1929	1912	1912
query29	2082	407	396	396
query30	285	143	143	143
query31	1002	759	783	759
query32	93	62	56	56
query33	741	270	265	265
query34	1024	437	429	429
query35	878	779	720	720
query36	1383	1245	1229	1229
query37	188	69	68	68
query38	3397	3289	3263	3263
query39	1349	1267	1294	1267
query40	293	97	89	89
query41	39	35	34	34
query42	98	86	96	86
query43	558	512	520	512
query44	1066	708	722	708
query45	207	189	183	183
query46	1042	655	663	655
query47	1707	1558	1558	1558
query48	390	318	321	318
query49	1288	323	314	314
query50	733	357	326	326
query51	5467	5286	5348	5286
query52	96	95	85	85
query53	220	154	152	152
query54	1399	586	618	586
query55	111	91	96	91
query56	217	211	204	204
query57	1133	1001	1104	1001
query58	231	216	218	216
query59	2834	2477	2464	2464
query60	248	242	241	241
query61	82	83	80	80
query62	679	474	480	474
query63	171	152	147	147
query64	5826	1696	1661	1661
query65	3309	3265	3254	3254
query66	1381	365	331	331
query67	15812	15329	15386	15329
query68	12608	542	502	502
query69	549	270	277	270
query70	1699	1584	1577	1577
query71	500	238	210	210
query72	4880	2827	2854	2827
query73	2961	318	308	308
query74	6999	6390	6363	6363
query75	5392	2304	2258	2258
query76	6473	1092	1125	1092
query77	662	288	279	279
query78	9876	8643	8661	8643
query79	1050	499	498	498
query80	560	344	357	344
query81	451	212	208	208
query82	211	94	88	88
query83	190	137	140	137
query84	263	57	57	57
query85	978	303	286	286
query86	420	359	404	359
query87	3538	3369	3346	3346
query88	2906	2220	2203	2203
query89	337	258	249	249
query90	1958	199	190	190
query91	164	127	131	127
query92	58	56	51	51
query93	1417	493	428	428
query94	886	189	192	189
query95	501	416	414	414
query96	638	319	314	314
query97	4270	4189	4185	4185
query98	218	199	184	184
query99	1140	894	856	856
Total cold run time: 298051 ms
Total hot run time: 178366 ms

doris-robot · 2024-01-10T19:04:28Z

ClickBench: Total hot run time: 30.62 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c39293ffc96a91ecd0f776ca49cf7c0191ef2bb0, data reload: false

query1	0.06	0.06	0.06
query2	0.06	0.03	0.03
query3	0.27	0.12	0.11
query4	2.21	0.14	0.12
query5	0.53	0.53	0.52
query6	1.37	0.63	0.62
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.49	0.50
query10	0.53	0.54	0.53
query11	0.13	0.10	0.09
query12	0.12	0.10	0.09
query13	0.60	0.60	0.59
query14	0.77	0.79	0.81
query15	0.81	0.79	0.80
query16	0.35	0.35	0.35
query17	0.96	0.94	0.99
query18	0.25	0.25	0.25
query19	1.84	1.78	1.79
query20	0.01	0.02	0.02
query21	15.39	0.56	0.56
query22	2.35	3.02	2.12
query23	16.99	0.88	0.82
query24	5.60	0.80	0.36
query25	1.67	0.15	0.15
query26	0.13	0.14	0.13
query27	0.15	0.14	0.14
query28	10.55	0.83	0.80
query29	12.55	3.26	3.16
query30	0.60	0.56	0.50
query31	2.79	0.36	0.35
query32	3.38	0.48	0.48
query33	3.18	3.22	3.22
query34	15.91	4.20	4.16
query35	4.18	4.16	4.16
query36	1.11	1.04	1.05
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.17	0.13	0.14
query41	0.07	0.02	0.02
query42	0.03	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 108.42 s
Total hot run time: 30.62 s

doris-robot · 2024-01-10T19:07:33Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c39293ffc96a91ecd0f776ca49cf7c0191ef2bb0 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.1 seconds inserted 10000000 Rows, about 709K ops/s

github-actions

clang-tidy made some suggestions

github-actions · 2024-01-11T00:18:26Z

be/src/vec/exec/format/parquet/parquet_column_convert.h

 // under the License.

+#pragma once
 #include <gen_cpp/PlanNodes_types.h>


warning: 'gen_cpp/PlanNodes_types.h' file not found [clang-diagnostic-error]

#include <gen_cpp/PlanNodes_types.h> ^

kaka11chen · 2024-01-11T00:29:45Z

run buildall

doris-robot · 2024-01-11T00:59:58Z

TPC-H: Total hot run time: 38400 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de509f0828191eb41bba37770c05036ac80b5cf3, data reload: false

------ Round 1 ----------------------------------
q1	18029	5024	4963	4963
q2	2044	145	141	141
q3	10666	1086	1190	1086
q4	10249	779	846	779
q5	7848	3121	3149	3121
q6	204	123	129	123
q7	863	517	492	492
q8	9243	1939	1983	1939
q9	6487	6479	6406	6406
q10	8247	3063	2997	2997
q11	414	200	199	199
q12	364	194	199	194
q13	18113	3436	3430	3430
q14	242	217	216	216
q15	554	515	510	510
q16	431	387	379	379
q17	937	510	533	510
q18	7192	6720	6617	6617
q19	1578	1299	1359	1299
q20	574	312	287	287
q21	2748	2408	2391	2391
q22	361	321	328	321
Total cold run time: 107388 ms
Total hot run time: 38400 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4899	5002	4923	4923
q2	319	200	195	195
q3	3313	3286	3285	3285
q4	2217	2219	2228	2219
q5	5794	5771	5769	5769
q6	194	121	119	119
q7	2305	1850	1834	1834
q8	3422	3537	3537	3537
q9	8847	8796	8821	8796
q10	3741	3804	3806	3804
q11	543	447	421	421
q12	781	600	599	599
q13	6406	3237	3191	3191
q14	284	257	269	257
q15	573	509	493	493
q16	518	456	470	456
q17	2049	2021	2025	2021
q18	8778	8370	8264	8264
q19	1597	1578	1585	1578
q20	2147	1933	1920	1920
q21	6019	5634	5640	5634
q22	574	485	496	485
Total cold run time: 65320 ms
Total hot run time: 59800 ms

doris-robot · 2024-01-11T01:11:27Z

TPC-DS: Total hot run time: 178587 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de509f0828191eb41bba37770c05036ac80b5cf3, data reload: false

query1	925	339	330	330
query2	6736	1901	1770	1770
query3	6709	207	203	203
query4	25954	22315	22142	22142
query5	5813	514	506	506
query6	280	191	195	191
query7	4626	288	264	264
query8	233	210	214	210
query9	9194	2784	2864	2784
query10	531	246	260	246
query11	16296	15338	15265	15265
query12	131	78	77	77
query13	1627	329	337	329
query14	11900	7252	7200	7200
query15	270	203	203	203
query16	6367	269	270	269
query17	1881	483	486	483
query18	1949	275	269	269
query19	294	143	140	140
query20	86	79	80	79
query21	190	92	95	92
query22	5161	4876	4903	4876
query23	31933	31174	31118	31118
query24	12146	2804	2757	2757
query25	574	334	339	334
query26	1762	150	149	149
query27	2858	293	287	287
query28	6834	1922	1908	1908
query29	2090	394	378	378
query30	284	145	143	143
query31	964	763	781	763
query32	97	57	60	57
query33	735	265	283	265
query34	910	435	449	435
query35	943	739	781	739
query36	1356	1200	1240	1200
query37	188	67	72	67
query38	3389	3264	3255	3255
query39	1321	1284	1281	1281
query40	325	98	93	93
query41	40	34	35	34
query42	97	91	94	91
query43	562	537	515	515
query44	1070	707	731	707
query45	213	183	194	183
query46	1051	656	638	638
query47	1656	1607	1626	1607
query48	414	324	329	324
query49	1268	325	313	313
query50	730	370	338	338
query51	5375	5287	5315	5287
query52	99	87	93	87
query53	218	160	152	152
query54	1411	576	594	576
query55	105	88	88	88
query56	225	204	204	204
query57	1054	982	982	982
query58	232	213	223	213
query59	2679	2585	2538	2538
query60	248	243	233	233
query61	81	82	82	82
query62	671	471	487	471
query63	170	156	155	155
query64	5963	1762	1709	1709
query65	3330	3253	3269	3253
query66	1325	341	329	329
query67	15737	15205	15406	15205
query68	11545	530	531	530
query69	562	285	281	281
query70	1738	1491	1539	1491
query71	514	210	221	210
query72	4894	2851	2842	2842
query73	2126	316	307	307
query74	6955	6448	6511	6448
query75	4761	2272	2299	2272
query76	6339	1037	1097	1037
query77	667	262	282	262
query78	9619	8698	8657	8657
query79	1017	499	512	499
query80	563	354	348	348
query81	453	222	229	222
query82	205	87	92	87
query83	157	140	137	137
query84	271	59	57	57
query85	968	285	282	282
query86	394	379	376	376
query87	3529	3426	3344	3344
query88	3051	2224	2231	2224
query89	350	262	265	262
query90	1788	210	196	196
query91	171	138	132	132
query92	65	56	52	52
query93	1035	432	478	432
query94	819	190	194	190
query95	489	431	415	415
query96	638	317	325	317
query97	4278	4169	4217	4169
query98	206	197	184	184
query99	1144	877	854	854
Total cold run time: 293197 ms
Total hot run time: 178587 ms

doris-robot · 2024-01-11T01:15:24Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.21 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
storage size: 17184131772 Bytes

doris-robot · 2024-01-11T01:17:02Z

ClickBench: Total hot run time: 30.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de509f0828191eb41bba37770c05036ac80b5cf3, data reload: false

query1	0.10	0.05	0.05
query2	0.06	0.02	0.02
query3	0.25	0.11	0.11
query4	1.78	0.11	0.11
query5	0.52	0.52	0.52
query6	1.36	0.64	0.62
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.48	0.48
query10	0.55	0.54	0.54
query11	0.12	0.09	0.08
query12	0.11	0.09	0.09
query13	0.61	0.60	0.60
query14	0.79	0.80	0.78
query15	0.81	0.80	0.80
query16	0.34	0.34	0.35
query17	0.98	0.98	1.00
query18	0.24	0.24	0.25
query19	1.82	1.76	1.78
query20	0.01	0.00	0.01
query21	15.40	0.57	0.55
query22	2.16	2.07	1.64
query23	17.42	0.72	0.80
query24	5.46	0.46	1.23
query25	1.88	0.17	0.14
query26	0.13	0.13	0.14
query27	0.15	0.14	0.14
query28	9.92	0.82	0.80
query29	12.58	3.22	3.22
query30	0.58	0.49	0.49
query31	2.78	0.34	0.36
query32	3.40	0.49	0.48
query33	3.19	3.25	3.26
query34	15.75	4.18	4.11
query35	4.16	4.14	4.17
query36	1.10	1.04	1.06
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.16	0.13	0.14
query41	0.06	0.02	0.02
query42	0.03	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 107.53 s
Total hot run time: 30.08 s

doris-robot · 2024-01-11T01:20:07Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit de509f0828191eb41bba37770c05036ac80b5cf3 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.1 seconds inserted 10000000 Rows, about 709K ops/s

kaka11chen · 2024-01-11T02:21:34Z

run buildall

doris-robot · 2024-01-11T03:06:39Z

TPC-H: Total hot run time: 38459 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b, data reload: false

------ Round 1 ----------------------------------
q1	18273	5103	4941	4941
q2	2044	147	135	135
q3	10857	1074	1147	1074
q4	10565	800	853	800
q5	7844	3110	3118	3110
q6	206	122	124	122
q7	872	498	492	492
q8	9485	1991	2008	1991
q9	7684	6505	6496	6496
q10	8317	3027	3080	3027
q11	431	217	213	213
q12	365	191	189	189
q13	18152	3414	3411	3411
q14	241	214	211	211
q15	558	523	513	513
q16	476	375	382	375
q17	949	549	497	497
q18	7028	6762	6599	6599
q19	1599	1269	1366	1269
q20	520	298	289	289
q21	2733	2410	2387	2387
q22	361	318	324	318
Total cold run time: 109560 ms
Total hot run time: 38459 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5005	4971	4945	4945
q2	312	197	207	197
q3	3319	3316	3286	3286
q4	2248	2259	2246	2246
q5	5791	5779	5805	5779
q6	194	118	121	118
q7	2293	1831	1856	1831
q8	3450	3538	3594	3538
q9	8865	8793	8784	8784
q10	3731	3813	3836	3813
q11	542	425	423	423
q12	772	593	596	593
q13	3859	3214	3205	3205
q14	286	258	261	258
q15	583	507	500	500
q16	500	461	465	461
q17	2046	2034	2056	2034
q18	8763	8282	8301	8282
q19	1613	1627	1618	1618
q20	2184	1928	1921	1921
q21	6114	5624	5683	5624
q22	543	500	493	493
Total cold run time: 63013 ms
Total hot run time: 59949 ms

doris-robot · 2024-01-11T03:16:54Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.61 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.0 seconds inserted 10000000 Rows, about 476K ops/s
storage size: 17183790933 Bytes

doris-robot · 2024-01-11T03:18:08Z

TPC-DS: Total hot run time: 178151 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b, data reload: false

query1	943	339	333	333
query2	6699	1987	1914	1914
query3	6709	211	202	202
query4	26096	22350	22281	22281
query5	5546	501	528	501
query6	270	203	191	191
query7	4620	279	284	279
query8	239	206	218	206
query9	9150	2732	2721	2721
query10	635	258	237	237
query11	16219	15424	15228	15228
query12	139	81	81	81
query13	1643	336	350	336
query14	12251	7309	7253	7253
query15	260	186	191	186
query16	6502	275	268	268
query17	1903	486	489	486
query18	1955	270	263	263
query19	277	138	136	136
query20	84	77	79	77
query21	188	89	89	89
query22	4972	4596	4876	4596
query23	32054	31290	31184	31184
query24	12308	2832	2768	2768
query25	569	348	338	338
query26	1774	155	150	150
query27	2912	265	280	265
query28	7097	1918	1915	1915
query29	2023	405	393	393
query30	282	142	142	142
query31	980	790	784	784
query32	92	62	57	57
query33	754	267	267	267
query34	1020	427	432	427
query35	893	743	779	743
query36	1353	1292	1265	1265
query37	185	69	69	69
query38	3418	3264	3290	3264
query39	1336	1280	1282	1280
query40	304	91	91	91
query41	40	35	35	35
query42	100	92	93	92
query43	525	521	506	506
query44	1107	703	724	703
query45	210	180	182	180
query46	1036	654	674	654
query47	1605	1540	1564	1540
query48	391	316	315	315
query49	1274	320	309	309
query50	730	317	354	317
query51	5315	5257	5219	5219
query52	106	84	92	84
query53	217	154	152	152
query54	1460	568	560	560
query55	91	82	88	82
query56	215	198	203	198
query57	990	887	919	887
query58	242	214	214	214
query59	2690	2612	2512	2512
query60	257	220	245	220
query61	86	88	88	88
query62	650	470	459	459
query63	173	150	154	150
query64	5848	1703	1700	1700
query65	3331	3283	3263	3263
query66	1332	351	328	328
query67	15946	15448	15225	15225
query68	10977	530	509	509
query69	571	283	282	282
query70	1696	1543	1546	1543
query71	501	207	231	207
query72	4866	2861	2843	2843
query73	2134	316	318	316
query74	6953	6428	6450	6428
query75	4898	2300	2280	2280
query76	6380	1012	1127	1012
query77	724	276	273	273
query78	9799	8569	8474	8474
query79	1028	524	489	489
query80	557	345	375	345
query81	453	208	209	208
query82	205	95	90	90
query83	171	133	136	133
query84	266	57	56	56
query85	959	293	291	291
query86	394	367	363	363
query87	3535	3427	3358	3358
query88	3033	2235	2251	2235
query89	320	263	253	253
query90	1881	201	197	197
query91	181	134	148	134
query92	57	59	50	50
query93	1115	425	427	425
query94	771	196	186	186
query95	472	435	427	427
query96	627	326	329	326
query97	4331	4134	4183	4134
query98	197	191	183	183
query99	1071	870	843	843
Total cold run time: 293744 ms
Total hot run time: 178151 ms

doris-robot · 2024-01-11T03:23:45Z

ClickBench: Total hot run time: 30.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b, data reload: false

query1	0.06	0.05	0.05
query2	0.06	0.02	0.03
query3	0.24	0.12	0.10
query4	1.78	0.12	0.11
query5	0.53	0.52	0.51
query6	1.40	0.64	0.63
query7	0.02	0.02	0.01
query8	0.04	0.03	0.02
query9	0.55	0.50	0.50
query10	0.55	0.55	0.56
query11	0.13	0.10	0.08
query12	0.11	0.09	0.09
query13	0.62	0.60	0.60
query14	0.77	0.79	0.80
query15	0.82	0.80	0.80
query16	0.34	0.35	0.35
query17	0.95	0.96	0.95
query18	0.26	0.26	0.25
query19	1.87	1.73	1.78
query20	0.01	0.01	0.02
query21	15.42	0.58	0.59
query22	2.63	2.02	2.34
query23	17.37	0.83	0.80
query24	5.40	1.17	0.66
query25	1.81	0.15	0.15
query26	0.14	0.15	0.13
query27	0.16	0.15	0.15
query28	9.98	0.83	0.82
query29	12.63	3.23	3.17
query30	0.61	0.57	0.50
query31	2.77	0.35	0.35
query32	3.40	0.49	0.49
query33	3.20	3.18	3.18
query34	15.89	4.16	4.24
query35	4.19	4.25	4.13
query36	1.11	1.06	1.05
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.16	0.15	0.13
query41	0.07	0.01	0.01
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 108.23 s
Total hot run time: 30.75 s

doris-robot · 2024-01-11T03:26:49Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       12.8 seconds inserted 10000000 Rows, about 781K ops/s

doris-robot · 2024-01-11T03:47:43Z

TeamCity be ut coverage result:
Function Coverage: 36.67% (8642/23565)
Line Coverage: 28.72% (70429/245242)
Region Coverage: 27.65% (36382/131582)
Branch Coverage: 24.35% (18608/76414)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b_9d739bc3b96ff6a42ac0b74f7b0c9aaba6ceea5b/report/index.html

github-actions · 2024-01-11T09:54:45Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-11T09:54:47Z

PR approved by anyone and no changes requested.

kaka11chen force-pushed the opt_parquet_fixed_length_decimal_reading branch from c39293f to de509f0 Compare January 11, 2024 00:10

github-actions bot reviewed Jan 11, 2024

View reviewed changes

[opt](parquet-reader) Opt parquet decimal type reading.

9d739bc

kaka11chen force-pushed the opt_parquet_fixed_length_decimal_reading branch from de509f0 to 9d739bc Compare January 11, 2024 02:19

kaka11chen marked this pull request as ready for review January 11, 2024 06:56

AshinGau approved these changes Jan 11, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 11, 2024

github-actions bot added the reviewed label Jan 11, 2024

xiaokang added the dev/2.0.5 label Jan 11, 2024

yiguolei approved these changes Jan 11, 2024

View reviewed changes

yiguolei merged commit ff20c30 into apache:master Jan 11, 2024

yiguolei pushed a commit that referenced this pull request Jan 12, 2024

[opt](parquet-reader) Opt parquet decimal type reading. (#29825)

1941087

yiguolei pushed a commit that referenced this pull request Jan 12, 2024

[opt](parquet-reader) Opt parquet decimal type reading. (#29825)

d494674

AshinGau pushed a commit to AshinGau/incubator-doris that referenced this pull request Jan 15, 2024

[opt](parquet-reader) Opt parquet decimal type reading. (apache#29825)

dda22d8

wm1581066 removed the dev/2.0.5 label Jan 29, 2024

kaka11chen mentioned this pull request Jan 29, 2024

[Fix](parquet-reader) Fix reading fixed length byte array decimal in parquet reader introduced by #29825. #30535

Merged

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[opt](parquet-reader) Opt parquet fixed length decimal type reading. #29825

[opt](parquet-reader) Opt parquet fixed length decimal type reading. #29825

Uh oh!

Conversation

kaka11chen commented Jan 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

kaka11chen commented Jan 10, 2024

Uh oh!

github-actions bot commented Jan 10, 2024

Uh oh!

doris-robot commented Jan 10, 2024

Uh oh!

doris-robot commented Jan 10, 2024

Uh oh!

doris-robot commented Jan 10, 2024

Uh oh!

doris-robot commented Jan 10, 2024

Uh oh!

doris-robot commented Jan 10, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jan 11, 2024

Choose a reason for hiding this comment

Uh oh!

kaka11chen commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

kaka11chen commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

doris-robot commented Jan 11, 2024

Uh oh!

github-actions bot commented Jan 11, 2024

Uh oh!

github-actions bot commented Jan 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

kaka11chen commented Jan 10, 2024 •

edited

Loading