Skip to content

[bugfix](paimon)merge meta-inf/services for paimon FileIOLoader for 2.0#33204

Merged
morningman merged 1 commit intoapache:branch-2.0from
wuwenchi:fix-paimon-jar-dep-2.0
Apr 3, 2024
Merged

[bugfix](paimon)merge meta-inf/services for paimon FileIOLoader for 2.0#33204
morningman merged 1 commit intoapache:branch-2.0from
wuwenchi:fix-paimon-jar-dep-2.0

Conversation

@wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented Apr 3, 2024

Proposed changes

mirror: #33166

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

wuwenchi commented Apr 3, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50031 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a71b28aab3db1a01a5d850305fadbd465b32bc22, data reload: false

------ Round 1 ----------------------------------
q1	17660	4373	4346	4346
q2	2032	151	144	144
q3	10277	1930	1932	1930
q4	10119	1229	1308	1229
q5	8454	3920	3998	3920
q6	233	124	124	124
q7	2049	1602	1619	1602
q8	9295	2726	2728	2726
q9	11212	10516	10447	10447
q10	8661	3528	3530	3528
q11	435	242	234	234
q12	468	296	312	296
q13	18452	4031	4007	4007
q14	347	323	337	323
q15	502	460	458	458
q16	706	604	584	584
q17	1118	974	965	965
q18	7353	6898	6797	6797
q19	1668	1610	1605	1605
q20	524	305	310	305
q21	4512	4069	4060	4060
q22	498	401	415	401
Total cold run time: 116575 ms
Total hot run time: 50031 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4305	4421	4305
q2	318	219	222	219
q3	4185	4174	4167	4167
q4	2761	2729	2742	2729
q5	7232	7135	7206	7135
q6	234	121	117	117
q7	3269	2815	2847	2815
q8	4379	4470	4491	4470
q9	17139	17013	17052	17013
q10	4266	4235	4251	4235
q11	728	700	695	695
q12	1021	856	858	856
q13	6858	3743	3750	3743
q14	453	427	424	424
q15	503	461	456	456
q16	752	704	707	704
q17	3833	3847	3897	3847
q18	8818	8812	8805	8805
q19	1729	1708	1661	1661
q20	2408	2146	2135	2135
q21	8654	8566	8536	8536
q22	1082	983	951	951
Total cold run time: 84950 ms
Total hot run time: 80018 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203152 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a71b28aab3db1a01a5d850305fadbd465b32bc22, data reload: false

query1	948	399	381	381
query2	6548	2297	2126	2126
query3	6921	204	204	204
query4	19909	18104	18047	18047
query5	19708	6556	6555	6555
query6	280	212	226	212
query7	4161	301	302	301
query8	271	239	264	239
query9	3158	2722	2655	2655
query10	407	296	295	295
query11	11350	10634	10618	10618
query12	117	74	68	68
query13	5584	664	647	647
query14	17946	13787	13885	13787
query15	375	241	231	231
query16	6468	277	259	259
query17	1737	1449	860	860
query18	2328	415	404	404
query19	205	149	148	148
query20	78	78	75	75
query21	189	95	91	91
query22	5249	5056	4929	4929
query23	32546	32047	32263	32047
query24	6951	6597	6543	6543
query25	516	422	420	420
query26	620	160	159	159
query27	2011	306	299	299
query28	6128	2247	2242	2242
query29	3059	2742	2832	2742
query30	244	170	159	159
query31	892	730	749	730
query32	65	62	58	58
query33	393	254	268	254
query34	848	478	488	478
query35	1118	898	905	898
query36	1461	1179	1268	1179
query37	98	63	61	61
query38	3087	2900	2942	2900
query39	1382	1323	1314	1314
query40	204	90	99	90
query41	41	33	33	33
query42	90	85	95	85
query43	615	586	597	586
query44	1133	740	736	736
query45	248	232	228	228
query46	1245	984	1004	984
query47	1897	1772	1726	1726
query48	972	699	678	678
query49	620	366	371	366
query50	863	585	609	585
query51	4741	4640	4695	4640
query52	87	78	85	78
query53	444	316	340	316
query54	2657	2465	2442	2442
query55	88	79	82	79
query56	230	201	198	198
query57	1200	1129	1048	1048
query58	214	210	198	198
query59	3501	3465	3338	3338
query60	211	195	190	190
query61	85	81	89	81
query62	799	552	472	472
query63	477	343	353	343
query64	2372	1496	1466	1466
query65	3616	3564	3559	3559
query66	810	361	364	361
query67	15859	15556	14821	14821
query68	11041	673	698	673
query69	586	338	342	338
query70	1975	1403	1377	1377
query71	409	302	306	302
query72	6528	3432	3488	3432
query73	755	325	334	325
query74	6289	5860	5879	5860
query75	5521	3647	3713	3647
query76	6951	1157	1244	1157
query77	1174	258	257	257
query78	12748	12576	13016	12576
query79	11463	685	655	655
query80	671	404	391	391
query81	456	239	229	229
query82	563	97	95	95
query83	174	142	129	129
query84	258	68	70	68
query85	747	284	281	281
query86	340	294	293	293
query87	3211	3092	2997	2997
query88	5322	2388	2411	2388
query89	401	278	275	275
query90	2636	213	213	213
query91	154	121	119	119
query92	54	51	56	51
query93	6303	638	620	620
query94	1180	207	208	207
query95	1126	1078	1065	1065
query96	666	338	335	335
query97	6460	6387	6400	6387
query98	187	173	170	170
query99	2789	898	963	898
Total cold run time: 319813 ms
Total hot run time: 203152 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.17 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a71b28aab3db1a01a5d850305fadbd465b32bc22, data reload: false

query1	0.03	0.01	0.01
query2	0.07	0.02	0.02
query3	0.25	0.04	0.04
query4	1.79	0.06	0.06
query5	0.54	0.54	0.52
query6	1.24	0.61	0.67
query7	0.01	0.00	0.00
query8	0.03	0.02	0.03
query9	0.52	0.48	0.48
query10	0.52	0.53	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.10
query13	0.61	0.61	0.61
query14	0.79	0.78	0.78
query15	0.78	0.76	0.76
query16	0.37	0.37	0.37
query17	1.02	1.02	1.01
query18	0.22	0.27	0.24
query19	1.93	1.85	1.85
query20	0.01	0.01	0.01
query21	15.50	0.56	0.56
query22	1.91	1.98	1.54
query23	17.31	1.16	1.00
query24	3.85	3.29	1.47
query25	0.38	0.06	0.04
query26	0.76	0.15	0.16
query27	0.04	0.04	0.04
query28	6.55	0.77	0.71
query29	12.63	2.24	2.27
query30	0.61	0.53	0.54
query31	2.81	0.38	0.37
query32	3.39	0.51	0.49
query33	3.07	3.08	3.02
query34	15.23	4.80	4.79
query35	4.85	4.84	4.87
query36	1.05	1.01	1.02
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.17	0.14	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 101.3 s
Total hot run time: 31.17 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a71b28aab3db1a01a5d850305fadbd465b32bc22 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.4 seconds inserted 10000000 Rows, about 490K ops/s

@morningman morningman merged commit 38eb571 into apache:branch-2.0 Apr 3, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants