Skip to content

[dependency](chore)Remove hadoop-yarn related dependencies and upgrade xnio to 3.8.14#34235

Merged
morningman merged 1 commit into
apache:branch-2.0from
CalvinKirs:branch-2.0-may-dependency
Apr 29, 2024
Merged

[dependency](chore)Remove hadoop-yarn related dependencies and upgrade xnio to 3.8.14#34235
morningman merged 1 commit into
apache:branch-2.0from
CalvinKirs:branch-2.0-may-dependency

Conversation

@CalvinKirs
Copy link
Copy Markdown
Member

We don’t need yarn and mr related dependencies

…e xnio to 3.8.14

We don’t need yarn and mr related dependencies
@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 49436 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e1d595561d1d60e1ee7bc88ba205f50a72a8ddc2, data reload: false

------ Round 1 ----------------------------------
q1	18036	4350	4300	4300
q2	2040	154	140	140
q3	10446	1907	1889	1889
q4	10337	1250	1326	1250
q5	8428	3846	3866	3846
q6	230	126	123	123
q7	2026	1586	1607	1586
q8	9486	2678	2698	2678
q9	10427	10299	10262	10262
q10	8625	3480	3532	3480
q11	424	237	239	237
q12	464	297	303	297
q13	18346	3915	3997	3915
q14	345	336	333	333
q15	519	463	451	451
q16	662	578	567	567
q17	1130	915	981	915
q18	7166	6929	6943	6929
q19	1690	1624	1530	1530
q20	543	302	304	302
q21	4405	4032	4055	4032
q22	471	374	396	374
Total cold run time: 116246 ms
Total hot run time: 49436 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4319	4309	4278	4278
q2	320	224	215	215
q3	4223	4159	4180	4159
q4	2728	2733	2743	2733
q5	7115	7071	7060	7060
q6	234	118	119	118
q7	3256	2798	2860	2798
q8	4317	4424	4468	4424
q9	16803	16672	16615	16615
q10	4214	4301	4251	4251
q11	742	702	697	697
q12	1037	842	871	842
q13	4628	3727	3742	3727
q14	454	425	425	425
q15	498	455	456	455
q16	723	705	689	689
q17	3871	3888	3796	3796
q18	8867	8669	8666	8666
q19	1713	1667	1626	1626
q20	2345	2200	2090	2090
q21	8585	8483	8439	8439
q22	1017	963	968	963
Total cold run time: 82009 ms
Total hot run time: 79066 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 203484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e1d595561d1d60e1ee7bc88ba205f50a72a8ddc2, data reload: false

query1	923	390	383	383
query2	6518	2931	2866	2866
query3	6938	195	192	192
query4	20535	18140	18019	18019
query5	19729	6575	6515	6515
query6	282	221	234	221
query7	4165	290	305	290
query8	256	243	254	243
query9	3121	2745	2658	2658
query10	430	287	298	287
query11	11510	10825	10791	10791
query12	122	75	70	70
query13	5583	690	678	678
query14	17934	13176	13606	13176
query15	382	250	256	250
query16	6528	279	254	254
query17	1773	1454	901	901
query18	2411	414	420	414
query19	211	143	147	143
query20	87	87	87	87
query21	196	99	90	90
query22	5868	5300	5205	5205
query23	32832	31758	31845	31758
query24	6895	6497	6489	6489
query25	514	423	429	423
query26	515	158	157	157
query27	1859	294	289	289
query28	6235	2410	2364	2364
query29	2954	2887	2639	2639
query30	240	163	160	160
query31	901	742	712	712
query32	65	56	55	55
query33	390	255	237	237
query34	843	457	497	457
query35	1095	918	906	906
query36	1297	1191	1155	1155
query37	86	63	63	63
query38	3074	2938	2912	2912
query39	1371	1346	1319	1319
query40	207	91	101	91
query41	37	34	34	34
query42	81	80	89	80
query43	740	804	699	699
query44	1146	717	726	717
query45	242	228	222	222
query46	1219	977	945	945
query47	1839	1754	1657	1657
query48	1029	689	690	689
query49	623	370	372	370
query50	867	595	613	595
query51	4770	4681	4664	4664
query52	92	84	78	78
query53	444	313	309	309
query54	2689	2474	2445	2445
query55	92	74	74	74
query56	205	201	200	200
query57	1237	1057	1055	1055
query58	206	196	184	184
query59	4163	3750	4316	3750
query60	199	193	186	186
query61	86	81	79	79
query62	813	467	471	467
query63	474	331	329	329
query64	2547	1485	1472	1472
query65	3645	3579	3550	3550
query66	807	363	393	363
query67	16042	15647	16829	15647
query68	8722	642	656	642
query69	554	324	343	324
query70	1760	1450	1358	1358
query71	413	312	307	307
query72	6419	3410	3424	3410
query73	721	321	317	317
query74	6312	5839	5880	5839
query75	4935	3660	3623	3623
query76	5012	1159	1207	1159
query77	705	257	251	251
query78	12616	12681	11903	11903
query79	9408	649	651	649
query80	1110	404	385	385
query81	497	224	235	224
query82	519	96	92	92
query83	174	141	128	128
query84	253	66	67	66
query85	714	291	290	290
query86	349	291	272	272
query87	3270	3036	3090	3036
query88	4882	2296	2315	2296
query89	423	275	279	275
query90	2011	209	206	206
query91	161	142	134	134
query92	53	57	50	50
query93	4759	574	578	574
query94	763	203	204	203
query95	1109	1040	1027	1027
query96	649	323	326	323
query97	6454	6434	6298	6298
query98	180	168	166	166
query99	2839	878	943	878
Total cold run time: 311955 ms
Total hot run time: 203484 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.84 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e1d595561d1d60e1ee7bc88ba205f50a72a8ddc2, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.04
query4	1.83	0.06	0.07
query5	0.53	0.53	0.52
query6	1.24	0.63	0.61
query7	0.01	0.00	0.01
query8	0.03	0.02	0.03
query9	0.52	0.48	0.47
query10	0.55	0.55	0.53
query11	0.13	0.08	0.09
query12	0.13	0.10	0.09
query13	0.61	0.61	0.61
query14	0.78	0.79	0.78
query15	0.78	0.75	0.77
query16	0.36	0.37	0.38
query17	1.00	1.00	1.02
query18	0.23	0.24	0.26
query19	1.90	1.86	1.85
query20	0.02	0.01	0.01
query21	15.46	0.54	0.55
query22	2.17	2.29	1.73
query23	17.53	0.99	0.82
query24	7.32	1.11	1.34
query25	0.36	0.11	0.06
query26	0.75	0.18	0.15
query27	0.03	0.04	0.03
query28	5.42	0.79	0.71
query29	12.71	2.34	2.29
query30	0.56	0.55	0.52
query31	2.80	0.40	0.37
query32	3.37	0.50	0.50
query33	3.05	3.11	3.09
query34	15.26	4.77	4.81
query35	4.87	4.85	4.83
query36	1.07	1.02	1.02
query37	0.05	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.13	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 104.08 s
Total hot run time: 30.84 s

@doris-robot
Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e1d595561d1d60e1ee7bc88ba205f50a72a8ddc2 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@morningman morningman merged commit 6e72a54 into apache:branch-2.0 Apr 29, 2024
@CalvinKirs CalvinKirs deleted the branch-2.0-may-dependency branch April 29, 2024 09:12
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…e xnio to 3.8.14 (apache#34235)

We don’t need yarn and mr related dependencies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants