[fix](build) Fix JindoFS jar packaging by CalvinKirs · Pull Request #61940 · apache/doris

CalvinKirs · 2026-03-31T08:36:24Z

#60856
Problem Summary: Fix JindoFS packaging so FE/BE copy all common jars from jindofs_libs and only add the matching platform jar, instead of relying on a brittle hard-coded whitelist.

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Fix JindoFS packaging so FE/BE copy all common jars from jindofs_libs and only add the matching platform jar, instead of relying on a brittle hard-coded whitelist. ### Release note None ### Check List (For Author) - Test: Manual test - bash docker/thirdparties/test-jindofs-helpers.sh - bash -n build.sh - bash -n docker/thirdparties/jindofs-helpers.sh - bash -n docker/thirdparties/test-jindofs-helpers.sh - Behavior changed: Yes (JindoFS packaging now includes all common jars and the matching platform jar) - Does this need documentation: No

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Refine JindoFS packaging helpers so platform-specific jars are configured via wildcard pattern lists and common jars are copied without assuming a numeric version prefix. ### Release note None ### Check List (For Author) - Test: Manual test - bash docker/thirdparties/test-jindofs-helpers.sh - bash -n docker/thirdparties/jindofs-helpers.sh - bash -n docker/thirdparties/test-jindofs-helpers.sh - Behavior changed: No - Does this need documentation: No

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Remove the standalone JindoFS helper test script from the branch while keeping the JindoFS packaging helper and build changes intact. ### Release note None ### Check List (For Author) - Test: Manual test - bash -n build.sh - bash -n docker/thirdparties/jindofs-helpers.sh - Verified jindofs_copy_jars against current thirdparty/installed/jindofs_libs on Linux aarch64 - Behavior changed: No - Does this need documentation: No

Thearas · 2026-03-31T08:36:30Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

CalvinKirs · 2026-03-31T08:37:21Z

run buildall

doris-robot · 2026-03-31T09:18:01Z

TPC-H: Total hot run time: 26620 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 204963051c74b0a4d312416ee3cd7e3ead469b99, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17644	4471	4286	4286
q2	q3	10658	798	539	539
q4	4714	365	252	252
q5	7997	1215	1006	1006
q6	243	177	146	146
q7	832	842	682	682
q8	10617	1550	1384	1384
q9	6695	4738	4774	4738
q10	6298	1924	1640	1640
q11	468	258	240	240
q12	692	593	466	466
q13	18039	2754	1967	1967
q14	234	239	219	219
q15	q16	733	771	666	666
q17	757	868	440	440
q18	6077	5382	5247	5247
q19	1133	1008	616	616
q20	545	508	387	387
q21	4450	1873	1449	1449
q22	354	286	250	250
Total cold run time: 99180 ms
Total hot run time: 26620 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4443	4359	4388	4359
q2	q3	3861	4327	3778	3778
q4	869	1173	771	771
q5	4056	4410	4323	4323
q6	184	177	142	142
q7	1768	1622	1504	1504
q8	2424	2672	2523	2523
q9	7337	7163	7087	7087
q10	3754	3911	3505	3505
q11	500	432	412	412
q12	490	576	438	438
q13	2389	2746	1937	1937
q14	272	287	265	265
q15	q16	699	768	692	692
q17	1108	1274	1241	1241
q18	7168	6737	6493	6493
q19	898	906	931	906
q20	2004	2104	1944	1944
q21	3956	3508	3295	3295
q22	444	437	382	382
Total cold run time: 48624 ms
Total hot run time: 45997 ms

doris-robot · 2026-03-31T09:28:36Z

TPC-DS: Total hot run time: 167520 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 204963051c74b0a4d312416ee3cd7e3ead469b99, data reload: false

query5	4328	625	509	509
query6	329	231	216	216
query7	4207	464	262	262
query8	327	237	219	219
query9	8610	2760	2733	2733
query10	511	428	355	355
query11	7000	5111	4854	4854
query12	183	134	127	127
query13	1282	476	354	354
query14	4014	3728	3441	3441
query14_1	2856	2823	2828	2823
query15	201	196	180	180
query16	624	496	376	376
query17	708	751	591	591
query18	447	440	337	337
query19	205	205	174	174
query20	135	127	124	124
query21	130	134	116	116
query22	13287	13424	13200	13200
query23	16274	15910	15515	15515
query23_1	15710	15659	15580	15580
query24	7310	1616	1231	1231
query24_1	1219	1238	1223	1223
query25	523	466	424	424
query26	1227	270	160	160
query27	2759	475	309	309
query28	4490	1840	1853	1840
query29	815	578	473	473
query30	303	228	193	193
query31	1041	984	873	873
query32	87	76	71	71
query33	512	345	285	285
query34	898	877	534	534
query35	649	695	618	618
query36	1141	1131	997	997
query37	141	100	92	92
query38	2982	2922	2888	2888
query39	842	857	802	802
query39_1	812	794	791	791
query40	235	153	139	139
query41	67	62	76	62
query42	268	263	260	260
query43	251	267	227	227
query44	
query45	205	195	185	185
query46	896	982	606	606
query47	2122	2876	2104	2104
query48	308	320	235	235
query49	628	471	385	385
query50	724	308	217	217
query51	4046	4063	4041	4041
query52	273	271	266	266
query53	299	341	288	288
query54	304	275	264	264
query55	92	92	84	84
query56	327	336	311	311
query57	1857	1578	1609	1578
query58	279	274	273	273
query59	2784	2947	2745	2745
query60	346	346	330	330
query61	156	154	153	153
query62	617	598	542	542
query63	310	287	274	274
query64	4936	1301	999	999
query65	
query66	1460	463	352	352
query67	24259	24337	24141	24141
query68	
query69	401	326	283	283
query70	1028	978	947	947
query71	337	307	295	295
query72	2842	2705	2424	2424
query73	542	557	331	331
query74	9678	9558	9411	9411
query75	2912	2788	2481	2481
query76	2272	1060	679	679
query77	394	411	335	335
query78	11067	11076	10454	10454
query79	1113	777	569	569
query80	731	650	585	585
query81	484	263	231	231
query82	1204	162	124	124
query83	349	285	256	256
query84	298	135	111	111
query85	955	577	535	535
query86	384	310	256	256
query87	3169	3102	3049	3049
query88	3637	2684	2719	2684
query89	440	374	353	353
query90	1835	193	192	192
query91	212	163	138	138
query92	84	77	72	72
query93	934	851	510	510
query94	463	315	292	292
query95	600	354	327	327
query96	663	530	231	231
query97	2469	2522	2418	2418
query98	237	221	224	221
query99	1018	1007	910	910
Total cold run time: 241265 ms
Total hot run time: 167520 ms

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Print the JindoFS source jar inventory before FE and BE packaging so build logs show how many jars were available and which jar names were copied from thirdparty/installed/jindofs_libs. ### Release note None ### Check List (For Author) - Test: Manual test - Manual test: - \ - \ for FE/BE temp targets - Behavior changed: Yes (build logs now include JindoFS source jar count and names before copy) - Does this need documentation: No

CalvinKirs · 2026-03-31T11:34:56Z

run buildall

doris-robot · 2026-03-31T11:50:49Z

TPC-H: Total hot run time: 26516 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ae82295507f40941f915cddb27f928d0c4e18e8e, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17616	4425	4285	4285
q2	q3	10710	791	540	540
q4	4683	358	254	254
q5	7557	1202	1028	1028
q6	177	174	148	148
q7	770	858	667	667
q8	9294	1468	1334	1334
q9	4807	4757	4703	4703
q10	6238	1899	1640	1640
q11	472	256	243	243
q12	719	575	471	471
q13	18033	2735	1980	1980
q14	223	234	225	225
q15	q16	730	748	666	666
q17	736	862	424	424
q18	5874	5421	5243	5243
q19	1102	995	647	647
q20	530	499	383	383
q21	4454	1833	1385	1385
q22	343	297	250	250
Total cold run time: 95068 ms
Total hot run time: 26516 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4942	4707	4682	4682
q2	q3	3924	4419	3852	3852
q4	899	1200	809	809
q5	4077	4452	4369	4369
q6	192	171	143	143
q7	1783	1711	1596	1596
q8	2481	2736	2540	2540
q9	7679	7479	7376	7376
q10	3790	4023	3652	3652
q11	510	435	410	410
q12	489	589	457	457
q13	2537	3199	2118	2118
q14	293	302	282	282
q15	q16	724	788	736	736
q17	1154	1391	1374	1374
q18	7057	6901	6597	6597
q19	979	933	944	933
q20	2137	2203	2069	2069
q21	3994	3561	3370	3370
q22	471	437	385	385
Total cold run time: 50112 ms
Total hot run time: 47750 ms

doris-robot · 2026-03-31T12:01:39Z

TPC-DS: Total hot run time: 169432 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ae82295507f40941f915cddb27f928d0c4e18e8e, data reload: false

query5	4330	643	503	503
query6	345	237	223	223
query7	4214	480	269	269
query8	353	254	235	235
query9	8714	2739	2732	2732
query10	537	403	322	322
query11	6997	5144	4908	4908
query12	186	132	124	124
query13	1273	461	350	350
query14	5735	3792	3488	3488
query14_1	2879	2878	2888	2878
query15	217	194	177	177
query16	977	471	466	466
query17	927	730	648	648
query18	2461	457	362	362
query19	223	220	192	192
query20	137	129	125	125
query21	220	135	112	112
query22	13474	14189	14781	14189
query23	16802	16255	16030	16030
query23_1	16090	15724	15847	15724
query24	7222	1662	1291	1291
query24_1	1239	1229	1251	1229
query25	550	479	410	410
query26	1243	276	150	150
query27	2761	490	291	291
query28	4511	1853	1873	1853
query29	851	563	480	480
query30	302	231	194	194
query31	1021	957	869	869
query32	85	75	73	73
query33	505	340	283	283
query34	921	867	534	534
query35	642	708	601	601
query36	1069	1139	985	985
query37	131	97	83	83
query38	2934	2913	2866	2866
query39	861	843	819	819
query39_1	800	795	802	795
query40	234	151	139	139
query41	65	60	61	60
query42	266	260	254	254
query43	238	261	227	227
query44	
query45	197	188	182	182
query46	882	988	613	613
query47	2452	3051	1973	1973
query48	323	317	230	230
query49	639	462	385	385
query50	701	276	215	215
query51	4059	4098	3992	3992
query52	265	275	260	260
query53	302	345	284	284
query54	302	265	264	264
query55	96	93	88	88
query56	361	327	308	308
query57	1939	1714	1575	1575
query58	284	275	270	270
query59	2821	2993	2762	2762
query60	346	339	323	323
query61	159	151	159	151
query62	647	582	525	525
query63	310	283	284	283
query64	5082	1292	1039	1039
query65	
query66	1465	472	365	365
query67	24356	24402	24283	24283
query68	
query69	409	331	294	294
query70	1005	915	960	915
query71	346	304	294	294
query72	2877	2664	2551	2551
query73	544	545	344	344
query74	9666	9593	9455	9455
query75	2918	2753	2480	2480
query76	2302	1055	668	668
query77	369	396	321	321
query78	11028	11158	10448	10448
query79	1149	791	589	589
query80	1345	619	535	535
query81	555	263	230	230
query82	1167	157	125	125
query83	345	258	248	248
query84	303	120	102	102
query85	927	478	449	449
query86	439	310	318	310
query87	3205	3159	3036	3036
query88	3576	2645	2650	2645
query89	446	376	359	359
query90	2004	181	181	181
query91	166	162	142	142
query92	79	76	70	70
query93	939	843	509	509
query94	647	318	307	307
query95	591	351	326	326
query96	641	519	227	227
query97	2461	2532	2390	2390
query98	245	219	230	219
query99	1011	971	903	903
Total cold run time: 251883 ms
Total hot run time: 169432 ms

morningman

LGTM

github-actions · 2026-04-01T02:30:07Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-04-01T02:30:10Z

PR approved by anyone and no changes requested.

#60856 Problem Summary: Fix JindoFS packaging so FE/BE copy all common jars from jindofs_libs and only add the matching platform jar, instead of relying on a brittle hard-coded whitelist.

CalvinKirs added 3 commits March 31, 2026 16:18

CalvinKirs added the dev/4.1.x label Mar 31, 2026

morningman approved these changes Apr 1, 2026

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Apr 1, 2026

hello-stephen approved these changes Apr 1, 2026

View reviewed changes

CalvinKirs merged commit 70faf2a into apache:master Apr 1, 2026
29 of 30 checks passed

CalvinKirs deleted the rich-master-fix-jindofs-copy-20260331 branch April 1, 2026 02:34

github-actions bot mentioned this pull request Apr 1, 2026

branch-4.1: [fix](build) Fix JindoFS jar packaging #61940 #61978

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](build) Fix JindoFS jar packaging#61940

[fix](build) Fix JindoFS jar packaging#61940
CalvinKirs merged 4 commits intoapache:masterfrom
CalvinKirs:rich-master-fix-jindofs-copy-20260331

CalvinKirs commented Mar 31, 2026 •

edited

Loading

Uh oh!

Thearas commented Mar 31, 2026

Uh oh!

CalvinKirs commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

CalvinKirs commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

morningman left a comment

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

CalvinKirs commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thearas commented Mar 31, 2026

Uh oh!

CalvinKirs commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

CalvinKirs commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CalvinKirs commented Mar 31, 2026 •

edited

Loading