Skip to content

Conversation

@TsukiokaKogane
Copy link
Contributor

@TsukiokaKogane TsukiokaKogane commented Feb 5, 2026

What problem does this PR solve?

Issue Number: close #60295

Related PR: #xxx

Problem Summary:

  1. support insert & insert overwrite on remote doris catalog
  2. support temporary partitions on remote doris catalog
  3. fix remote doris catalog rpc timeout unit
  4. DO NOT support txn insert & group commit

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@TsukiokaKogane
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.29% (1792/2260)
Line Coverage 64.74% (31825/49158)
Region Coverage 65.41% (15882/24280)
Branch Coverage 55.94% (8435/15078)

@doris-robot
Copy link

TPC-H: Total hot run time: 31471 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 97168897275f4683257ec006a83a0f0b792a824d, data reload: false

------ Round 1 ----------------------------------
q1	17660	4522	4407	4407
q2	2051	349	224	224
q3	10469	1342	767	767
q4	10343	809	329	329
q5	9507	2263	1927	1927
q6	228	182	156	156
q7	883	750	590	590
q8	9266	1447	1221	1221
q9	5356	4836	4890	4836
q10	6888	1959	1582	1582
q11	503	304	290	290
q12	398	377	231	231
q13	17813	4066	3274	3274
q14	231	246	214	214
q15	924	811	804	804
q16	689	684	621	621
q17	708	767	547	547
q18	6755	6412	6330	6330
q19	1326	1004	654	654
q20	395	357	256	256
q21	2709	2004	1937	1937
q22	358	318	274	274
Total cold run time: 105460 ms
Total hot run time: 31471 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4442	4333	4374	4333
q2	257	330	265	265
q3	2137	2601	2229	2229
q4	1372	1735	1327	1327
q5	4327	4207	4182	4182
q6	216	184	140	140
q7	1884	2156	1917	1917
q8	2654	2501	2431	2431
q9	7635	7459	7620	7459
q10	2784	3063	2750	2750
q11	570	503	455	455
q12	684	752	658	658
q13	3851	4494	3582	3582
q14	305	302	288	288
q15	851	802	793	793
q16	694	720	728	720
q17	1198	1489	1385	1385
q18	8328	8094	7920	7920
q19	936	920	914	914
q20	2119	2144	1962	1962
q21	4524	4352	4109	4109
q22	544	564	504	504
Total cold run time: 52312 ms
Total hot run time: 50323 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 97168897275f4683257ec006a83a0f0b792a824d, data reload: false

query1	0.05	0.05	0.05
query2	0.13	0.07	0.07
query3	0.32	0.07	0.07
query4	1.61	0.10	0.10
query5	0.25	0.25	0.24
query6	1.15	0.66	0.64
query7	0.03	0.02	0.02
query8	0.08	0.06	0.06
query9	0.59	0.50	0.49
query10	0.54	0.56	0.54
query11	0.27	0.13	0.14
query12	0.27	0.14	0.14
query13	0.64	0.63	0.60
query14	0.98	0.98	0.98
query15	0.94	0.83	0.83
query16	0.39	0.40	0.39
query17	1.06	1.04	1.05
query18	0.26	0.23	0.23
query19	1.84	1.83	1.79
query20	0.02	0.01	0.02
query21	15.38	0.34	0.29
query22	4.98	0.11	0.12
query23	15.36	0.45	0.28
query24	2.30	0.56	0.39
query25	0.11	0.11	0.11
query26	0.19	0.18	0.18
query27	0.10	0.10	0.10
query28	3.62	1.14	0.98
query29	12.52	4.12	3.31
query30	0.34	0.12	0.11
query31	2.79	0.70	0.44
query32	3.24	0.64	0.51
query33	2.95	3.11	3.04
query34	16.30	5.07	4.46
query35	4.47	4.47	4.48
query36	0.62	0.50	0.50
query37	0.30	0.10	0.09
query38	0.26	0.06	0.06
query39	0.08	0.05	0.04
query40	0.22	0.17	0.18
query41	0.13	0.08	0.06
query42	0.08	0.05	0.05
query43	0.06	0.06	0.06
Total cold run time: 97.82 s
Total hot run time: 28.23 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 13.33% (2/15) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.56% (19376/36862)
Line Coverage 36.06% (180041/499291)
Region Coverage 32.42% (139548/430471)
Branch Coverage 33.42% (60437/180846)

@TsukiokaKogane
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.29% (1792/2260)
Line Coverage 64.75% (31831/49158)
Region Coverage 65.42% (15884/24280)
Branch Coverage 55.97% (8439/15078)

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 2.44% (29/1189) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-H: Total hot run time: 31323 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9f3938347c127a752b04e13d9f40db00f84820c1, data reload: false

------ Round 1 ----------------------------------
q1	17617	4508	4290	4290
q2	2054	355	238	238
q3	10153	1364	770	770
q4	10211	824	325	325
q5	7499	2248	1990	1990
q6	193	178	147	147
q7	909	751	600	600
q8	9262	1405	1161	1161
q9	4847	4686	4653	4653
q10	6857	1960	1548	1548
q11	538	322	304	304
q12	336	387	230	230
q13	17778	4095	3252	3252
q14	241	234	218	218
q15	881	817	798	798
q16	672	681	642	642
q17	708	861	520	520
q18	6624	6218	6365	6218
q19	1201	1083	688	688
q20	575	545	421	421
q21	2924	2019	2068	2019
q22	380	335	291	291
Total cold run time: 102460 ms
Total hot run time: 31323 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4618	4530	4678	4530
q2	264	342	272	272
q3	2466	2879	2389	2389
q4	1520	1937	1503	1503
q5	4636	4691	4697	4691
q6	228	186	139	139
q7	1953	1882	1745	1745
q8	2553	2367	2398	2367
q9	7691	7717	7420	7420
q10	2944	3124	2614	2614
q11	540	482	468	468
q12	675	730	617	617
q13	3818	4335	3613	3613
q14	290	296	269	269
q15	825	775	782	775
q16	661	690	636	636
q17	1096	1274	1285	1274
q18	7534	7362	7433	7362
q19	897	847	841	841
q20	1977	2104	1878	1878
q21	4597	4245	4078	4078
q22	560	544	506	506
Total cold run time: 52343 ms
Total hot run time: 49987 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9f3938347c127a752b04e13d9f40db00f84820c1, data reload: false

query1	0.05	0.05	0.05
query2	0.13	0.07	0.07
query3	0.32	0.08	0.08
query4	1.61	0.10	0.10
query5	0.28	0.25	0.25
query6	1.15	0.65	0.65
query7	0.03	0.03	0.02
query8	0.07	0.06	0.06
query9	0.58	0.51	0.50
query10	0.55	0.56	0.54
query11	0.27	0.14	0.14
query12	0.26	0.14	0.15
query13	0.62	0.60	0.60
query14	0.98	0.99	0.98
query15	0.89	0.82	0.83
query16	0.45	0.39	0.39
query17	0.99	1.08	0.97
query18	0.25	0.23	0.23
query19	1.94	1.85	1.88
query20	0.02	0.02	0.01
query21	15.40	0.34	0.28
query22	4.92	0.11	0.12
query23	15.34	0.44	0.27
query24	2.34	0.56	0.37
query25	0.11	0.10	0.11
query26	0.19	0.18	0.19
query27	0.11	0.11	0.10
query28	3.62	1.16	0.99
query29	12.54	4.13	3.34
query30	0.33	0.13	0.11
query31	2.79	0.70	0.45
query32	3.24	0.62	0.50
query33	3.00	3.06	3.01
query34	16.27	5.10	4.42
query35	4.53	4.91	5.05
query36	0.69	0.58	0.55
query37	0.33	0.08	0.09
query38	0.28	0.06	0.06
query39	0.08	0.05	0.05
query40	0.20	0.18	0.16
query41	0.14	0.08	0.07
query42	0.09	0.05	0.05
query43	0.06	0.06	0.05
Total cold run time: 98.04 s
Total hot run time: 28.66 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 13.33% (2/15) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.61% (19409/36895)
Line Coverage 36.12% (180587/499982)
Region Coverage 32.47% (140040/431269)
Branch Coverage 33.48% (60631/181101)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 73.33% (11/15) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.72% (25930/36156)
Line Coverage 54.39% (271270/498758)
Region Coverage 51.96% (226378/435653)
Branch Coverage 53.36% (97020/181809)

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 29.44% (350/1189) 🎉
Increment coverage report
Complete coverage report

@morningman morningman self-assigned this Feb 9, 2026
@TsukiokaKogane
Copy link
Contributor Author

run buildall

@morningman morningman changed the title [Feature] doris cross-cluster insert [feat](doris-catalog) doris cross-cluster insert Feb 9, 2026
@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.35% (1795/2262)
Line Coverage 64.87% (31962/49269)
Region Coverage 65.53% (15940/24323)
Branch Coverage 56.09% (8478/15114)

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 2.02% (24/1189) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-H: Total hot run time: 30323 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0c8af71fa683293df3e943ebd237bb3029c468ca, data reload: false

------ Round 1 ----------------------------------
q1	17633	4474	4351	4351
q2	2042	347	228	228
q3	10642	1375	735	735
q4	10370	779	315	315
q5	9658	2230	1932	1932
q6	238	183	143	143
q7	896	735	613	613
q8	9272	1398	1171	1171
q9	4850	4679	4637	4637
q10	6852	1985	1557	1557
q11	515	320	288	288
q12	404	381	223	223
q13	17793	4060	3237	3237
q14	245	245	212	212
q15	905	828	802	802
q16	683	679	622	622
q17	924	815	530	530
q18	7147	5838	5631	5631
q19	1251	997	619	619
q20	499	505	386	386
q21	2585	1866	1810	1810
q22	368	324	281	281
Total cold run time: 105772 ms
Total hot run time: 30323 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4469	4385	4366	4366
q2	275	332	257	257
q3	2076	2704	2231	2231
q4	1345	1756	1301	1301
q5	4287	4200	4200	4200
q6	219	182	137	137
q7	1853	1816	2087	1816
q8	2669	2417	2380	2380
q9	7576	7570	7535	7535
q10	2878	3171	2621	2621
q11	546	480	481	480
q12	742	759	596	596
q13	3898	4551	3681	3681
q14	289	296	298	296
q15	841	796	806	796
q16	675	829	858	829
q17	1187	1279	1323	1279
q18	8252	7949	7857	7857
q19	905	918	872	872
q20	2074	2121	2006	2006
q21	4709	4380	4146	4146
q22	582	546	509	509
Total cold run time: 52347 ms
Total hot run time: 50191 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0c8af71fa683293df3e943ebd237bb3029c468ca, data reload: false

query1	0.05	0.05	0.05
query2	0.14	0.07	0.07
query3	0.32	0.08	0.08
query4	1.61	0.10	0.09
query5	0.27	0.24	0.24
query6	1.14	0.64	0.64
query7	0.03	0.03	0.03
query8	0.07	0.05	0.07
query9	0.58	0.51	0.50
query10	0.54	0.55	0.55
query11	0.26	0.13	0.14
query12	0.27	0.15	0.15
query13	0.62	0.61	0.61
query14	0.98	0.97	0.98
query15	0.91	0.82	0.81
query16	0.39	0.40	0.40
query17	1.06	0.99	0.99
query18	0.24	0.25	0.23
query19	2.00	1.87	1.86
query20	0.02	0.02	0.01
query21	15.40	0.35	0.30
query22	4.93	0.13	0.12
query23	15.38	0.45	0.29
query24	2.36	0.59	0.41
query25	0.11	0.11	0.12
query26	0.20	0.18	0.19
query27	0.11	0.12	0.11
query28	3.58	1.17	0.98
query29	12.51	4.07	3.29
query30	0.33	0.12	0.11
query31	2.80	0.68	0.44
query32	3.24	0.63	0.50
query33	3.00	3.03	3.13
query34	16.33	5.09	4.45
query35	4.49	4.44	4.43
query36	0.62	0.49	0.49
query37	0.31	0.08	0.08
query38	0.28	0.06	0.06
query39	0.08	0.05	0.05
query40	0.21	0.17	0.17
query41	0.13	0.07	0.08
query42	0.09	0.05	0.05
query43	0.06	0.06	0.05
Total cold run time: 98.05 s
Total hot run time: 28.23 s

@TsukiokaKogane
Copy link
Contributor Author

run buildall

@TsukiokaKogane
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.35% (1795/2262)
Line Coverage 64.88% (31968/49269)
Region Coverage 65.60% (15956/24323)
Branch Coverage 56.12% (8482/15114)

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 1.89% (24/1272) 🎉
Increment coverage report
Complete coverage report

@TsukiokaKogane
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.35% (1795/2262)
Line Coverage 64.86% (31956/49269)
Region Coverage 65.53% (15940/24323)
Branch Coverage 56.08% (8476/15114)

@doris-robot
Copy link

TPC-H: Total hot run time: 30363 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef0c94e649ed972209a13ed67e7e65fa49ea4dc9, data reload: false

------ Round 1 ----------------------------------
q1	17591	4505	4291	4291
q2	2034	341	254	254
q3	10165	1262	733	733
q4	10201	767	311	311
q5	7540	2226	1918	1918
q6	203	185	157	157
q7	879	740	611	611
q8	9268	1390	1117	1117
q9	4666	4636	4614	4614
q10	6772	1932	1567	1567
q11	495	309	280	280
q12	333	375	227	227
q13	17757	4049	3278	3278
q14	231	240	209	209
q15	884	814	815	814
q16	684	656	625	625
q17	690	798	550	550
q18	6620	5960	5720	5720
q19	1237	985	627	627
q20	498	490	402	402
q21	2699	1848	1781	1781
q22	356	327	277	277
Total cold run time: 101803 ms
Total hot run time: 30363 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4369	4306	4359	4306
q2	265	347	264	264
q3	2105	2712	2197	2197
q4	1370	1750	1286	1286
q5	4336	4160	4249	4160
q6	214	181	134	134
q7	1857	1765	1648	1648
q8	2463	2703	2453	2453
q9	7680	7711	7461	7461
q10	2900	3055	2692	2692
q11	538	469	459	459
q12	722	770	632	632
q13	3958	4401	3503	3503
q14	274	297	280	280
q15	1004	838	786	786
q16	693	726	677	677
q17	1151	1293	1348	1293
q18	8109	8038	8014	8014
q19	855	896	901	896
q20	2096	2149	2024	2024
q21	5148	4510	4381	4381
q22	537	549	501	501
Total cold run time: 52644 ms
Total hot run time: 50047 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.03 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef0c94e649ed972209a13ed67e7e65fa49ea4dc9, data reload: false

query1	0.06	0.05	0.04
query2	0.13	0.07	0.07
query3	0.31	0.08	0.07
query4	1.61	0.10	0.10
query5	0.27	0.25	0.27
query6	1.14	0.65	0.64
query7	0.03	0.03	0.03
query8	0.07	0.06	0.06
query9	0.58	0.50	0.48
query10	0.54	0.56	0.55
query11	0.26	0.13	0.14
query12	0.26	0.14	0.17
query13	0.61	0.62	0.59
query14	0.97	0.97	0.98
query15	0.91	0.82	0.82
query16	0.38	0.38	0.38
query17	1.05	1.08	1.02
query18	0.23	0.22	0.23
query19	1.88	1.81	1.82
query20	0.02	0.02	0.01
query21	15.39	0.33	0.31
query22	4.94	0.13	0.12
query23	15.34	0.46	0.28
query24	2.24	0.56	0.38
query25	0.11	0.10	0.10
query26	0.19	0.18	0.18
query27	0.10	0.10	0.11
query28	3.51	1.17	0.98
query29	12.50	4.02	3.24
query30	0.32	0.12	0.11
query31	2.80	0.66	0.43
query32	3.24	0.63	0.50
query33	3.07	3.01	3.08
query34	16.33	5.04	4.45
query35	4.49	4.50	4.46
query36	0.61	0.50	0.48
query37	0.30	0.09	0.08
query38	0.28	0.06	0.06
query39	0.08	0.05	0.05
query40	0.19	0.18	0.17
query41	0.13	0.07	0.06
query42	0.08	0.06	0.05
query43	0.07	0.05	0.05
Total cold run time: 97.62 s
Total hot run time: 28.03 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 13.33% (2/15) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.72% (19445/36883)
Line Coverage 36.21% (181027/499913)
Region Coverage 32.60% (140562/431180)
Branch Coverage 33.63% (60883/181057)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] doris cross-cluster insert

4 participants