[Improvement](rpc) Use 1-phase commit if only 1 BE exists

[Gabriel39]

Proposed changes

2-phase execution is used to ensure both of RPC receiver and sender are ready before the query starts. But if only 1 BE is used, we do not use RPC to shuffle data .

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[Gabriel39]

run buildall

[Gabriel39]

run buildall

[doris-robot]

TPC-H: Total hot run time: 41627 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b4d0872b44e7cbf7306335c7d5934c6af0b73dc, data reload: false

------ Round 1 ----------------------------------
q1	17727	7389	7303	7303
q2	2052	159	149	149
q3	10712	1109	1240	1109
q4	10255	765	695	695
q5	7761	3085	3107	3085
q6	234	152	142	142
q7	1045	627	600	600
q8	9418	2058	2071	2058
q9	6832	6462	6458	6458
q10	7014	2291	2276	2276
q11	433	236	241	236
q12	404	210	213	210
q13	17767	2968	2934	2934
q14	247	212	200	200
q15	557	508	516	508
q16	706	610	611	610
q17	997	828	840	828
q18	7497	6669	6703	6669
q19	1409	973	1030	973
q20	586	299	293	293
q21	4083	3304	3315	3304
q22	1084	1040	987	987
Total cold run time: 108820 ms
Total hot run time: 41627 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7247	7248	7263	7248
q2	318	221	224	221
q3	3103	3019	2956	2956
q4	2053	1890	1889	1889
q5	5551	5571	5607	5571
q6	244	146	147	146
q7	2237	1819	1797	1797
q8	3334	3422	3455	3422
q9	8766	8949	8775	8775
q10	3511	3509	3533	3509
q11	575	477	473	473
q12	839	671	629	629
q13	10361	3166	3173	3166
q14	305	268	267	267
q15	557	515	525	515
q16	723	666	665	665
q17	1809	1578	1578	1578
q18	8309	7620	7998	7620
q19	1730	1632	1465	1465
q20	2152	1877	1857	1857
q21	5777	5239	5416	5239
q22	1204	1089	1060	1060
Total cold run time: 70705 ms
Total hot run time: 60068 ms

[doris-robot]

TPC-DS: Total hot run time: 197696 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b4d0872b44e7cbf7306335c7d5934c6af0b73dc, data reload: false

query1	1280	875	879	875
query2	6402	2087	1983	1983
query3	10780	3986	3900	3900
query4	61165	33193	23631	23631
query5	5372	480	462	462
query6	294	165	160	160
query7	4855	293	299	293
query8	273	219	222	219
query9	6984	2614	2617	2614
query10	419	286	279	279
query11	15753	15114	15651	15114
query12	166	101	103	101
query13	1006	423	416	416
query14	9782	6576	6364	6364
query15	216	187	180	180
query16	6758	487	510	487
query17	1132	613	613	613
query18	1506	327	318	318
query19	217	146	147	146
query20	126	115	111	111
query21	211	105	105	105
query22	5020	4776	4705	4705
query23	35040	34061	33888	33888
query24	6285	2972	2829	2829
query25	511	418	405	405
query26	641	158	159	158
query27	2288	278	283	278
query28	4997	2504	2446	2446
query29	675	436	434	434
query30	246	156	149	149
query31	921	778	779	778
query32	84	53	58	53
query33	463	298	303	298
query34	923	502	501	501
query35	818	740	716	716
query36	1058	911	931	911
query37	149	85	85	85
query38	3969	3892	3888	3888
query39	1450	1404	1400	1400
query40	210	95	95	95
query41	50	46	49	46
query42	125	98	96	96
query43	516	468	473	468
query44	1171	821	796	796
query45	196	159	165	159
query46	1102	777	773	773
query47	1905	1777	1828	1777
query48	472	363	381	363
query49	701	396	395	395
query50	818	417	417	417
query51	7061	7045	6858	6858
query52	97	85	83	83
query53	253	174	175	174
query54	560	445	450	445
query55	77	74	76	74
query56	274	258	267	258
query57	1210	1088	1079	1079
query58	220	216	221	216
query59	3181	2930	2818	2818
query60	292	266	268	266
query61	101	110	98	98
query62	752	647	653	647
query63	212	186	188	186
query64	1363	645	646	645
query65	3265	3170	3184	3170
query66	689	272	279	272
query67	16130	15688	15553	15553
query68	1149	576	568	568
query69	437	275	294	275
query70	1132	1108	1031	1031
query71	324	272	276	272
query72	6249	3911	3943	3911
query73	772	318	325	318
query74	9235	8848	8978	8848
query75	3266	2603	2648	2603
query76	1876	1099	1035	1035
query77	412	286	271	271
query78	10371	9764	9263	9263
query79	1155	900	877	877
query80	844	556	562	556
query81	502	249	292	249
query82	1058	231	222	222
query83	218	140	141	140
query84	238	104	103	103
query85	744	367	349	349
query86	322	314	315	314
query87	4423	4347	4436	4347
query88	5387	3957	3971	3957
query89	387	360	353	353
query90	1689	299	301	299
query91	155	159	157	157
query92	74	72	72	72
query93	990	898	897	897
query94	652	371	378	371
query95	443	400	401	400
query96	486	484	488	484
query97	3187	3121	3131	3121
query98	222	240	227	227
query99	1442	1286	1306	1286
Total cold run time: 303079 ms
Total hot run time: 197696 ms

[doris-robot]

ClickBench: Total hot run time: 32.11 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b4d0872b44e7cbf7306335c7d5934c6af0b73dc, data reload: false

query1	0.04	0.04	0.04
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.51	0.50	0.49
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.03	0.03	0.03
query9	0.56	0.49	0.51
query10	0.54	0.57	0.56
query11	0.14	0.10	0.11
query12	0.14	0.11	0.10
query13	0.61	0.59	0.59
query14	2.98	3.00	2.96
query15	0.88	0.82	0.82
query16	0.40	0.38	0.38
query17	1.00	1.01	1.04
query18	0.20	0.19	0.20
query19	1.98	1.79	2.00
query20	0.01	0.02	0.01
query21	15.35	0.58	0.59
query22	2.72	2.28	2.04
query23	17.29	0.88	0.84
query24	2.62	0.26	0.26
query25	0.12	0.04	0.05
query26	0.46	0.14	0.13
query27	0.04	0.04	0.04
query28	12.40	1.07	1.06
query29	12.57	3.26	3.22
query30	0.25	0.06	0.06
query31	2.88	0.36	0.38
query32	3.27	0.46	0.46
query33	2.99	2.98	3.02
query34	16.57	4.40	4.40
query35	4.42	4.40	4.44
query36	0.68	0.47	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 108.23 s
Total hot run time: 32.11 s

[Gabriel39]

run buildall

[doris-robot]

TeamCity be ut coverage result:
Function Coverage: 37.33% (9583/25670)
Line Coverage: 28.73% (79244/275871)
Region Coverage: 28.20% (41024/145500)
Branch Coverage: 24.81% (20905/84246)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e39c67a7402de7584b82c3e05eb3a4b4a706feac_e39c67a7402de7584b82c3e05eb3a4b4a706feac/report/index.html

[doris-robot]

TPC-H: Total hot run time: 41348 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e39c67a7402de7584b82c3e05eb3a4b4a706feac, data reload: false

------ Round 1 ----------------------------------
q1	17569	7463	7330	7330
q2	2052	152	163	152
q3	10583	1073	1218	1073
q4	10472	721	727	721
q5	7755	3068	3043	3043
q6	238	153	147	147
q7	1025	612	606	606
q8	9442	2035	2055	2035
q9	6772	6441	6434	6434
q10	7006	2283	2251	2251
q11	429	237	240	237
q12	402	213	214	213
q13	17786	3018	3011	3011
q14	232	213	207	207
q15	589	522	501	501
q16	675	602	599	599
q17	975	797	841	797
q18	7283	6587	6661	6587
q19	1412	962	1006	962
q20	587	287	285	285
q21	4170	3358	3168	3168
q22	1083	989	1017	989
Total cold run time: 108537 ms
Total hot run time: 41348 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7327	7295	7319	7295
q2	314	221	216	216
q3	2992	2892	2995	2892
q4	1985	1819	1839	1819
q5	5584	5597	5572	5572
q6	237	155	146	146
q7	2235	1828	1774	1774
q8	3316	3479	3408	3408
q9	8775	8949	8796	8796
q10	3551	3449	3523	3449
q11	575	480	472	472
q12	833	647	608	608
q13	11517	3180	3174	3174
q14	292	276	260	260
q15	565	528	517	517
q16	707	683	662	662
q17	1817	1589	1598	1589
q18	8178	7712	7709	7709
q19	1757	1541	1464	1464
q20	2183	1872	1861	1861
q21	5549	5327	5399	5327
q22	1149	1049	1049	1049
Total cold run time: 71438 ms
Total hot run time: 60059 ms

[doris-robot]

TPC-DS: Total hot run time: 197629 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e39c67a7402de7584b82c3e05eb3a4b4a706feac, data reload: false

query1	1280	890	875	875
query2	6406	2049	2022	2022
query3	10833	3962	3915	3915
query4	61184	29734	23469	23469
query5	5157	439	451	439
query6	410	162	167	162
query7	5397	292	299	292
query8	301	215	221	215
query9	9055	2620	2613	2613
query10	448	297	275	275
query11	17637	15424	15848	15424
query12	157	97	99	97
query13	1472	421	409	409
query14	9763	6348	6394	6348
query15	205	174	193	174
query16	6824	526	451	451
query17	1211	614	566	566
query18	1508	299	302	299
query19	214	149	150	149
query20	120	107	112	107
query21	214	100	103	100
query22	4730	4574	4726	4574
query23	34645	33891	33866	33866
query24	6885	2825	2879	2825
query25	503	421	391	391
query26	636	156	165	156
query27	2153	287	288	287
query28	5005	2457	2425	2425
query29	676	420	431	420
query30	251	151	164	151
query31	911	759	757	757
query32	72	53	52	52
query33	461	297	291	291
query34	877	485	491	485
query35	829	739	726	726
query36	1077	964	955	955
query37	150	85	86	85
query38	3992	3953	3894	3894
query39	1443	1412	1395	1395
query40	201	98	95	95
query41	48	46	48	46
query42	121	97	95	95
query43	514	480	479	479
query44	1244	833	792	792
query45	191	163	162	162
query46	1093	748	751	748
query47	1884	1782	1886	1782
query48	452	365	358	358
query49	656	371	369	369
query50	823	404	412	404
query51	7133	6903	6916	6903
query52	95	84	90	84
query53	247	178	181	178
query54	545	438	457	438
query55	74	79	76	76
query56	271	243	250	243
query57	1183	1081	1082	1081
query58	212	215	226	215
query59	3331	2889	2861	2861
query60	276	256	247	247
query61	108	101	101	101
query62	778	659	638	638
query63	213	183	178	178
query64	1344	656	609	609
query65	3283	3197	3157	3157
query66	707	280	268	268
query67	15901	15648	15340	15340
query68	1486	860	842	842
query69	440	344	350	344
query70	1126	1181	1174	1174
query71	362	337	337	337
query72	6191	3445	3365	3365
query73	581	581	588	581
query74	9136	9002	9121	9002
query75	2960	2845	2900	2845
query76	1152	859	859	859
query77	460	338	333	333
query78	9353	9169	9241	9169
query79	895	894	868	868
query80	599	545	551	545
query81	463	243	245	243
query82	237	232	240	232
query83	144	144	146	144
query84	262	110	102	102
query85	687	398	352	352
query86	308	320	306	306
query87	4397	4284	4213	4213
query88	5226	4017	3980	3980
query89	373	354	351	351
query90	1582	302	303	302
query91	162	165	166	165
query92	73	72	69	69
query93	907	896	900	896
query94	592	385	370	370
query95	416	401	402	401
query96	493	491	484	484
query97	3094	3149	3140	3140
query98	228	228	219	219
query99	1438	1313	1315	1313
Total cold run time: 303843 ms
Total hot run time: 197629 ms

[doris-robot]

ClickBench: Total hot run time: 31.57 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e39c67a7402de7584b82c3e05eb3a4b4a706feac, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.02	0.02
query3	0.23	0.07	0.06
query4	1.65	0.10	0.10
query5	0.52	0.49	0.49
query6	1.14	0.72	0.74
query7	0.02	0.02	0.01
query8	0.03	0.03	0.02
query9	0.56	0.49	0.48
query10	0.54	0.55	0.54
query11	0.15	0.10	0.10
query12	0.13	0.11	0.12
query13	0.60	0.59	0.58
query14	3.04	3.06	2.94
query15	0.92	0.82	0.82
query16	0.39	0.39	0.40
query17	1.07	0.99	1.02
query18	0.20	0.20	0.19
query19	1.87	1.82	2.06
query20	0.01	0.01	0.02
query21	15.36	0.57	0.59
query22	2.35	2.89	1.80
query23	17.28	0.86	0.69
query24	2.98	0.27	0.13
query25	0.16	0.04	0.04
query26	0.24	0.13	0.14
query27	0.05	0.03	0.03
query28	12.28	1.10	1.06
query29	12.52	3.25	3.23
query30	0.24	0.06	0.06
query31	2.88	0.38	0.38
query32	3.26	0.46	0.47
query33	3.01	3.03	3.06
query34	16.71	4.45	4.39
query35	4.44	4.47	4.38
query36	0.65	0.48	0.49
query37	0.09	0.06	0.05
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 108.05 s
Total hot run time: 31.57 s

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.

[HappenLee]

here must keep fragment ready order， should think we really need to do the work