Skip to content

branch-3.0: [fix](ccr) Wrong tosql for alt tbl enable feature #46620#47208

Merged
dataroaring merged 1 commit intoapache:branch-3.0from
wyxxxcat:pick-46620-branch-3.0
Jan 21, 2025
Merged

branch-3.0: [fix](ccr) Wrong tosql for alt tbl enable feature #46620#47208
dataroaring merged 1 commit intoapache:branch-3.0from
wyxxxcat:pick-46620-branch-3.0

Conversation

@wyxxxcat
Copy link
Contributor

pick: #46620

@wyxxxcat wyxxxcat requested a review from dataroaring as a code owner January 19, 2025 13:08
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wyxxxcat
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41051 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 162dfa7dbbffcafc0b9a2161b9709708c9c7caa3, data reload: false

------ Round 1 ----------------------------------
q1	17589	7388	7255	7255
q2	2057	175	199	175
q3	10601	1087	1159	1087
q4	10556	715	712	712
q5	7769	2836	2850	2836
q6	241	148	147	147
q7	967	611	605	605
q8	9368	1951	1989	1951
q9	6553	6377	6395	6377
q10	6979	2325	2340	2325
q11	478	267	263	263
q12	398	218	207	207
q13	17845	3121	3127	3121
q14	239	213	217	213
q15	566	544	525	525
q16	706	622	625	622
q17	976	565	558	558
q18	7278	6677	6709	6677
q19	1392	1040	1022	1022
q20	445	204	194	194
q21	4231	3301	3200	3200
q22	1135	979	1003	979
Total cold run time: 108369 ms
Total hot run time: 41051 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7329	7212	7188	7188
q2	332	230	229	229
q3	2895	2943	2979	2943
q4	2080	1815	1803	1803
q5	5729	5703	5738	5703
q6	220	147	146	146
q7	2201	1834	1816	1816
q8	3344	3607	3508	3508
q9	8726	8833	8743	8743
q10	3556	3532	3532	3532
q11	601	513	504	504
q12	811	592	621	592
q13	9135	3194	3151	3151
q14	306	281	276	276
q15	584	548	516	516
q16	744	698	685	685
q17	1807	1610	1591	1591
q18	8083	7569	7478	7478
q19	1653	1528	1542	1528
q20	2098	1884	1837	1837
q21	5709	5312	5245	5245
q22	1205	1047	1037	1037
Total cold run time: 69148 ms
Total hot run time: 60051 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197464 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 162dfa7dbbffcafc0b9a2161b9709708c9c7caa3, data reload: false

query1	1324	911	948	911
query2	6247	2101	2062	2062
query3	10852	4395	4296	4296
query4	66648	28885	23628	23628
query5	4933	453	442	442
query6	392	174	167	167
query7	5475	315	311	311
query8	304	221	220	220
query9	8344	2667	2659	2659
query10	424	266	272	266
query11	17241	15185	15817	15185
query12	161	103	105	103
query13	1450	453	418	418
query14	10901	7026	7763	7026
query15	200	187	188	187
query16	7194	489	496	489
query17	1107	616	595	595
query18	1865	325	340	325
query19	212	160	160	160
query20	115	117	107	107
query21	214	116	106	106
query22	4847	4795	4614	4614
query23	34715	34047	34227	34047
query24	6189	2924	2952	2924
query25	526	428	420	420
query26	674	174	170	170
query27	1925	359	363	359
query28	4275	2510	2474	2474
query29	735	497	458	458
query30	230	164	165	164
query31	986	818	818	818
query32	68	55	56	55
query33	396	286	291	286
query34	908	498	520	498
query35	857	721	738	721
query36	1108	966	957	957
query37	117	80	74	74
query38	4127	4123	4122	4122
query39	1541	1485	1478	1478
query40	208	101	97	97
query41	51	49	47	47
query42	116	100	100	100
query43	533	488	485	485
query44	1191	837	838	837
query45	186	167	171	167
query46	1166	739	715	715
query47	2032	1936	2000	1936
query48	482	386	383	383
query49	720	390	385	385
query50	841	429	422	422
query51	7314	7176	7088	7088
query52	99	90	88	88
query53	261	182	183	182
query54	547	462	445	445
query55	75	82	76	76
query56	260	249	265	249
query57	1240	1143	1103	1103
query58	208	200	209	200
query59	3025	2842	2839	2839
query60	286	249	243	243
query61	171	108	103	103
query62	770	655	669	655
query63	214	193	187	187
query64	1360	683	642	642
query65	3252	3169	3185	3169
query66	647	296	311	296
query67	16070	15707	15622	15622
query68	3965	585	563	563
query69	424	263	264	263
query70	1182	1091	1109	1091
query71	319	262	260	260
query72	6402	4081	3966	3966
query73	748	340	344	340
query74	10241	9081	9039	9039
query75	3330	2653	2609	2609
query76	1870	1080	955	955
query77	487	277	265	265
query78	10509	9612	9607	9607
query79	2412	588	597	588
query80	1375	422	414	414
query81	531	242	237	237
query82	1272	118	116	116
query83	161	149	146	146
query84	286	78	75	75
query85	982	304	286	286
query86	448	306	297	297
query87	4461	4269	4281	4269
query88	4051	2384	2353	2353
query89	416	291	287	287
query90	1933	182	188	182
query91	179	151	148	148
query92	65	49	50	49
query93	3134	543	557	543
query94	901	299	258	258
query95	360	257	252	252
query96	624	280	282	280
query97	3353	3157	3155	3155
query98	223	197	195	195
query99	1579	1315	1313	1313
Total cold run time: 321375 ms
Total hot run time: 197464 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.68 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 162dfa7dbbffcafc0b9a2161b9709708c9c7caa3, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.52	0.50	0.51
query6	1.13	0.74	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.51	0.50
query10	0.53	0.54	0.60
query11	0.14	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.59	0.59
query14	2.95	2.94	2.92
query15	0.89	0.82	0.81
query16	0.37	0.37	0.36
query17	1.07	1.05	1.05
query18	0.24	0.23	0.22
query19	1.91	1.91	2.00
query20	0.01	0.01	0.01
query21	15.35	0.60	0.60
query22	2.27	2.20	1.96
query23	16.89	1.10	0.93
query24	3.25	1.12	2.22
query25	0.28	0.09	0.16
query26	0.54	0.15	0.14
query27	0.04	0.05	0.05
query28	9.66	1.13	1.07
query29	12.57	3.27	3.27
query30	0.24	0.06	0.05
query31	2.88	0.39	0.39
query32	3.22	0.46	0.45
query33	3.04	3.02	3.04
query34	17.28	4.55	4.55
query35	4.53	4.60	4.60
query36	0.67	0.48	0.48
query37	0.10	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.28 s
Total hot run time: 33.68 s

@wyxxxcat wyxxxcat force-pushed the pick-46620-branch-3.0 branch from 162dfa7 to 8b2f2c1 Compare January 20, 2025 01:48
@wyxxxcat
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41055 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b2f2c1e078666cf8c67f04f46b3252ff5427f96, data reload: false

------ Round 1 ----------------------------------
q1	17583	7426	7279	7279
q2	2079	174	163	163
q3	10600	1117	1199	1117
q4	10791	834	699	699
q5	7851	2935	2877	2877
q6	235	153	148	148
q7	1005	612	616	612
q8	9356	1992	2074	1992
q9	6649	6406	6421	6406
q10	6994	2311	2314	2311
q11	489	266	258	258
q12	402	215	208	208
q13	17801	2960	3011	2960
q14	241	211	217	211
q15	562	530	520	520
q16	671	617	608	608
q17	980	562	579	562
q18	7275	6795	6773	6773
q19	1400	1091	1000	1000
q20	499	207	198	198
q21	4038	3159	3167	3159
q22	1126	994	1002	994
Total cold run time: 108627 ms
Total hot run time: 41055 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7280	7251	7266	7251
q2	324	239	238	238
q3	2955	2897	2950	2897
q4	2107	1939	1810	1810
q5	5722	5699	5676	5676
q6	228	141	144	141
q7	2228	1772	1827	1772
q8	3391	3561	3566	3561
q9	8746	8873	8859	8859
q10	3609	3521	3545	3521
q11	596	512	494	494
q12	827	621	622	621
q13	10092	3166	3125	3125
q14	318	273	275	273
q15	570	515	538	515
q16	746	689	678	678
q17	1865	1619	1603	1603
q18	8217	7833	7683	7683
q19	1677	1505	1655	1505
q20	2113	1914	1891	1891
q21	5523	5384	5260	5260
q22	1157	1033	1049	1033
Total cold run time: 70291 ms
Total hot run time: 60407 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198497 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b2f2c1e078666cf8c67f04f46b3252ff5427f96, data reload: false

query1	1302	927	969	927
query2	6246	2115	2042	2042
query3	10838	4260	4250	4250
query4	66360	29234	23518	23518
query5	4970	462	453	453
query6	426	188	180	180
query7	5643	312	311	311
query8	317	222	223	222
query9	9321	2656	2651	2651
query10	494	277	263	263
query11	17328	15522	15833	15522
query12	159	101	102	101
query13	1574	438	428	428
query14	10365	7591	7529	7529
query15	205	184	187	184
query16	7109	485	466	466
query17	1127	601	625	601
query18	1986	331	337	331
query19	252	169	162	162
query20	122	114	113	113
query21	206	101	100	100
query22	4843	4545	4741	4545
query23	34417	34098	34168	34098
query24	6159	2932	2910	2910
query25	527	430	436	430
query26	681	177	173	173
query27	1937	367	358	358
query28	4304	2488	2445	2445
query29	694	446	435	435
query30	236	166	164	164
query31	1031	830	822	822
query32	70	52	55	52
query33	464	285	287	285
query34	921	505	515	505
query35	866	746	724	724
query36	1119	961	971	961
query37	120	76	71	71
query38	4076	4034	4032	4032
query39	1552	1498	1482	1482
query40	208	99	99	99
query41	48	48	52	48
query42	114	102	99	99
query43	534	484	489	484
query44	1169	814	815	814
query45	190	166	173	166
query46	1138	724	725	724
query47	2053	1950	1988	1950
query48	467	377	405	377
query49	752	379	388	379
query50	836	430	417	417
query51	7318	7174	7307	7174
query52	98	89	84	84
query53	255	184	183	183
query54	555	453	453	453
query55	74	77	75	75
query56	251	239	243	239
query57	1228	1129	1109	1109
query58	211	208	215	208
query59	3082	2920	2903	2903
query60	289	261	252	252
query61	106	105	108	105
query62	792	660	659	659
query63	213	189	184	184
query64	1370	681	642	642
query65	3271	3189	3173	3173
query66	703	299	304	299
query67	16142	15684	15879	15684
query68	4000	611	556	556
query69	434	264	259	259
query70	1141	1122	1137	1122
query71	370	251	248	248
query72	6365	4073	4060	4060
query73	742	348	349	348
query74	10172	9053	9062	9053
query75	3320	2618	2607	2607
query76	1954	1052	1059	1052
query77	464	279	270	270
query78	10606	9682	9712	9682
query79	1324	590	599	590
query80	847	433	415	415
query81	527	245	235	235
query82	1173	119	117	117
query83	164	141	140	140
query84	288	78	79	78
query85	885	303	291	291
query86	333	293	296	293
query87	4471	4239	4357	4239
query88	3957	2387	2334	2334
query89	419	290	289	289
query90	2018	185	183	183
query91	180	151	147	147
query92	70	50	49	49
query93	1828	544	552	544
query94	814	299	265	265
query95	365	253	255	253
query96	611	284	289	284
query97	3334	3173	3254	3173
query98	220	203	196	196
query99	1584	1296	1296	1296
Total cold run time: 318814 ms
Total hot run time: 198497 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b2f2c1e078666cf8c67f04f46b3252ff5427f96, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.63	0.10	0.11
query5	0.54	0.52	0.52
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.49	0.50
query10	0.57	0.55	0.56
query11	0.14	0.11	0.11
query12	0.14	0.12	0.11
query13	0.62	0.59	0.59
query14	3.04	2.94	3.02
query15	0.90	0.82	0.80
query16	0.38	0.38	0.38
query17	1.04	1.00	1.04
query18	0.24	0.23	0.22
query19	1.96	1.86	1.96
query20	0.02	0.01	0.00
query21	15.40	0.60	0.58
query22	2.52	2.18	1.92
query23	16.93	0.86	0.85
query24	2.84	1.80	1.53
query25	0.16	0.09	0.22
query26	0.50	0.14	0.14
query27	0.05	0.04	0.04
query28	9.73	1.12	1.07
query29	12.58	3.30	3.28
query30	0.24	0.06	0.06
query31	2.86	0.39	0.40
query32	3.25	0.46	0.45
query33	3.04	3.01	3.06
query34	16.69	4.43	4.45
query35	4.60	4.52	4.49
query36	0.66	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.02	0.02
Total cold run time: 105.87 s
Total hot run time: 33.67 s

@wyxxxcat
Copy link
Contributor Author

run feut

@wyxxxcat wyxxxcat force-pushed the pick-46620-branch-3.0 branch from 8b2f2c1 to e201da4 Compare January 21, 2025 05:57
@wyxxxcat wyxxxcat force-pushed the pick-46620-branch-3.0 branch from e201da4 to 79ae7af Compare January 21, 2025 05:58
@wyxxxcat
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41337 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 79ae7af2b85c1d31cb4048ea05d99712dc91c2e4, data reload: false

------ Round 1 ----------------------------------
q1	17633	7519	7300	7300
q2	2104	174	165	165
q3	11670	1059	1216	1059
q4	10544	735	774	735
q5	7747	2810	2783	2783
q6	238	149	148	148
q7	985	614	607	607
q8	9819	1967	2021	1967
q9	6700	6489	6454	6454
q10	7068	2303	2331	2303
q11	465	271	264	264
q12	408	211	212	211
q13	17909	3060	3187	3060
q14	260	240	227	227
q15	573	532	544	532
q16	715	632	600	600
q17	995	551	587	551
q18	8408	7139	6831	6831
q19	2493	1054	1075	1054
q20	485	199	197	197
q21	4022	3368	3304	3304
q22	1113	999	985	985
Total cold run time: 112354 ms
Total hot run time: 41337 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7405	7298	7278	7278
q2	340	249	244	244
q3	3178	2951	3016	2951
q4	2086	1849	1898	1849
q5	5825	5823	5864	5823
q6	224	141	148	141
q7	2242	1813	1837	1813
q8	3371	3438	3534	3438
q9	8862	8881	8796	8796
q10	3599	3586	3537	3537
q11	602	498	498	498
q12	838	639	621	621
q13	15230	3136	3134	3134
q14	308	271	279	271
q15	587	528	522	522
q16	709	683	685	683
q17	1849	1620	1602	1602
q18	8232	7874	7599	7599
q19	1730	1590	1621	1590
q20	2100	1845	1863	1845
q21	5533	5270	5389	5270
q22	1123	1051	1048	1048
Total cold run time: 75973 ms
Total hot run time: 60553 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196564 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 79ae7af2b85c1d31cb4048ea05d99712dc91c2e4, data reload: false

query1	1315	912	930	912
query2	6248	2048	2000	2000
query3	10827	4247	4025	4025
query4	66467	29765	23537	23537
query5	4872	436	429	429
query6	411	168	189	168
query7	5617	300	302	300
query8	310	222	221	221
query9	9115	2670	2657	2657
query10	447	268	258	258
query11	17370	15249	15787	15249
query12	162	103	107	103
query13	1532	460	427	427
query14	9874	7845	6728	6728
query15	213	181	184	181
query16	7076	532	521	521
query17	1097	597	590	590
query18	1904	333	336	333
query19	208	152	158	152
query20	119	110	112	110
query21	214	106	105	105
query22	4735	4594	4577	4577
query23	34618	34357	34152	34152
query24	6141	2863	2955	2863
query25	551	419	428	419
query26	654	171	171	171
query27	1769	357	362	357
query28	4237	2462	2476	2462
query29	746	491	477	477
query30	246	163	164	163
query31	1020	824	856	824
query32	70	53	55	53
query33	437	272	280	272
query34	912	520	519	519
query35	843	743	742	742
query36	1067	936	959	936
query37	123	74	82	74
query38	4145	3997	3954	3954
query39	1528	1496	1507	1496
query40	207	107	100	100
query41	47	48	49	48
query42	116	96	99	96
query43	538	489	499	489
query44	1166	831	829	829
query45	186	167	169	167
query46	1190	737	738	737
query47	2057	1908	1987	1908
query48	478	390	377	377
query49	724	399	392	392
query50	860	421	422	421
query51	7337	7337	7164	7164
query52	101	87	86	86
query53	257	177	176	176
query54	556	443	453	443
query55	74	74	75	74
query56	264	235	247	235
query57	1216	1098	1101	1098
query58	212	204	207	204
query59	3231	2994	2810	2810
query60	276	251	255	251
query61	135	111	107	107
query62	773	676	659	659
query63	210	191	199	191
query64	1382	674	655	655
query65	3264	3195	3204	3195
query66	699	296	302	296
query67	15931	15728	15452	15452
query68	3894	587	570	570
query69	417	291	280	280
query70	1174	1150	1080	1080
query71	356	258	276	258
query72	6385	4102	3909	3909
query73	761	352	346	346
query74	10212	8934	9037	8934
query75	3352	2606	2687	2606
query76	1810	1208	1058	1058
query77	473	273	276	273
query78	10633	9608	9604	9604
query79	1496	593	607	593
query80	886	424	431	424
query81	514	246	240	240
query82	1275	122	121	121
query83	158	150	142	142
query84	290	92	85	85
query85	890	292	286	286
query86	330	312	300	300
query87	4388	4261	4294	4261
query88	3579	2405	2366	2366
query89	418	282	290	282
query90	2013	183	184	183
query91	186	149	148	148
query92	63	51	52	51
query93	1890	542	548	542
query94	842	292	293	292
query95	362	259	257	257
query96	620	280	288	280
query97	3390	3178	3153	3153
query98	215	211	194	194
query99	1605	1304	1269	1269
Total cold run time: 317604 ms
Total hot run time: 196564 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.6 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 79ae7af2b85c1d31cb4048ea05d99712dc91c2e4, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.11	0.11
query5	0.53	0.52	0.52
query6	1.13	0.73	0.73
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.57	0.52	0.50
query10	0.55	0.55	0.56
query11	0.13	0.11	0.11
query12	0.14	0.13	0.11
query13	0.61	0.60	0.60
query14	3.10	2.95	2.92
query15	0.91	0.84	0.83
query16	0.37	0.38	0.38
query17	1.04	1.05	1.03
query18	0.25	0.23	0.22
query19	1.88	1.84	2.00
query20	0.01	0.00	0.01
query21	15.38	0.61	0.59
query22	3.11	2.32	2.23
query23	17.09	1.11	0.79
query24	3.06	1.50	0.21
query25	0.12	0.21	0.05
query26	0.46	0.14	0.14
query27	0.05	0.03	0.06
query28	11.04	1.10	1.06
query29	12.59	3.31	3.27
query30	0.24	0.06	0.06
query31	2.86	0.40	0.38
query32	3.25	0.48	0.46
query33	2.99	3.01	3.05
query34	16.99	4.47	4.48
query35	4.47	4.46	4.47
query36	0.68	0.48	0.50
query37	0.10	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 108.12 s
Total hot run time: 32.6 s

@dataroaring dataroaring merged commit 0fbba02 into apache:branch-3.0 Jan 21, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants