Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](tvf) fix FE cannot start when replay alter view from tvf #40872

Merged
merged 4 commits into from
Sep 20, 2024

Conversation

BePPPower
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 43058 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8da1e336dc28d3d8d0d7d678f67df58811b83d1, data reload: false

------ Round 1 ----------------------------------
q1	17594	7349	7219	7219
q2	2064	192	185	185
q3	10461	1376	1346	1346
q4	10158	955	991	955
q5	7708	3257	3174	3174
q6	244	160	156	156
q7	1058	643	636	636
q8	9458	2346	2036	2036
q9	6790	6306	6365	6306
q10	7028	2568	2546	2546
q11	444	255	260	255
q12	414	235	230	230
q13	17769	3043	3009	3009
q14	282	259	260	259
q15	591	556	513	513
q16	523	435	434	434
q17	996	949	944	944
q18	7733	6809	6903	6809
q19	1380	1237	1236	1236
q20	604	338	335	335
q21	3900	3538	3511	3511
q22	1086	1028	964	964
Total cold run time: 108285 ms
Total hot run time: 43058 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7200	7184	7179	7179
q2	344	233	227	227
q3	2934	2943	2907	2907
q4	1946	2000	1971	1971
q5	5451	5431	5475	5431
q6	233	145	147	145
q7	2066	1672	1696	1672
q8	3244	3335	3348	3335
q9	8440	8448	8486	8448
q10	3395	3484	3456	3456
q11	568	475	465	465
q12	770	578	627	578
q13	7612	3077	3030	3030
q14	297	288	282	282
q15	560	521	532	521
q16	488	459	444	444
q17	1762	1705	1700	1700
q18	8143	7632	7622	7622
q19	1734	1695	1710	1695
q20	2043	1780	1817	1780
q21	5758	5578	5509	5509
q22	1078	990	982	982
Total cold run time: 66066 ms
Total hot run time: 59379 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195369 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8da1e336dc28d3d8d0d7d678f67df58811b83d1, data reload: false

query1	908	392	384	384
query2	6461	1755	1847	1755
query3	6662	213	221	213
query4	26109	23965	23871	23871
query5	5190	556	558	556
query6	271	177	167	167
query7	4583	311	297	297
query8	297	223	227	223
query9	8502	2566	2560	2560
query10	458	288	279	279
query11	16221	15764	15485	15485
query12	160	100	103	100
query13	1689	398	389	389
query14	11441	7059	6846	6846
query15	210	175	172	172
query16	7619	461	492	461
query17	1557	591	575	575
query18	1945	296	294	294
query19	195	149	148	148
query20	123	110	118	110
query21	211	106	107	106
query22	4804	4554	4416	4416
query23	34795	33976	33720	33720
query24	9502	3088	3119	3088
query25	679	418	419	418
query26	1313	160	159	159
query27	2823	281	297	281
query28	6718	2110	2081	2081
query29	933	441	436	436
query30	292	160	154	154
query31	1007	776	794	776
query32	103	56	58	56
query33	725	311	310	310
query34	923	481	475	475
query35	878	756	743	743
query36	1069	887	885	885
query37	174	85	89	85
query38	3977	3918	3924	3918
query39	1486	1419	1422	1419
query40	234	120	119	119
query41	50	49	48	48
query42	128	101	100	100
query43	499	441	434	434
query44	1200	807	775	775
query45	202	171	173	171
query46	1122	824	850	824
query47	1904	1765	1781	1765
query48	366	294	309	294
query49	1145	455	457	455
query50	920	429	448	429
query51	7011	6939	6881	6881
query52	102	93	90	90
query53	260	190	183	183
query54	771	483	482	482
query55	83	79	79	79
query56	302	282	285	282
query57	1235	1080	1051	1051
query58	259	248	352	248
query59	2745	2623	2533	2533
query60	323	278	276	276
query61	109	105	114	105
query62	930	755	660	660
query63	220	192	184	184
query64	5221	695	709	695
query65	3239	3154	3163	3154
query66	1366	299	309	299
query67	16125	15650	15538	15538
query68	3464	867	856	856
query69	427	332	330	330
query70	1166	1118	1156	1118
query71	351	369	349	349
query72	6036	3425	3403	3403
query73	590	573	585	573
query74	9168	9242	9080	9080
query75	3158	3036	2990	2990
query76	1945	839	854	839
query77	444	404	403	403
query78	9775	9304	9325	9304
query79	917	888	873	873
query80	843	825	814	814
query81	451	269	266	266
query82	263	265	259	259
query83	211	188	192	188
query84	233	112	108	108
query85	688	414	411	411
query86	343	317	329	317
query87	4331	4489	4391	4391
query88	4165	4101	4090	4090
query89	377	371	371	371
query90	1303	323	319	319
query91	125	123	124	123
query92	75	76	74	74
query93	1046	1074	1062	1062
query94	594	401	350	350
query95	454	420	419	419
query96	471	477	470	470
query97	3181	3131	3116	3116
query98	227	233	228	228
query99	1529	1310	1316	1310
Total cold run time: 279435 ms
Total hot run time: 195369 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8da1e336dc28d3d8d0d7d678f67df58811b83d1, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.51
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.56	0.53	0.50
query10	0.57	0.58	0.56
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.63	0.60	0.60
query14	1.44	1.47	1.44
query15	0.90	0.88	0.88
query16	0.37	0.37	0.36
query17	1.00	1.02	1.03
query18	0.22	0.21	0.21
query19	1.89	1.81	1.82
query20	0.01	0.01	0.01
query21	15.43	0.69	0.68
query22	3.95	7.68	1.37
query23	18.02	1.27	1.30
query24	2.24	0.22	0.22
query25	0.18	0.07	0.09
query26	0.30	0.18	0.18
query27	0.08	0.07	0.07
query28	13.18	1.13	1.11
query29	12.57	3.36	3.33
query30	0.24	0.06	0.06
query31	2.88	0.42	0.41
query32	3.23	0.51	0.50
query33	3.04	3.08	3.04
query34	15.44	4.30	4.31
query35	4.36	4.35	4.33
query36	0.70	0.49	0.48
query37	0.19	0.16	0.17
query38	0.15	0.16	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.27 s
Total hot run time: 31.14 s

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 42124 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9afd1417c857b7446bcf4ce135f4d4f5ff9fc48, data reload: false

------ Round 1 ----------------------------------
q1	17587	7171	7143	7143
q2	2047	162	166	162
q3	10737	1167	1298	1167
q4	10447	838	837	837
q5	7730	3219	3159	3159
q6	234	149	144	144
q7	997	613	600	600
q8	9439	2025	2072	2025
q9	6759	6367	6384	6367
q10	7051	2310	2321	2310
q11	455	250	250	250
q12	396	218	220	218
q13	17771	3002	3009	3002
q14	242	224	224	224
q15	556	522	520	520
q16	524	421	418	418
q17	978	899	915	899
q18	7375	6757	6727	6727
q19	1386	1193	1195	1193
q20	575	288	281	281
q21	3952	3477	3546	3477
q22	1111	1013	1001	1001
Total cold run time: 108349 ms
Total hot run time: 42124 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7084	7077	7070	7070
q2	322	232	229	229
q3	3016	3027	3043	3027
q4	2000	2048	2035	2035
q5	5577	5614	5596	5596
q6	244	144	147	144
q7	2193	1793	1801	1793
q8	3310	3391	3406	3391
q9	8680	8829	8740	8740
q10	3551	3456	3490	3456
q11	581	485	485	485
q12	811	619	629	619
q13	9596	3130	3214	3130
q14	299	286	288	286
q15	573	527	517	517
q16	508	464	461	461
q17	1798	1712	1710	1710
q18	8300	7908	8007	7908
q19	1703	1729	1723	1723
q20	2101	1863	1885	1863
q21	5327	5385	5322	5322
q22	1113	1022	1023	1022
Total cold run time: 68687 ms
Total hot run time: 60527 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198214 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9afd1417c857b7446bcf4ce135f4d4f5ff9fc48, data reload: false

query1	1310	878	892	878
query2	6385	1779	1773	1773
query3	10733	3918	3821	3821
query4	55012	28483	23734	23734
query5	5285	505	508	505
query6	411	175	166	166
query7	5792	311	308	308
query8	272	220	235	220
query9	7819	2597	2606	2597
query10	432	287	272	272
query11	16928	15476	15412	15412
query12	162	112	109	109
query13	1514	428	422	422
query14	10559	6733	7268	6733
query15	200	181	188	181
query16	7044	469	502	469
query17	1196	585	605	585
query18	1924	320	319	319
query19	190	155	154	154
query20	125	111	108	108
query21	214	102	113	102
query22	4824	4475	4590	4475
query23	34610	34127	33660	33660
query24	6113	3123	3068	3068
query25	509	404	418	404
query26	637	164	163	163
query27	1666	294	293	293
query28	2847	2100	2070	2070
query29	671	430	436	430
query30	225	153	151	151
query31	982	812	771	771
query32	79	52	58	52
query33	460	308	310	308
query34	881	490	470	470
query35	860	739	717	717
query36	1018	905	901	901
query37	140	76	79	76
query38	4045	4000	3977	3977
query39	1483	1405	1388	1388
query40	207	105	99	99
query41	49	44	48	44
query42	124	97	94	94
query43	488	451	468	451
query44	1186	787	760	760
query45	193	167	169	167
query46	1099	822	823	822
query47	1851	1761	1794	1761
query48	433	361	354	354
query49	721	413	428	413
query50	917	435	439	435
query51	6953	6928	6829	6829
query52	101	91	89	89
query53	252	180	179	179
query54	566	456	469	456
query55	76	75	75	75
query56	266	268	277	268
query57	1172	1069	1089	1069
query58	225	246	244	244
query59	2849	2684	2839	2684
query60	285	273	275	273
query61	103	116	96	96
query62	770	683	675	675
query63	211	185	185	185
query64	1469	632	644	632
query65	3249	3175	3201	3175
query66	654	314	293	293
query67	16041	15689	15466	15466
query68	1960	740	578	578
query69	450	285	299	285
query70	1187	1119	1086	1086
query71	329	277	285	277
query72	5183	3986	3951	3951
query73	755	334	342	334
query74	9443	8931	8949	8931
query75	3341	2673	2723	2673
query76	1343	1286	1286	1286
query77	503	295	282	282
query78	9981	11216	9315	9315
query79	923	873	846	846
query80	743	572	572	572
query81	488	232	235	232
query82	249	187	191	187
query83	196	197	153	153
query84	288	101	97	97
query85	740	363	348	348
query86	358	311	326	311
query87	4347	4337	4339	4337
query88	4126	4062	4078	4062
query89	369	354	351	351
query90	1870	303	296	296
query91	162	164	159	159
query92	77	72	72	72
query93	1016	994	993	993
query94	731	365	367	365
query95	465	420	407	407
query96	469	465	471	465
query97	3111	3119	3103	3103
query98	228	227	243	227
query99	1686	1315	1303	1303
Total cold run time: 295184 ms
Total hot run time: 198214 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.99 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9afd1417c857b7446bcf4ce135f4d4f5ff9fc48, data reload: false

query1	0.05	0.04	0.05
query2	0.05	0.03	0.02
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.50	0.50	0.50
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.52	0.52
query10	0.56	0.58	0.57
query11	0.14	0.10	0.10
query12	0.13	0.11	0.10
query13	0.59	0.59	0.60
query14	1.44	1.46	1.49
query15	0.88	0.86	0.84
query16	0.38	0.38	0.38
query17	1.00	1.01	1.03
query18	0.18	0.19	0.19
query19	1.93	1.83	1.81
query20	0.01	0.01	0.00
query21	15.42	0.58	0.58
query22	3.16	5.02	2.53
query23	17.54	1.12	0.90
query24	2.41	0.65	0.21
query25	0.30	0.08	0.05
query26	0.16	0.14	0.13
query27	0.05	0.04	0.03
query28	11.90	1.03	0.99
query29	12.60	3.31	3.28
query30	0.25	0.06	0.05
query31	2.88	0.38	0.37
query32	3.27	0.46	0.47
query33	2.98	3.00	3.05
query34	15.46	4.31	4.28
query35	4.30	4.34	4.31
query36	0.70	0.50	0.48
query37	0.09	0.06	0.05
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.02	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.31 s
Total hot run time: 30.99 s

morningman
morningman previously approved these changes Sep 14, 2024
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 14, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

yiguolei
yiguolei previously approved these changes Sep 15, 2024
@BePPPower BePPPower marked this pull request as draft September 18, 2024 02:12
@morningman
Copy link
Contributor

Need to add upgrade test cases

@BePPPower BePPPower dismissed stale reviews from yiguolei and morningman via 117b676 September 18, 2024 06:29
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 18, 2024
@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41259 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit caaa7304e981de03e481a397f1143521582a401d, data reload: false

------ Round 1 ----------------------------------
q1	17582	7851	7229	7229
q2	2053	166	156	156
q3	10894	1094	1195	1094
q4	10578	731	685	685
q5	7753	3027	3088	3027
q6	237	149	146	146
q7	1016	605	608	605
q8	9420	2018	2025	2018
q9	6938	6394	6454	6394
q10	6993	2292	2324	2292
q11	441	247	245	245
q12	405	210	217	210
q13	17787	3016	2973	2973
q14	242	216	218	216
q15	571	532	522	522
q16	689	604	615	604
q17	975	858	786	786
q18	7290	6652	6685	6652
q19	1393	962	961	961
q20	588	295	281	281
q21	4098	3183	3138	3138
q22	1123	1031	1025	1025
Total cold run time: 109066 ms
Total hot run time: 41259 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7229	7243	7245	7243
q2	326	230	224	224
q3	3027	2965	2931	2931
q4	2060	1809	1829	1809
q5	5667	5625	5693	5625
q6	237	144	139	139
q7	2216	1823	1814	1814
q8	3302	3397	3432	3397
q9	8797	8940	8795	8795
q10	3519	3448	3503	3448
q11	572	489	487	487
q12	810	651	625	625
q13	9034	3157	3140	3140
q14	305	277	281	277
q15	583	537	530	530
q16	726	685	689	685
q17	1793	1592	1584	1584
q18	8294	7744	7683	7683
q19	1723	1519	1463	1463
q20	2143	1919	1916	1916
q21	5638	5244	5330	5244
q22	1176	1074	1045	1045
Total cold run time: 69177 ms
Total hot run time: 60104 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199416 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit caaa7304e981de03e481a397f1143521582a401d, data reload: false

query1	1302	862	875	862
query2	6401	1976	2009	1976
query3	10796	3922	3910	3910
query4	64836	29163	23651	23651
query5	5091	478	462	462
query6	413	175	173	173
query7	5457	299	296	296
query8	307	264	225	225
query9	8451	2693	2645	2645
query10	448	290	286	286
query11	17223	15167	15801	15167
query12	156	101	101	101
query13	1456	449	423	423
query14	10783	7406	7543	7406
query15	205	186	176	176
query16	6819	468	451	451
query17	1207	624	593	593
query18	1513	309	310	309
query19	216	150	159	150
query20	138	111	114	111
query21	220	105	106	105
query22	4758	4634	4594	4594
query23	34965	34134	33908	33908
query24	5948	2908	2883	2883
query25	506	410	427	410
query26	613	155	156	155
query27	1599	276	290	276
query28	4213	2462	2414	2414
query29	669	436	424	424
query30	230	157	163	157
query31	970	763	773	763
query32	72	54	54	54
query33	397	295	296	295
query34	882	480	487	480
query35	848	735	755	735
query36	1045	925	945	925
query37	143	80	89	80
query38	3971	3861	3869	3861
query39	1499	1387	1401	1387
query40	205	97	96	96
query41	50	48	51	48
query42	116	95	95	95
query43	518	472	479	472
query44	1128	794	810	794
query45	195	162	163	162
query46	1133	771	848	771
query47	1927	1785	1820	1785
query48	475	352	361	352
query49	672	391	397	391
query50	831	403	393	393
query51	7109	6964	6877	6877
query52	101	84	83	83
query53	252	177	175	175
query54	558	446	440	440
query55	80	76	77	76
query56	278	258	245	245
query57	1221	1100	1089	1089
query58	221	242	234	234
query59	3093	2875	3013	2875
query60	288	271	270	270
query61	107	99	100	99
query62	750	652	652	652
query63	219	180	186	180
query64	1319	664	643	643
query65	3233	3186	3179	3179
query66	656	298	299	298
query67	16014	15555	15540	15540
query68	1389	595	546	546
query69	422	283	284	283
query70	1213	1103	1133	1103
query71	319	256	274	256
query72	6105	4005	4165	4005
query73	771	326	325	325
query74	9308	9044	9033	9033
query75	3361	2603	2674	2603
query76	1347	907	913	907
query77	496	294	291	291
query78	10048	9690	9622	9622
query79	1179	903	875	875
query80	835	561	575	561
query81	508	253	254	253
query82	1160	230	235	230
query83	168	160	156	156
query84	286	107	94	94
query85	744	373	375	373
query86	339	331	329	329
query87	4555	4300	4395	4300
query88	4440	4067	4027	4027
query89	384	361	357	357
query90	1806	310	304	304
query91	161	157	162	157
query92	74	75	74	74
query93	956	900	900	900
query94	702	376	391	376
query95	441	397	400	397
query96	490	480	489	480
query97	3176	3113	3155	3113
query98	227	234	217	217
query99	1465	1276	1340	1276
Total cold run time: 308390 ms
Total hot run time: 199416 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit caaa7304e981de03e481a397f1143521582a401d, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.02	0.03
query3	0.22	0.06	0.06
query4	1.64	0.10	0.10
query5	0.51	0.50	0.54
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.57	0.56	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.10
query13	0.60	0.59	0.59
query14	3.02	2.95	3.10
query15	0.90	0.82	0.82
query16	0.39	0.36	0.38
query17	1.02	1.07	1.04
query18	0.20	0.20	0.19
query19	1.85	1.87	1.99
query20	0.01	0.00	0.01
query21	15.37	0.62	0.58
query22	3.16	2.59	1.57
query23	17.13	1.03	0.72
query24	2.60	1.36	2.09
query25	0.31	0.17	0.06
query26	0.37	0.14	0.13
query27	0.04	0.04	0.04
query28	9.72	1.11	1.07
query29	12.63	3.29	3.30
query30	0.25	0.06	0.05
query31	2.89	0.38	0.38
query32	3.27	0.46	0.46
query33	3.03	3.01	3.08
query34	16.77	4.38	4.38
query35	4.45	4.39	4.52
query36	0.67	0.48	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.19 s
Total hot run time: 32.82 s

@BePPPower BePPPower marked this pull request as ready for review September 18, 2024 12:42
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 20, 2024
@morningman morningman merged commit a2b9aa4 into apache:master Sep 20, 2024
25 of 28 checks passed
morningman pushed a commit to morningman/doris that referenced this pull request Sep 23, 2024
morningman added a commit that referenced this pull request Sep 24, 2024
) (#41159)

bp #40872

---------

Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>
BePPPower added a commit to BePPPower/doris that referenced this pull request Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants