[feature](external) process tbl/db exist when create/drop db/tbl #33119

wsjz · 2024-04-01T10:30:42Z

Proposed changes

Issue Number: #31442

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-04-01T10:30:46Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

wsjz · 2024-04-01T10:45:20Z

run buildall

wsjz · 2024-04-02T04:04:26Z

run buildall

doris-robot · 2024-04-02T04:28:12Z

TPC-H: Total hot run time: 38986 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc649c444eecddf505ec1fcb40d48f138d3ed03a, data reload: false

------ Round 1 ----------------------------------
q1	18055	4272	4180	4180
q2	2468	200	187	187
q3	11324	1281	1464	1281
q4	11276	855	983	855
q5	8096	3049	2987	2987
q6	226	137	136	136
q7	1144	636	640	636
q8	10259	2027	2036	2027
q9	6718	6194	6151	6151
q10	8434	3513	3505	3505
q11	416	247	236	236
q12	392	224	220	220
q13	17788	2903	2905	2903
q14	276	241	240	240
q15	533	482	473	473
q16	502	397	381	381
q17	970	931	891	891
q18	7519	6542	6479	6479
q19	1625	1541	1536	1536
q20	607	322	319	319
q21	3620	3131	3062	3062
q22	367	301	302	301
Total cold run time: 112615 ms
Total hot run time: 38986 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4082	4067	4056	4056
q2	329	219	214	214
q3	2949	2945	2945	2945
q4	1868	1896	1833	1833
q5	5245	5227	5215	5215
q6	210	125	127	125
q7	2240	1821	1823	1821
q8	3222	3301	3288	3288
q9	8499	8499	8518	8499
q10	3784	3827	3818	3818
q11	542	454	464	454
q12	704	537	550	537
q13	12608	2926	2877	2877
q14	293	261	265	261
q15	516	476	474	474
q16	472	410	412	410
q17	1728	1676	1679	1676
q18	7727	7260	7143	7143
q19	1635	1661	1652	1652
q20	1949	1707	1704	1704
q21	5065	4817	4735	4735
q22	496	421	427	421
Total cold run time: 66163 ms
Total hot run time: 54158 ms

doris-robot · 2024-04-02T04:39:55Z

TPC-DS: Total hot run time: 182428 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc649c444eecddf505ec1fcb40d48f138d3ed03a, data reload: false

query1	1211	1131	1125	1125
query2	6475	1909	1923	1909
query3	6666	213	230	213
query4	24847	21279	21338	21279
query5	4225	436	404	404
query6	266	196	179	179
query7	4606	308	299	299
query8	227	191	193	191
query9	8483	2267	2270	2267
query10	590	259	265	259
query11	14995	14498	14492	14492
query12	140	95	95	95
query13	1652	395	384	384
query14	8684	6872	6878	6872
query15	222	176	183	176
query16	7043	276	273	273
query17	993	626	581	581
query18	1889	287	282	282
query19	204	166	163	163
query20	98	94	94	94
query21	211	133	127	127
query22	4946	4780	4817	4780
query23	33546	32877	32745	32745
query24	12595	3210	3213	3210
query25	710	405	413	405
query26	1950	163	171	163
query27	3007	335	325	325
query28	6819	1859	1839	1839
query29	1305	597	611	597
query30	310	164	172	164
query31	1022	716	735	716
query32	101	63	60	60
query33	722	269	264	264
query34	990	510	518	510
query35	853	708	712	708
query36	996	868	906	868
query37	268	85	80	80
query38	3553	3398	3379	3379
query39	1579	1531	1524	1524
query40	291	133	135	133
query41	50	47	46	46
query42	111	103	104	103
query43	446	401	399	399
query44	1060	712	697	697
query45	283	263	271	263
query46	1081	789	776	776
query47	1879	1775	1778	1775
query48	388	314	312	312
query49	1179	361	382	361
query50	799	405	390	390
query51	6818	6727	6787	6727
query52	108	98	99	98
query53	356	288	289	288
query54	322	249	243	243
query55	91	80	77	77
query56	247	228	230	228
query57	1205	1133	1124	1124
query58	257	224	229	224
query59	2574	2393	2430	2393
query60	260	248	254	248
query61	110	110	107	107
query62	708	467	459	459
query63	312	290	285	285
query64	6486	3021	3052	3021
query65	3049	3006	3011	3006
query66	1431	343	324	324
query67	15565	14989	15000	14989
query68	9022	575	606	575
query69	558	333	327	327
query70	1327	1100	1105	1100
query71	512	270	277	270
query72	6353	2563	2432	2432
query73	1335	327	333	327
query74	6725	6337	6289	6289
query75	3844	2291	2273	2273
query76	5505	1031	1206	1031
query77	619	249	254	249
query78	10836	10253	10166	10166
query79	9457	542	534	534
query80	1483	432	416	416
query81	513	248	240	240
query82	717	112	114	112
query83	214	163	161	161
query84	269	88	94	88
query85	1436	298	288	288
query86	423	274	280	274
query87	3685	3518	3496	3496
query88	3720	2360	2348	2348
query89	566	386	385	385
query90	2011	183	184	183
query91	133	107	102	102
query92	59	50	52	50
query93	6691	547	534	534
query94	1114	198	196	196
query95	1093	1096	1097	1096
query96	606	271	291	271
query97	2676	2450	2480	2450
query98	244	215	211	211
query99	1359	857	873	857
Total cold run time: 301570 ms
Total hot run time: 182428 ms

doris-robot · 2024-04-02T04:45:20Z

ClickBench: Total hot run time: 30.01 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc649c444eecddf505ec1fcb40d48f138d3ed03a, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.48	0.49
query6	1.12	0.65	0.66
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.56	0.50	0.49
query10	0.56	0.57	0.56
query11	0.15	0.11	0.11
query12	0.13	0.12	0.12
query13	0.61	0.60	0.60
query14	0.78	0.79	0.80
query15	0.87	0.85	0.84
query16	0.35	0.36	0.35
query17	0.98	1.00	1.00
query18	0.25	0.24	0.25
query19	1.88	1.75	1.73
query20	0.01	0.02	0.01
query21	15.41	0.75	0.69
query22	2.98	5.47	1.61
query23	17.75	1.40	1.12
query24	1.86	0.22	0.22
query25	0.15	0.09	0.09
query26	0.30	0.18	0.20
query27	0.08	0.08	0.08
query28	13.46	0.98	0.93
query29	12.61	3.35	3.53
query30	0.26	0.08	0.06
query31	2.86	0.39	0.38
query32	3.27	0.48	0.46
query33	2.86	2.82	2.90
query34	15.48	4.35	4.33
query35	4.34	4.35	4.38
query36	0.68	0.47	0.46
query37	0.20	0.16	0.17
query38	0.17	0.16	0.16
query39	0.05	0.04	0.04
query40	0.19	0.15	0.16
query41	0.10	0.05	0.05
query42	0.07	0.06	0.06
query43	0.05	0.05	0.04
Total cold run time: 106 s
Total hot run time: 30.01 s

doris-robot · 2024-04-02T04:48:30Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fc649c444eecddf505ec1fcb40d48f138d3ed03a with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       16.4 seconds inserted 10000000 Rows, about 609K ops/s

wsjz · 2024-04-02T04:58:06Z

run buildall

doris-robot · 2024-04-02T05:22:03Z

TPC-H: Total hot run time: 38968 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7316605412ff18d003ca993bb1514adc8fc9bebc, data reload: false

------ Round 1 ----------------------------------
q1	19272	4244	4208	4208
q2	2650	210	189	189
q3	10763	1252	1415	1252
q4	10214	942	1036	942
q5	7478	2981	2935	2935
q6	220	132	134	132
q7	1119	657	619	619
q8	9412	2065	2055	2055
q9	6801	6211	6154	6154
q10	8437	3547	3509	3509
q11	411	242	251	242
q12	387	225	220	220
q13	17773	2903	2941	2903
q14	280	249	250	249
q15	546	487	482	482
q16	488	412	381	381
q17	964	912	905	905
q18	7365	6493	6434	6434
q19	1618	1542	1556	1542
q20	601	315	319	315
q21	3611	3121	3002	3002
q22	362	298	308	298
Total cold run time: 110772 ms
Total hot run time: 38968 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4110	4151	4049	4049
q2	326	221	225	221
q3	2972	2936	2955	2936
q4	1908	1845	1842	1842
q5	5217	5187	5211	5187
q6	208	125	128	125
q7	2222	1821	1776	1776
q8	3204	3289	3284	3284
q9	8512	8515	8486	8486
q10	3767	3803	3810	3803
q11	542	460	464	460
q12	699	557	572	557
q13	16797	2900	2892	2892
q14	310	254	270	254
q15	507	483	484	483
q16	463	424	422	422
q17	1725	1697	1684	1684
q18	7660	7232	7165	7165
q19	1636	1639	1643	1639
q20	1957	1730	1722	1722
q21	4979	4763	4753	4753
q22	492	418	419	418
Total cold run time: 70213 ms
Total hot run time: 54158 ms

doris-robot · 2024-04-02T05:33:35Z

TPC-DS: Total hot run time: 181724 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7316605412ff18d003ca993bb1514adc8fc9bebc, data reload: false

query1	1220	1128	1117	1117
query2	6474	1882	1871	1871
query3	6666	223	223	223
query4	24922	21587	21483	21483
query5	4162	409	428	409
query6	296	196	192	192
query7	4595	318	309	309
query8	235	186	178	178
query9	8483	2299	2300	2299
query10	583	268	282	268
query11	15007	14470	14415	14415
query12	149	99	93	93
query13	1640	384	392	384
query14	8635	6938	6895	6895
query15	216	176	187	176
query16	7178	285	279	279
query17	979	627	588	588
query18	1923	297	300	297
query19	209	164	161	161
query20	98	96	98	96
query21	199	129	129	129
query22	4938	4810	4800	4800
query23	33721	32715	32552	32552
query24	12602	3188	3078	3078
query25	690	412	429	412
query26	1896	176	169	169
query27	2994	332	353	332
query28	6728	1861	1845	1845
query29	1351	613	602	602
query30	319	169	166	166
query31	1030	757	725	725
query32	101	61	63	61
query33	707	277	267	267
query34	998	502	512	502
query35	852	694	711	694
query36	1003	864	896	864
query37	290	84	79	79
query38	3539	3391	3366	3366
query39	1610	1558	1537	1537
query40	298	135	143	135
query41	52	48	48	48
query42	116	112	105	105
query43	431	399	399	399
query44	1099	707	704	704
query45	288	258	261	258
query46	1077	806	791	791
query47	1902	1787	1805	1787
query48	392	310	310	310
query49	1190	367	363	363
query50	813	393	405	393
query51	6883	6648	6707	6648
query52	107	93	94	93
query53	359	295	297	295
query54	315	241	250	241
query55	96	84	78	78
query56	248	230	226	226
query57	1223	1092	1143	1092
query58	250	226	219	219
query59	2547	2407	2448	2407
query60	260	235	246	235
query61	96	95	96	95
query62	704	446	448	446
query63	315	295	293	293
query64	6413	3008	3050	3008
query65	3070	3003	3019	3003
query66	1450	321	331	321
query67	15445	14970	15106	14970
query68	6215	589	585	585
query69	538	348	339	339
query70	1185	1125	1108	1108
query71	460	285	281	281
query72	6350	2598	2420	2420
query73	792	327	330	327
query74	6606	6342	6258	6258
query75	3060	2271	2340	2271
query76	3770	1117	1188	1117
query77	574	260	260	260
query78	10870	10218	10278	10218
query79	7834	556	550	550
query80	1260	465	455	455
query81	527	244	245	244
query82	673	107	107	107
query83	210	170	171	170
query84	271	95	99	95
query85	1438	350	336	336
query86	393	260	265	260
query87	3698	3567	3527	3527
query88	3712	2376	2380	2376
query89	559	377	370	370
query90	1992	179	189	179
query91	136	109	110	109
query92	69	54	56	54
query93	6121	540	534	534
query94	1244	197	202	197
query95	435	342	338	338
query96	611	272	269	269
query97	2637	2486	2454	2454
query98	239	216	213	213
query99	1279	860	827	827
Total cold run time: 292435 ms
Total hot run time: 181724 ms

doris-robot · 2024-04-02T05:39:02Z

ClickBench: Total hot run time: 30.46 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7316605412ff18d003ca993bb1514adc8fc9bebc, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.47	0.50	0.49
query6	1.14	0.65	0.65
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.60	0.52	0.52
query10	0.58	0.57	0.57
query11	0.14	0.11	0.12
query12	0.14	0.12	0.12
query13	0.62	0.60	0.60
query14	0.78	0.79	0.80
query15	0.86	0.84	0.83
query16	0.35	0.36	0.35
query17	0.99	0.96	1.03
query18	0.25	0.26	0.24
query19	1.84	1.78	1.72
query20	0.01	0.01	0.01
query21	15.41	0.77	0.73
query22	3.44	6.05	1.98
query23	17.74	1.40	1.14
query24	1.51	0.21	0.24
query25	0.14	0.10	0.08
query26	0.30	0.18	0.18
query27	0.08	0.09	0.08
query28	13.79	0.96	0.92
query29	12.56	3.39	3.40
query30	0.26	0.06	0.05
query31	2.85	0.38	0.37
query32	3.29	0.48	0.47
query33	2.89	2.88	2.87
query34	15.51	4.34	4.34
query35	4.37	4.40	4.35
query36	0.68	0.47	0.48
query37	0.19	0.16	0.17
query38	0.18	0.16	0.15
query39	0.05	0.04	0.04
query40	0.17	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.05	0.05	0.04
Total cold run time: 106.48 s
Total hot run time: 30.46 s

doris-robot · 2024-04-02T05:42:10Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 7316605412ff18d003ca993bb1514adc8fc9bebc with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       16.2 seconds inserted 10000000 Rows, about 617K ops/s

github-actions · 2024-04-02T05:52:27Z

PR approved by anyone and no changes requested.

fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetadataOps.java

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataOps.java

doris-robot · 2024-04-02T08:14:37Z

TPC-H: Total hot run time: 39253 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 26b8bb622e13d1462e19c41ef19d83551efd4d21, data reload: false

------ Round 1 ----------------------------------
q1	17733	4256	4183	4183
q2	2392	197	195	195
q3	11991	1405	1438	1405
q4	10287	948	983	948
q5	8164	2962	2915	2915
q6	219	138	141	138
q7	1117	644	617	617
q8	9420	2060	2025	2025
q9	6682	6211	6171	6171
q10	8408	3521	3490	3490
q11	412	242	246	242
q12	385	222	213	213
q13	17767	2882	2921	2882
q14	274	249	254	249
q15	524	496	476	476
q16	493	397	374	374
q17	965	924	901	901
q18	7263	6579	6596	6579
q19	1606	1552	1539	1539
q20	603	336	307	307
q21	3545	3102	3105	3102
q22	374	302	327	302
Total cold run time: 110624 ms
Total hot run time: 39253 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4049	4165	4053	4053
q2	328	219	220	219
q3	2960	2938	2944	2938
q4	1897	1821	1826	1821
q5	5201	5212	5205	5205
q6	209	128	129	128
q7	2254	1813	1807	1807
q8	3206	3275	3305	3275
q9	8531	8483	8460	8460
q10	3757	3845	3804	3804
q11	547	461	464	461
q12	708	544	557	544
q13	16780	2927	2886	2886
q14	290	258	268	258
q15	512	471	471	471
q16	466	407	407	407
q17	1730	1695	1689	1689
q18	7689	7289	7206	7206
q19	1639	1640	1640	1640
q20	1954	1722	1748	1722
q21	4987	4729	4694	4694
q22	476	412	444	412
Total cold run time: 70170 ms
Total hot run time: 54100 ms

wsjz · 2024-04-02T08:22:15Z

run buildall

doris-robot · 2024-04-02T08:40:04Z

TPC-H: Total hot run time: 38856 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e279b3952c9834593fbd9f08fa6981fa4c5364f0, data reload: false

------ Round 1 ----------------------------------
q1	18153	4127	4078	4078
q2	2022	193	181	181
q3	10490	1253	1351	1253
q4	10591	961	1038	961
q5	7478	3011	2923	2923
q6	215	132	138	132
q7	1117	611	614	611
q8	9405	2052	2031	2031
q9	6813	6200	6147	6147
q10	8441	3507	3487	3487
q11	411	243	238	238
q12	379	221	210	210
q13	17787	2919	2903	2903
q14	271	245	246	245
q15	534	478	492	478
q16	488	400	384	384
q17	957	907	902	902
q18	7318	6603	6485	6485
q19	1608	1537	1547	1537
q20	602	316	300	300
q21	3532	3110	3065	3065
q22	357	305	311	305
Total cold run time: 108969 ms
Total hot run time: 38856 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4069	4050	4046	4046
q2	327	217	222	217
q3	2976	2974	2975	2974
q4	1873	1861	1843	1843
q5	5238	5244	5232	5232
q6	208	126	124	124
q7	2259	1814	1804	1804
q8	3220	3285	3287	3285
q9	8540	8485	8514	8485
q10	3775	3835	3823	3823
q11	542	452	450	450
q12	695	570	539	539
q13	16818	2920	2933	2920
q14	295	269	254	254
q15	514	477	469	469
q16	444	414	409	409
q17	1715	1664	1694	1664
q18	7728	7215	7222	7215
q19	1624	1635	1634	1634
q20	1942	1724	1712	1712
q21	5004	4675	4788	4675
q22	482	430	424	424
Total cold run time: 70288 ms
Total hot run time: 54198 ms

doris-robot · 2024-04-02T08:51:45Z

TPC-DS: Total hot run time: 182003 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e279b3952c9834593fbd9f08fa6981fa4c5364f0, data reload: false

query1	1225	1122	1122	1122
query2	6475	2015	1860	1860
query3	6658	205	212	205
query4	24670	21598	21448	21448
query5	4241	395	388	388
query6	269	187	187	187
query7	4598	319	299	299
query8	243	188	180	180
query9	8480	2290	2285	2285
query10	564	257	268	257
query11	15226	14371	14378	14371
query12	145	95	91	91
query13	1638	390	376	376
query14	8671	6784	6732	6732
query15	207	188	179	179
query16	7154	280	280	280
query17	992	616	587	587
query18	1915	295	286	286
query19	212	163	159	159
query20	99	99	93	93
query21	198	127	126	126
query22	5025	4886	4866	4866
query23	33537	32788	32663	32663
query24	12613	3169	3121	3121
query25	707	408	436	408
query26	1935	170	172	170
query27	2994	325	349	325
query28	6703	1866	1846	1846
query29	1380	590	595	590
query30	308	172	164	164
query31	979	725	740	725
query32	101	62	60	60
query33	715	261	254	254
query34	1016	498	503	498
query35	825	697	703	697
query36	984	893	859	859
query37	292	80	80	80
query38	3525	3428	3425	3425
query39	1577	1533	1519	1519
query40	287	136	136	136
query41	47	49	49	49
query42	110	101	103	101
query43	450	397	400	397
query44	1116	710	704	704
query45	280	256	256	256
query46	1056	798	779	779
query47	1906	1820	1806	1806
query48	374	313	299	299
query49	1155	357	361	357
query50	795	389	398	389
query51	6883	6814	6804	6804
query52	114	102	101	101
query53	362	291	285	285
query54	332	233	253	233
query55	94	87	80	80
query56	244	226	226	226
query57	1205	1115	1103	1103
query58	251	219	226	219
query59	2533	2397	2508	2397
query60	253	230	256	230
query61	111	117	111	111
query62	711	468	474	468
query63	318	285	282	282
query64	6443	3051	3068	3051
query65	3095	2996	3004	2996
query66	1433	322	321	321
query67	15619	15231	14826	14826
query68	8981	575	576	575
query69	547	332	326	326
query70	1231	1086	1084	1084
query71	508	282	263	263
query72	6337	2562	2411	2411
query73	933	324	323	323
query74	6708	6283	6349	6283
query75	3714	2261	2273	2261
query76	5373	1078	1213	1078
query77	613	252	248	248
query78	10855	10200	10175	10175
query79	9353	535	527	527
query80	1139	418	406	406
query81	519	243	238	238
query82	365	106	105	105
query83	210	157	162	157
query84	265	89	86	86
query85	1033	282	288	282
query86	373	300	280	280
query87	3699	3470	3516	3470
query88	3658	2351	2354	2351
query89	549	369	370	369
query90	2088	172	177	172
query91	126	107	104	104
query92	62	55	54	54
query93	6623	535	527	527
query94	1340	194	197	194
query95	1092	1089	1088	1088
query96	608	273	272	272
query97	2640	2455	2461	2455
query98	231	224	207	207
query99	1224	846	845	845
Total cold run time: 299670 ms
Total hot run time: 182003 ms

doris-robot · 2024-04-02T08:57:07Z

ClickBench: Total hot run time: 30.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e279b3952c9834593fbd9f08fa6981fa4c5364f0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.06
query5	0.48	0.49	0.50
query6	1.13	0.65	0.66
query7	0.03	0.01	0.01
query8	0.06	0.04	0.04
query9	0.57	0.52	0.51
query10	0.57	0.57	0.57
query11	0.15	0.11	0.11
query12	0.14	0.12	0.11
query13	0.62	0.59	0.59
query14	0.76	0.78	0.79
query15	0.87	0.83	0.84
query16	0.35	0.35	0.35
query17	0.97	0.99	0.99
query18	0.25	0.25	0.26
query19	1.86	1.69	1.80
query20	0.02	0.00	0.00
query21	15.41	0.73	0.74
query22	3.38	5.14	1.64
query23	17.39	1.22	1.28
query24	1.35	0.42	0.25
query25	0.12	0.10	0.08
query26	0.28	0.17	0.19
query27	0.08	0.09	0.08
query28	13.37	0.96	0.93
query29	12.50	3.45	3.37
query30	0.26	0.06	0.06
query31	2.87	0.39	0.38
query32	3.27	0.46	0.47
query33	2.84	2.92	2.87
query34	15.48	4.39	4.33
query35	4.38	4.38	4.36
query36	0.67	0.48	0.48
query37	0.19	0.16	0.18
query38	0.17	0.16	0.17
query39	0.05	0.04	0.04
query40	0.18	0.15	0.14
query41	0.09	0.06	0.06
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 105.3 s
Total hot run time: 30.2 s

doris-robot · 2024-04-02T09:00:13Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e279b3952c9834593fbd9f08fa6981fa4c5364f0 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       15.9 seconds inserted 10000000 Rows, about 628K ops/s

wsjz · 2024-04-02T09:55:56Z

run buildall

doris-robot · 2024-04-02T10:14:00Z

TPC-H: Total hot run time: 39662 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a37a11be84cead0f55fff4f5bdccc8e02b3c1025, data reload: false

------ Round 1 ----------------------------------
q1	17660	4106	4156	4106
q2	2012	187	184	184
q3	10494	1284	1409	1284
q4	10771	897	1035	897
q5	8019	3085	3023	3023
q6	228	140	137	137
q7	1126	665	652	652
q8	10946	2089	2103	2089
q9	7431	6792	6729	6729
q10	8413	3516	3501	3501
q11	408	248	243	243
q12	385	221	214	214
q13	17787	2902	2921	2902
q14	276	241	247	241
q15	539	506	485	485
q16	502	393	383	383
q17	956	914	917	914
q18	7439	6496	6403	6403
q19	1603	1537	1525	1525
q20	612	316	320	316
q21	3546	3156	3124	3124
q22	364	328	310	310
Total cold run time: 111517 ms
Total hot run time: 39662 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4150	4070	4075	4070
q2	334	218	220	218
q3	2940	2952	2966	2952
q4	1889	1842	1885	1842
q5	5228	5213	5257	5213
q6	208	128	126	126
q7	2271	1789	1848	1789
q8	3212	3308	3279	3279
q9	9141	9136	9142	9136
q10	3779	3852	3883	3852
q11	544	450	457	450
q12	698	544	540	540
q13	10343	2897	2951	2897
q14	291	258	265	258
q15	516	481	471	471
q16	469	408	413	408
q17	1713	1691	1668	1668
q18	7664	7404	7137	7137
q19	1633	1624	1643	1624
q20	1974	1733	1725	1725
q21	5005	4799	4816	4799
q22	517	433	429	429
Total cold run time: 64519 ms
Total hot run time: 54883 ms

doris-robot · 2024-04-02T10:25:38Z

TPC-DS: Total hot run time: 182629 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a37a11be84cead0f55fff4f5bdccc8e02b3c1025, data reload: false

query1	1229	1117	1127	1117
query2	6500	1861	1845	1845
query3	6663	217	223	217
query4	25010	21447	21693	21447
query5	4181	417	427	417
query6	272	188	182	182
query7	4596	316	308	308
query8	232	181	178	178
query9	8451	2278	2273	2273
query10	586	258	262	258
query11	14954	14468	14595	14468
query12	146	94	94	94
query13	1646	408	392	392
query14	8704	6758	6817	6758
query15	225	182	189	182
query16	7094	303	275	275
query17	970	609	581	581
query18	1896	293	281	281
query19	213	165	166	165
query20	102	99	93	93
query21	198	131	128	128
query22	4961	4817	4789	4789
query23	33601	32871	32877	32871
query24	12646	3148	3138	3138
query25	702	405	412	405
query26	1909	174	169	169
query27	2993	336	334	334
query28	6775	1863	1849	1849
query29	1337	602	603	602
query30	326	170	168	168
query31	998	719	747	719
query32	98	65	61	61
query33	714	261	262	261
query34	1020	498	512	498
query35	840	706	728	706
query36	1003	867	871	867
query37	269	81	85	81
query38	3498	3361	3412	3361
query39	1598	1560	1535	1535
query40	289	136	138	136
query41	47	45	43	43
query42	111	106	101	101
query43	441	405	412	405
query44	1101	704	711	704
query45	269	258	262	258
query46	1061	841	796	796
query47	1881	1773	1808	1773
query48	389	304	310	304
query49	1153	369	349	349
query50	802	405	396	396
query51	6880	6800	6795	6795
query52	111	97	101	97
query53	362	306	299	299
query54	323	236	233	233
query55	88	85	76	76
query56	236	218	228	218
query57	1221	1124	1139	1124
query58	245	219	221	219
query59	2444	2399	2355	2355
query60	276	236	241	236
query61	92	105	106	105
query62	720	450	450	450
query63	312	285	282	282
query64	6405	3028	3352	3028
query65	3077	3022	3000	3000
query66	1447	330	321	321
query67	15266	14778	14761	14761
query68	6645	574	591	574
query69	560	344	335	335
query70	1170	1127	1136	1127
query71	448	281	281	281
query72	6407	2744	2563	2563
query73	792	331	337	331
query74	6862	6341	6347	6341
query75	3165	2313	2309	2309
query76	3861	1141	1220	1141
query77	580	259	262	259
query78	10837	10109	10111	10109
query79	8228	541	544	541
query80	1387	458	456	456
query81	520	245	243	243
query82	1070	110	108	108
query83	213	172	172	172
query84	271	98	96	96
query85	1555	343	288	288
query86	465	276	306	276
query87	3705	3512	3461	3461
query88	4213	2358	2361	2358
query89	576	367	380	367
query90	1952	181	184	181
query91	138	109	106	106
query92	63	53	52	52
query93	6327	558	531	531
query94	1159	195	196	195
query95	1096	1093	1096	1093
query96	605	279	273	273
query97	2661	2509	2500	2500
query98	236	228	214	214
query99	1306	821	811	811
Total cold run time: 295278 ms
Total hot run time: 182629 ms

doris-robot · 2024-04-02T10:31:04Z

ClickBench: Total hot run time: 29.89 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a37a11be84cead0f55fff4f5bdccc8e02b3c1025, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.06	0.07
query5	0.47	0.48	0.49
query6	1.14	0.66	0.65
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.56	0.52	0.50
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	0.77	0.79	0.80
query15	0.87	0.85	0.82
query16	0.36	0.34	0.35
query17	0.98	0.96	1.00
query18	0.25	0.25	0.25
query19	1.82	1.73	1.75
query20	0.02	0.01	0.00
query21	15.42	0.76	0.71
query22	3.04	5.00	1.49
query23	17.56	1.44	1.11
query24	1.46	0.22	0.23
query25	0.14	0.09	0.09
query26	0.27	0.18	0.19
query27	0.08	0.08	0.07
query28	13.84	0.96	0.96
query29	12.61	3.46	3.36
query30	0.27	0.06	0.06
query31	2.86	0.40	0.38
query32	3.25	0.48	0.47
query33	2.82	2.91	2.86
query34	15.49	4.36	4.35
query35	4.38	4.36	4.36
query36	0.67	0.48	0.48
query37	0.20	0.16	0.18
query38	0.16	0.17	0.15
query39	0.05	0.04	0.04
query40	0.18	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 105.76 s
Total hot run time: 29.89 s

doris-robot · 2024-04-02T10:34:13Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a37a11be84cead0f55fff4f5bdccc8e02b3c1025 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       16.1 seconds inserted 10000000 Rows, about 621K ops/s

morningman

LGTM

github-actions · 2024-04-02T13:23:08Z

PR approved by at least one committer and no changes requested.

kaka11chen

LGTM

morningman · 2024-04-02T14:57:49Z

run buildall

doris-robot · 2024-04-02T15:31:39Z

TPC-H: Total hot run time: 39256 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8c1dff8a1769cbf4a2ad7b010b47254ab384236, data reload: false

------ Round 1 ----------------------------------
q1	17646	4357	4281	4281
q2	2003	178	180	178
q3	10611	1249	1431	1249
q4	10368	922	1033	922
q5	7498	2954	2948	2948
q6	220	131	136	131
q7	1101	647	610	610
q8	9412	2071	2048	2048
q9	6749	6220	6209	6209
q10	8477	3545	3526	3526
q11	438	253	242	242
q12	384	226	216	216
q13	17868	2913	2920	2913
q14	276	234	237	234
q15	517	479	486	479
q16	503	388	399	388
q17	971	911	899	899
q18	7297	6567	6476	6476
q19	1627	1564	1559	1559
q20	602	316	311	311
q21	3619	3137	3153	3137
q22	369	300	313	300
Total cold run time: 108556 ms
Total hot run time: 39256 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4146	4080	4056	4056
q2	333	223	220	220
q3	2962	2976	2969	2969
q4	1915	1868	1859	1859
q5	5265	5215	5208	5208
q6	207	125	122	122
q7	2249	1791	1834	1791
q8	3242	3305	3291	3291
q9	8519	8546	8504	8504
q10	3764	3849	3841	3841
q11	532	453	441	441
q12	721	536	555	536
q13	15732	2881	2912	2881
q14	292	264	271	264
q15	511	474	472	472
q16	460	401	420	401
q17	1738	1703	1689	1689
q18	7721	8096	7819	7819
q19	1755	1773	1724	1724
q20	2081	1823	1796	1796
q21	5302	4968	4999	4968
q22	489	449	441	441
Total cold run time: 69936 ms
Total hot run time: 55293 ms

) Issue Number: #31442

[feature](multi-catalog)process tbl/db exist when create/drop db/tbl

a6a7ce3

wsjz and others added 2 commits April 2, 2024 12:57

fix ut

705ab64

Merge branch 'master' into support_exist

7316605

wsjz changed the title ~~[feature](multi-catalog)process tbl/db exist when create/drop db/tbl~~ [feature](insert)process tbl/db exist when create/drop db/tbl Apr 2, 2024

wuwenchi approved these changes Apr 2, 2024

View reviewed changes

github-actions bot added the reviewed label Apr 2, 2024

morningman reviewed Apr 2, 2024

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetadataOps.java Outdated Show resolved Hide resolved

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataOps.java Outdated Show resolved Hide resolved

fix tbl check

e279b39

fix db exists

a37a11b

morningman previously approved these changes Apr 2, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 2, 2024

kaka11chen approved these changes Apr 2, 2024

View reviewed changes

Merge branch 'master' into support_exist

d8c1dff

morningman dismissed their stale review via d8c1dff April 2, 2024 14:57

morningman approved these changes Apr 2, 2024

View reviewed changes

morningman changed the title ~~[feature](insert)process tbl/db exist when create/drop db/tbl~~ [feature](external) process tbl/db exist when create/drop db/tbl Apr 2, 2024

morningman merged commit 10ff3f4 into apache:master Apr 3, 2024

yiguolei added the 2.1.0-conflict label Apr 10, 2024

morningman added dev/2.1.3-merged and removed 2.1.0-conflict labels Apr 12, 2024

morningman pushed a commit that referenced this pull request Apr 12, 2024

[feature](external) process tbl/db exist when create/drop db/tbl (#33119

f0ac21e

) Issue Number: #31442

wsjz deleted the support_exist branch July 4, 2024 08:35

[feature](external) process tbl/db exist when create/drop db/tbl #33119

[feature](external) process tbl/db exist when create/drop db/tbl #33119

Uh oh!

Conversation

wsjz commented Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented Apr 1, 2024

Uh oh!

wsjz commented Apr 1, 2024

Uh oh!

wsjz commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

wsjz commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

github-actions bot commented Apr 2, 2024

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

wsjz commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

wsjz commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2024

Uh oh!

kaka11chen left a comment

Choose a reason for hiding this comment

Uh oh!

morningman commented Apr 2, 2024

Uh oh!

doris-robot commented Apr 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wsjz commented Apr 1, 2024 •

edited

Loading