Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](create table) create table fail not write drop table editlog #37488

Merged
merged 9 commits into from
Jul 9, 2024
Merged

[fix](create table) create table fail not write drop table editlog #37488

merged 9 commits into from
Jul 9, 2024

Conversation

yujun777
Copy link
Collaborator

@yujun777 yujun777 commented Jul 8, 2024 •
edited
Loading

Fix: for creating table, if create dynamic partition throw exception (pr #35778), then create table will fail, then drop table, but it forget to write an editlog.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Collaborator Author

yujun777 commented Jul 8, 2024

run buildall

@yujun777 yujun777 changed the title [fix](create table) create table fail no drop table [fix](create table) create table fail not write drop table editlog Jul 8, 2024
deardeng
deardeng approved these changes Jul 8, 2024
View reviewed changes
Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39926 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 472787ec34d37e91da2cf915f865ff3769ae3c4e, data reload: false

------ Round 1 ----------------------------------
q1	17602	4412	4316	4316
q2	2020	190	186	186
q3	10502	1177	1098	1098
q4	10188	802	848	802
q5	7493	2656	2591	2591
q6	222	138	135	135
q7	963	610	599	599
q8	9225	2057	2091	2057
q9	8830	6479	6485	6479
q10	8951	3746	3739	3739
q11	478	232	227	227
q12	458	232	223	223
q13	17766	3000	2982	2982
q14	274	228	229	228
q15	519	495	482	482
q16	516	377	373	373
q17	946	736	786	736
q18	7952	7400	7408	7400
q19	3489	1462	1460	1460
q20	654	317	323	317
q21	4818	3768	3155	3155
q22	394	344	341	341
Total cold run time: 114260 ms
Total hot run time: 39926 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4426	4236	4233	4233
q2	377	279	276	276
q3	2992	2737	2829	2737
q4	1902	1728	1686	1686
q5	5557	5673	5479	5479
q6	235	133	137	133
q7	2224	1805	1820	1805
q8	3337	3444	3416	3416
q9	8670	8707	8730	8707
q10	4114	3874	3873	3873
q11	561	487	501	487
q12	767	596	641	596
q13	16862	3174	3136	3136
q14	304	285	275	275
q15	539	488	503	488
q16	481	426	431	426
q17	1813	1526	1518	1518
q18	7931	7924	7957	7924
q19	4837	1648	1583	1583
q20	2168	1941	1864	1864
q21	5045	4951	4949	4949
q22	627	537	585	537
Total cold run time: 75769 ms
Total hot run time: 56128 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173624 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 472787ec34d37e91da2cf915f865ff3769ae3c4e, data reload: false

query1	911	367	361	361
query2	6445	2436	2226	2226
query3	6651	207	216	207
query4	27685	17717	17559	17559
query5	3632	482	478	478
query6	254	158	161	158
query7	4587	290	277	277
query8	330	303	300	300
query9	8423	2404	2384	2384
query10	586	292	288	288
query11	12285	10019	10087	10019
query12	117	88	82	82
query13	1648	374	368	368
query14	10149	8203	7692	7692
query15	235	190	180	180
query16	7607	320	306	306
query17	1815	547	523	523
query18	1923	277	272	272
query19	202	145	146	145
query20	86	81	82	81
query21	207	130	126	126
query22	4297	4079	4014	4014
query23	33922	33414	33366	33366
query24	10616	2881	2873	2873
query25	584	383	398	383
query26	708	150	153	150
query27	2259	273	276	273
query28	6112	2125	2126	2125
query29	875	642	630	630
query30	254	149	152	149
query31	983	766	758	758
query32	97	54	63	54
query33	655	304	285	285
query34	877	486	486	486
query35	733	593	573	573
query36	1148	967	989	967
query37	143	87	83	83
query38	2939	2813	2882	2813
query39	898	855	837	837
query40	219	135	122	122
query41	57	55	52	52
query42	115	104	101	101
query43	595	534	539	534
query44	1109	743	724	724
query45	192	162	158	158
query46	1077	711	706	706
query47	1850	1790	1816	1790
query48	368	291	298	291
query49	823	413	438	413
query50	767	385	386	385
query51	6819	6816	6796	6796
query52	108	90	94	90
query53	353	284	287	284
query54	873	444	439	439
query55	74	72	72	72
query56	287	256	268	256
query57	1117	1062	1037	1037
query58	248	244	238	238
query59	3135	3168	3333	3168
query60	296	273	278	273
query61	98	97	102	97
query62	774	667	638	638
query63	328	292	282	282
query64	9160	2207	1621	1621
query65	3205	3113	3116	3113
query66	753	325	328	325
query67	15327	14893	15363	14893
query68	4790	520	516	516
query69	696	443	360	360
query70	1144	1134	1146	1134
query71	421	289	277	277
query72	8768	5176	5681	5176
query73	761	315	314	314
query74	5902	5442	5416	5416
query75	3894	2682	2674	2674
query76	3181	960	892	892
query77	714	309	300	300
query78	9424	8969	8882	8882
query79	2197	515	514	514
query80	1193	467	551	467
query81	584	213	218	213
query82	762	131	127	127
query83	331	167	163	163
query84	255	85	84	84
query85	1515	307	301	301
query86	448	328	311	311
query87	3253	3051	3066	3051
query88	4268	2324	2325	2324
query89	488	376	384	376
query90	1823	188	185	185
query91	128	100	104	100
query92	57	49	50	49
query93	2378	503	495	495
query94	1167	208	212	208
query95	397	315	323	315
query96	583	262	264	262
query97	3184	3079	2998	2998
query98	228	202	190	190
query99	1533	1255	1201	1201
Total cold run time: 280472 ms
Total hot run time: 173624 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 472787ec34d37e91da2cf915f865ff3769ae3c4e, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.05
query4	1.67	0.09	0.09
query5	0.50	0.48	0.51
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.53	0.48	0.50
query10	0.55	0.54	0.53
query11	0.14	0.11	0.10
query12	0.14	0.12	0.11
query13	0.59	0.59	0.58
query14	0.78	0.78	0.78
query15	0.84	0.81	0.81
query16	0.38	0.37	0.36
query17	0.95	1.03	0.98
query18	0.22	0.22	0.20
query19	1.91	1.80	1.68
query20	0.02	0.01	0.01
query21	15.40	0.73	0.63
query22	4.40	7.43	1.99
query23	18.30	1.44	1.32
query24	2.06	0.24	0.22
query25	0.15	0.09	0.08
query26	0.29	0.21	0.21
query27	0.46	0.22	0.22
query28	13.30	1.01	1.00
query29	12.67	3.30	3.23
query30	0.26	0.06	0.06
query31	2.85	0.38	0.38
query32	3.29	0.46	0.45
query33	2.92	2.88	2.91
query34	17.13	4.34	4.38
query35	4.39	4.41	4.46
query36	0.64	0.48	0.47
query37	0.18	0.15	0.15
query38	0.15	0.14	0.13
query39	0.04	0.03	0.03
query40	0.15	0.12	0.12
query41	0.09	0.04	0.05
query42	0.05	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.97 s
Total hot run time: 30.5 s

@yujun777
Copy link
Collaborator Author

yujun777 commented Jul 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40262 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9261095ba197b88ae2c5d3570a77d006e9eb8d96, data reload: false

------ Round 1 ----------------------------------
q1	17609	4376	4287	4287
q2	2032	191	205	191
q3	10443	1167	1221	1167
q4	10187	767	893	767
q5	7489	2695	2674	2674
q6	222	135	136	135
q7	967	603	607	603
q8	9234	2095	2112	2095
q9	9256	6543	6583	6543
q10	9006	3734	3735	3734
q11	460	246	242	242
q12	487	228	234	228
q13	17773	2998	3018	2998
q14	258	228	231	228
q15	523	479	493	479
q16	510	386	381	381
q17	973	645	759	645
q18	8133	7563	7413	7413
q19	1683	1517	1486	1486
q20	653	325	337	325
q21	4994	3968	3299	3299
q22	404	342	346	342
Total cold run time: 113296 ms
Total hot run time: 40262 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4294	4237	4202	4202
q2	370	269	269	269
q3	3022	2775	2767	2767
q4	1885	1641	1579	1579
q5	5290	5320	5321	5320
q6	217	131	131	131
q7	2152	1786	1776	1776
q8	3218	3364	3342	3342
q9	8424	8429	8365	8365
q10	3914	3720	3669	3669
q11	576	484	497	484
q12	788	607	604	604
q13	15402	2977	3016	2977
q14	291	276	276	276
q15	520	478	467	467
q16	481	411	411	411
q17	1782	1500	1458	1458
q18	7568	7723	7482	7482
q19	1693	1582	1594	1582
q20	1981	1800	1821	1800
q21	4744	4818	4743	4743
q22	611	551	543	543
Total cold run time: 69223 ms
Total hot run time: 54247 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173658 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9261095ba197b88ae2c5d3570a77d006e9eb8d96, data reload: false

query1	916	377	363	363
query2	6467	2428	2293	2293
query3	6648	205	216	205
query4	27935	17661	17205	17205
query5	4203	464	472	464
query6	289	163	177	163
query7	4574	279	282	279
query8	314	291	299	291
query9	8438	2420	2409	2409
query10	622	299	300	299
query11	12517	10047	10345	10047
query12	134	88	86	86
query13	1680	367	368	367
query14	9897	7900	7782	7782
query15	232	196	185	185
query16	7757	312	320	312
query17	1804	569	549	549
query18	1405	283	284	283
query19	205	151	154	151
query20	92	88	84	84
query21	217	132	129	129
query22	4314	4134	3966	3966
query23	33924	33255	33193	33193
query24	9353	2862	2860	2860
query25	598	381	396	381
query26	700	150	151	150
query27	2160	272	273	272
query28	5949	2073	2062	2062
query29	876	652	657	652
query30	288	150	148	148
query31	949	745	750	745
query32	97	55	59	55
query33	701	304	312	304
query34	889	482	485	482
query35	710	611	572	572
query36	1089	941	947	941
query37	133	81	82	81
query38	2818	2732	2740	2732
query39	851	788	812	788
query40	213	129	126	126
query41	56	57	57	57
query42	117	96	103	96
query43	581	549	539	539
query44	1057	746	734	734
query45	191	165	162	162
query46	1074	716	697	697
query47	1895	1794	1776	1776
query48	365	297	308	297
query49	1057	433	414	414
query50	781	399	393	393
query51	6781	6824	6771	6771
query52	114	93	96	93
query53	368	291	287	287
query54	884	449	446	446
query55	76	76	76	76
query56	314	298	285	285
query57	1140	1069	1048	1048
query58	265	252	362	252
query59	3458	3076	3050	3050
query60	312	269	279	269
query61	97	93	90	90
query62	796	656	658	656
query63	316	289	292	289
query64	9219	2178	1638	1638
query65	3209	3090	3133	3090
query66	784	343	342	342
query67	15917	14994	15035	14994
query68	8351	540	530	530
query69	748	464	356	356
query70	1140	1076	1157	1076
query71	489	281	276	276
query72	8473	5332	5989	5332
query73	935	321	321	321
query74	5873	5538	5506	5506
query75	4744	2666	2709	2666
query76	4629	966	880	880
query77	797	295	294	294
query78	9715	9063	8961	8961
query79	8628	503	513	503
query80	1134	480	481	480
query81	597	216	232	216
query82	747	141	135	135
query83	333	166	163	163
query84	270	90	91	90
query85	1313	300	297	297
query86	435	321	336	321
query87	3260	3119	3110	3110
query88	5017	2385	2341	2341
query89	549	384	380	380
query90	1922	188	186	186
query91	129	102	100	100
query92	61	47	48	47
query93	6990	501	498	498
query94	1276	214	205	205
query95	402	314	307	307
query96	614	268	263	263
query97	3158	3095	3040	3040
query98	222	201	200	200
query99	1605	1281	1291	1281
Total cold run time: 298738 ms
Total hot run time: 173658 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.49 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9261095ba197b88ae2c5d3570a77d006e9eb8d96, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.04	0.04
query4	1.69	0.07	0.07
query5	0.50	0.50	0.50
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.51
query10	0.53	0.54	0.55
query11	0.14	0.11	0.11
query12	0.15	0.12	0.13
query13	0.60	0.59	0.59
query14	0.77	0.77	0.79
query15	0.86	0.82	0.82
query16	0.36	0.36	0.36
query17	1.00	1.05	1.05
query18	0.22	0.23	0.22
query19	1.84	1.83	1.71
query20	0.01	0.02	0.01
query21	15.44	0.75	0.64
query22	4.32	6.86	1.72
query23	18.27	1.43	1.30
query24	2.10	0.22	0.21
query25	0.14	0.08	0.08
query26	0.31	0.22	0.21
query27	0.46	0.23	0.23
query28	13.28	1.00	0.99
query29	12.63	3.29	3.26
query30	0.26	0.06	0.06
query31	2.89	0.39	0.39
query32	3.25	0.49	0.47
query33	2.91	2.95	2.86
query34	17.18	4.35	4.44
query35	4.44	4.51	4.45
query36	0.66	0.45	0.49
query37	0.19	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.15	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.02 s
Total hot run time: 30.49 s

dataroaring
dataroaring approved these changes Jul 9, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 9, 2024
@dataroaring dataroaring merged commit 45e6073 into apache:master Jul 9, 2024
25 of 27 checks passed
dataroaring pushed a commit that referenced this pull request Jul 9, 2024
@yujun777 @dataroaring
[fix](create table) create table fail not write drop table editlog (#…
a306588 
…37488)

Fix: for creating table, if create dynamic partition throw exception (pr
#35778), then create table will fail, then drop table, but it forget to
write an editlog.
yujun777 added a commit to yujun777/doris that referenced this pull request Jul 9, 2024
@yujun777
[fix](create table) create table fail not write drop table editlog (a…
1680a45 
…pache#37488)

Fix: for creating table, if create dynamic partition throw exception (pr
apache#35778), then create table will fail, then drop table, but it forget to
write an editlog.
@yujun777 yujun777 mentioned this pull request Jul 9, 2024
Merged
yujun777 added a commit to yujun777/doris that referenced this pull request Jul 9, 2024
@yujun777
[fix](create table) create table fail not write drop table editlog (a…
382debf 
…pache#37488)

Fix: for creating table, if create dynamic partition throw exception (pr
apache#35778), then create table will fail, then drop table, but it forget to
write an editlog.
@yujun777 yujun777 mentioned this pull request Jul 9, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jul 9, 2024
@yujun777
[fix](create table) create table fail not write drop table editlog #3…
9b500fa 
…7488 (#37506)

cherry pick from #37488
dataroaring pushed a commit that referenced this pull request Jul 9, 2024
@yujun777 @deardeng
[fix](create table) fix create table fail cause meta leaky #37488, #3…
6789a56 
…3574 (#37516)

cherry-pick:  #37488,  #33574

---------

Co-authored-by: deardeng <565620795@qq.com>
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@yujun777 @deardeng
[fix](create table) fix create table fail cause meta leaky apache#37488…
1e9c2fd 
…, apache#33574 (apache#37516)

cherry-pick:  apache#37488,  apache#33574

---------

Co-authored-by: deardeng <565620795@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@deardeng deardeng deardeng approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.13-merged dev/2.1.5-merged dev/3.0.0-merged p0_c reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yujun777 @doris-robot @deardeng @dataroaring