Skip to content

[fix](txn insert) Forbid delete condition in txn insert#37135

Merged
dataroaring merged 1 commit intoapache:masterfrom
mymeiyi:txn-delete
Jul 2, 2024
Merged

[fix](txn insert) Forbid delete condition in txn insert#37135
dataroaring merged 1 commit intoapache:masterfrom
mymeiyi:txn-delete

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Jul 2, 2024

Proposed changes

Currently, there are two kinds of delete in doris, one is delete-where which is implemented via fast path, while the other one is delete-using which is implemented via insert into alike.

The first one can read writings in the same txn while the second one can not. This is confusing for users, so we just disable delete after insert and update now.

doc: apache/doris-website#820

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jul 2, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40418 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3f06e48c77125a4aed150d812be588c4361793ce, data reload: false

------ Round 1 ----------------------------------
q1	18275	4337	4277	4277
q2	2019	191	184	184
q3	10471	1248	1138	1138
q4	10638	827	762	762
q5	7519	2689	2602	2602
q6	244	139	141	139
q7	965	595	602	595
q8	9215	2047	2083	2047
q9	9297	6545	6533	6533
q10	9030	3919	3874	3874
q11	468	228	252	228
q12	456	235	237	235
q13	17926	3258	3235	3235
q14	287	248	249	248
q15	541	502	497	497
q16	835	411	408	408
q17	1134	734	760	734
q18	8960	7671	7402	7402
q19	7393	1494	1472	1472
q20	672	332	347	332
q21	4880	3150	3312	3150
q22	382	326	333	326
Total cold run time: 121607 ms
Total hot run time: 40418 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4378	4194	4206	4194
q2	400	284	279	279
q3	3096	2965	2878	2878
q4	2040	1790	1742	1742
q5	5497	5448	5542	5448
q6	217	142	139	139
q7	2197	1822	1906	1822
q8	3272	3400	3377	3377
q9	8643	8852	8694	8694
q10	4028	3765	3847	3765
q11	597	501	495	495
q12	807	637	641	637
q13	16546	3186	3201	3186
q14	310	285	303	285
q15	527	478	485	478
q16	496	431	432	431
q17	1820	1531	1489	1489
q18	8216	8021	7883	7883
q19	1756	1469	1621	1469
q20	3015	1885	1878	1878
q21	5052	4715	4824	4715
q22	643	536	580	536
Total cold run time: 73553 ms
Total hot run time: 55820 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174239 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3f06e48c77125a4aed150d812be588c4361793ce, data reload: false

query1	917	376	380	376
query2	6391	2322	2431	2322
query3	6651	211	222	211
query4	19005	17359	17189	17189
query5	3672	485	474	474
query6	276	171	171	171
query7	4592	293	298	293
query8	299	275	288	275
query9	8516	2352	2339	2339
query10	565	370	276	276
query11	10630	10123	9964	9964
query12	113	85	80	80
query13	1640	349	349	349
query14	9066	8250	7986	7986
query15	251	182	185	182
query16	8005	260	263	260
query17	1903	547	525	525
query18	1971	263	261	261
query19	191	149	149	149
query20	88	81	85	81
query21	210	139	133	133
query22	4287	4108	4013	4013
query23	33898	33735	33496	33496
query24	10751	2902	2912	2902
query25	626	372	367	367
query26	1172	159	159	159
query27	2427	313	338	313
query28	7339	2075	2081	2075
query29	909	638	643	638
query30	279	161	156	156
query31	1013	755	750	750
query32	126	54	52	52
query33	750	299	284	284
query34	1076	486	519	486
query35	767	657	631	631
query36	1167	995	976	976
query37	168	78	83	78
query38	2954	2862	2828	2828
query39	899	867	847	847
query40	230	132	131	131
query41	52	53	53	53
query42	116	101	98	98
query43	583	548	527	527
query44	1175	745	721	721
query45	193	163	174	163
query46	1081	733	706	706
query47	1834	1769	1777	1769
query48	372	304	296	296
query49	945	420	435	420
query50	768	379	385	379
query51	6924	6847	6826	6826
query52	106	93	96	93
query53	376	300	292	292
query54	890	446	447	446
query55	76	76	77	76
query56	307	283	292	283
query57	1111	1024	1050	1024
query58	260	254	257	254
query59	3525	3073	3202	3073
query60	324	300	326	300
query61	118	111	116	111
query62	605	446	451	446
query63	331	314	297	297
query64	8952	2353	1845	1845
query65	3179	3093	3127	3093
query66	755	339	342	339
query67	15421	14913	15094	14913
query68	4598	529	545	529
query69	583	423	330	330
query70	1099	1150	1134	1134
query71	446	293	283	283
query72	8243	6085	5134	5134
query73	730	331	336	331
query74	5790	5417	5506	5417
query75	4166	2687	2658	2658
query76	3173	991	1024	991
query77	685	298	305	298
query78	10388	9935	9648	9648
query79	7344	507	510	507
query80	2472	483	525	483
query81	563	222	234	222
query82	1605	114	101	101
query83	315	170	167	167
query84	270	88	86	86
query85	1537	282	276	276
query86	464	305	285	285
query87	3303	3083	3103	3083
query88	4919	2356	2353	2353
query89	514	420	401	401
query90	1963	181	187	181
query91	127	98	98	98
query92	61	50	49	49
query93	5817	496	484	484
query94	1332	185	187	185
query95	407	315	326	315
query96	611	265	261	261
query97	3214	3032	3014	3014
query98	215	199	188	188
query99	1209	828	826	826
Total cold run time: 284332 ms
Total hot run time: 174239 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3f06e48c77125a4aed150d812be588c4361793ce, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.10	0.09
query5	0.50	0.48	0.49
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.54	0.49	0.50
query10	0.54	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.59	0.59	0.58
query14	0.80	0.77	0.77
query15	0.85	0.81	0.81
query16	0.37	0.36	0.36
query17	1.01	0.97	0.95
query18	0.21	0.25	0.24
query19	1.79	1.66	1.75
query20	0.02	0.01	0.01
query21	15.44	0.78	0.67
query22	4.77	6.76	2.10
query23	18.31	1.26	1.24
query24	2.07	0.23	0.22
query25	0.15	0.09	0.08
query26	0.28	0.18	0.18
query27	0.08	0.08	0.08
query28	13.24	1.02	0.99
query29	12.63	3.34	3.33
query30	0.25	0.06	0.06
query31	2.91	0.39	0.39
query32	3.25	0.48	0.46
query33	2.88	2.91	2.90
query34	17.13	4.40	4.46
query35	4.53	4.46	4.55
query36	0.65	0.46	0.46
query37	0.19	0.16	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.04	0.04
query42	0.06	0.04	0.04
query43	0.05	0.05	0.04
Total cold run time: 110.06 s
Total hot run time: 30.7 s

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jul 2, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39442 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b00c371943b8f44218adb0d0dfaa3a9457d5f9b, data reload: false

------ Round 1 ----------------------------------
q1	17623	4394	4277	4277
q2	2014	187	183	183
q3	10482	1146	1015	1015
q4	10196	809	709	709
q5	7471	2674	2602	2602
q6	218	139	138	138
q7	960	602	600	600
q8	9218	2056	2045	2045
q9	8875	6473	6452	6452
q10	8942	3702	3718	3702
q11	454	230	233	230
q12	471	229	234	229
q13	17931	2978	2967	2967
q14	265	218	213	213
q15	529	474	495	474
q16	527	375	374	374
q17	949	672	645	645
q18	8093	7383	7335	7335
q19	6180	1403	1480	1403
q20	651	325	329	325
q21	4914	3236	3189	3189
q22	388	335	339	335
Total cold run time: 117351 ms
Total hot run time: 39442 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4443	4293	4271	4271
q2	366	276	267	267
q3	3004	2767	2820	2767
q4	1970	1764	1657	1657
q5	5630	5519	5436	5436
q6	224	131	130	130
q7	2182	1828	1882	1828
q8	3260	3407	3411	3407
q9	8698	8676	8862	8676
q10	4077	3909	3782	3782
q11	576	510	482	482
q12	827	660	651	651
q13	16156	3217	3167	3167
q14	304	290	282	282
q15	532	507	496	496
q16	475	423	431	423
q17	1816	1528	1481	1481
q18	8231	7919	7884	7884
q19	1775	1559	1698	1559
q20	2118	1887	1871	1871
q21	9221	4859	4977	4859
q22	596	535	558	535
Total cold run time: 76481 ms
Total hot run time: 55911 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172882 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b00c371943b8f44218adb0d0dfaa3a9457d5f9b, data reload: false

query1	930	378	382	378
query2	6441	2633	2349	2349
query3	6625	209	221	209
query4	19083	17485	17296	17296
query5	3666	493	465	465
query6	263	184	177	177
query7	4593	285	288	285
query8	305	309	280	280
query9	8578	2355	2343	2343
query10	572	292	282	282
query11	10629	10117	9939	9939
query12	121	91	81	81
query13	1642	358	362	358
query14	10095	6676	7568	6676
query15	229	197	196	196
query16	7848	270	272	270
query17	1906	571	539	539
query18	1998	277	272	272
query19	207	158	159	158
query20	87	84	90	84
query21	212	129	124	124
query22	4225	4003	4111	4003
query23	33698	33647	33520	33520
query24	10715	2867	2878	2867
query25	618	403	384	384
query26	718	155	156	155
query27	2272	341	333	333
query28	5987	2108	2104	2104
query29	922	623	640	623
query30	255	164	164	164
query31	1015	806	766	766
query32	99	56	55	55
query33	685	310	307	307
query34	874	471	477	471
query35	860	637	610	610
query36	1098	964	986	964
query37	144	85	84	84
query38	2935	2826	2866	2826
query39	883	849	838	838
query40	205	129	134	129
query41	51	50	51	50
query42	105	99	101	99
query43	608	559	543	543
query44	1069	728	716	716
query45	196	165	163	163
query46	1071	693	721	693
query47	1861	1755	1729	1729
query48	361	292	298	292
query49	861	399	396	396
query50	767	383	387	383
query51	6951	6801	6829	6801
query52	109	90	94	90
query53	365	297	291	291
query54	890	432	434	432
query55	73	79	76	76
query56	275	268	269	268
query57	1120	1042	1026	1026
query58	254	251	242	242
query59	3491	3157	3074	3074
query60	331	278	276	276
query61	92	94	87	87
query62	598	458	437	437
query63	332	286	284	284
query64	8493	2243	1733	1733
query65	3147	3067	3077	3067
query66	743	322	319	319
query67	15576	14799	14865	14799
query68	6258	530	521	521
query69	724	481	339	339
query70	1193	1094	1133	1094
query71	498	284	284	284
query72	8833	5356	5224	5224
query73	761	328	319	319
query74	5864	5468	5487	5468
query75	4736	2668	2683	2668
query76	4240	1008	950	950
query77	739	301	300	300
query78	10414	9921	9862	9862
query79	4944	511	509	509
query80	1080	475	480	475
query81	552	218	216	216
query82	1463	113	106	106
query83	362	173	173	173
query84	270	90	83	83
query85	1339	286	268	268
query86	445	350	277	277
query87	3265	3070	3118	3070
query88	4596	2370	2364	2364
query89	486	387	380	380
query90	1895	182	178	178
query91	125	97	97	97
query92	61	48	49	48
query93	5439	491	496	491
query94	1176	190	185	185
query95	407	315	311	311
query96	619	268	271	268
query97	3172	2984	3003	2984
query98	218	207	206	206
query99	1113	865	834	834
Total cold run time: 281315 ms
Total hot run time: 172882 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b00c371943b8f44218adb0d0dfaa3a9457d5f9b, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.10	0.10
query5	0.49	0.49	0.49
query6	1.15	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.55	0.50	0.50
query10	0.55	0.55	0.54
query11	0.15	0.12	0.12
query12	0.16	0.12	0.13
query13	0.59	0.59	0.59
query14	0.80	0.78	0.79
query15	0.84	0.82	0.83
query16	0.36	0.37	0.35
query17	1.03	1.06	1.01
query18	0.21	0.28	0.24
query19	1.80	1.65	1.82
query20	0.01	0.01	0.02
query21	15.43	0.78	0.67
query22	4.61	6.63	2.02
query23	18.30	1.47	1.30
query24	2.14	0.23	0.24
query25	0.15	0.09	0.09
query26	0.26	0.18	0.18
query27	0.08	0.07	0.08
query28	13.22	1.01	1.00
query29	12.64	3.29	3.28
query30	0.25	0.06	0.06
query31	2.86	0.38	0.39
query32	3.29	0.48	0.47
query33	2.88	2.85	2.96
query34	16.95	4.40	4.40
query35	4.53	4.48	4.48
query36	0.65	0.46	0.46
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.10	0.04	0.04
query42	0.05	0.05	0.05
query43	0.04	0.05	0.04
Total cold run time: 109.75 s
Total hot run time: 30.76 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by anyone and no changes requested.

Copy link
Collaborator

@Yukang-Lian Yukang-Lian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit ec264b8 into apache:master Jul 2, 2024
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
## Proposed changes

Currently, there are two kind of delete in doris, one is delete where
which is implemented via fast path, while the other one is delete using
which is implemented via insert into alike.

The first one can read writings in the same txn while the second one can
not. This is confusing for users, so we just disable delete after insert
and update now.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants