[enchancement](delete) fix delete stmt return error with fold on be #28557

zhangstar333 · 2023-12-18T07:18:06Z

Proposed changes

set enable_fold_constant_by_be = true;
DELETE FROM test  WHERE col_2 = cast(123.45 as decimalv3(10,3));
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Right expr of binary predicate should be value, predicate: `col_2` = CAST(CAST(123.45 AS DECIMALV3(10, 3)) AS DECIMAL(10, 3)), right expr type:DECIMAL(10, 3)

exprRewriter.rewrite() function will not apply when:
      if (rule instanceof FoldConstantsRule && analyzer.safeIsEnableFoldConstantByBe()) {
          continue;
      }

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

zhangstar333 · 2023-12-18T07:18:17Z

run buildall

doris-robot · 2023-12-18T08:04:07Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.9 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.9 seconds inserted 10000000 Rows, about 358K ops/s
storage size: 17219889980 Bytes

doris-robot · 2023-12-18T08:16:15Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit f87408ad4ffb3800cc4271860326498571f3baff, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4710	4521	4459	4459
q2	367	148	158	148
q3	1468	1266	1242	1242
q4	1115	909	938	909
q5	3146	3186	3158	3158
q6	248	128	129	128
q7	992	481	486	481
q8	2198	2224	2202	2202
q9	6703	6656	6697	6656
q10	3227	3280	3243	3243
q11	331	208	205	205
q12	357	214	201	201
q13	4560	3804	3811	3804
q14	241	216	217	216
q15	567	511	521	511
q16	446	383	387	383
q17	1019	580	477	477
q18	7222	6932	7074	6932
q19	1562	1440	1386	1386
q20	544	328	297	297
q21	3071	2641	2654	2641
q22	345	277	286	277
Total cold run time: 44439 ms
Total hot run time: 39956 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4409	4373	4379	4373
q2	273	161	170	161
q3	3538	3522	3524	3522
q4	2380	2371	2354	2354
q5	5725	5726	5725	5725
q6	243	120	121	120
q7	2391	1867	1844	1844
q8	3541	3540	3577	3540
q9	9002	8980	9017	8980
q10	3915	3992	4014	3992
q11	501	376	392	376
q12	762	608	590	590
q13	4285	3570	3566	3566
q14	293	257	252	252
q15	570	521	526	521
q16	519	441	465	441
q17	1899	1851	1844	1844
q18	8724	8176	8204	8176
q19	1724	1766	1713	1713
q20	2263	1945	1921	1921
q21	6511	6149	6158	6149
q22	497	436	416	416
Total cold run time: 63965 ms
Total hot run time: 60576 ms

github-actions · 2024-01-04T11:26:04Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-04T11:26:07Z

PR approved by anyone and no changes requested.

fe/fe-core/src/main/java/org/apache/doris/analysis/DeleteStmt.java

zhangstar333 · 2024-01-04T13:00:58Z

run buildall

doris-robot · 2024-01-04T13:29:19Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit dcb088ffeb3e2b6e52a0a144acb0c545dcb092db, data reload: false

------ Round 1 ----------------------------------
q1	17681	5117	5092	5092
q2	2028	152	141	141
q3	10544	1098	1110	1098
q4	10176	836	847	836
q5	7783	2925	2830	2830
q6	213	136	140	136
q7	925	530	504	504
q8	9270	2024	2031	2024
q9	6845	6394	6398	6394
q10	8216	2991	3002	2991
q11	440	228	208	208
q12	395	223	229	223
q13	17997	3627	3605	3605
q14	235	219	214	214
q15	561	523	512	512
q16	452	407	394	394
q17	955	438	495	438
q18	7230	6642	6668	6642
q19	1597	1308	1387	1308
q20	698	354	316	316
q21	2828	2406	2399	2399
q22	375	319	320	319
Total cold run time: 107444 ms
Total hot run time: 38624 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5110	5088	5130	5088
q2	341	244	249	244
q3	3285	3265	3257	3257
q4	2113	2028	1984	1984
q5	5783	5766	5756	5756
q6	216	126	129	126
q7	2324	1922	1919	1919
q8	3370	3429	3476	3429
q9	8821	8734	8694	8694
q10	3778	3816	3843	3816
q11	582	498	496	496
q12	793	637	618	618
q13	6188	3230	3193	3193
q14	303	267	281	267
q15	582	523	525	523
q16	533	498	497	497
q17	1919	1737	1735	1735
q18	8656	8282	8239	8239
q19	1639	1591	1576	1576
q20	2182	1946	1941	1941
q21	5527	5317	5190	5190
q22	534	454	479	454
Total cold run time: 64579 ms
Total hot run time: 59042 ms

doris-robot · 2024-01-04T13:40:43Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit dcb088ffeb3e2b6e52a0a144acb0c545dcb092db, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	925	350	333	333
query2	6436	1921	2010	1921
query3	6669	210	202	202
query4	27262	22462	22347	22347
query5	3876	561	492	492
query6	261	182	173	173
query7	4587	256	264	256
query8	218	200	211	200
query9	8253	2542	2491	2491
query10	400	238	234	234
query11	16238	15413	15404	15404
query12	133	74	78	74
query13	1636	319	315	315
query14	11281	7087	7093	7087
query15	216	182	189	182
query16	6194	268	263	263
query17	1837	499	494	494
query18	1833	262	266	262
query19	178	135	127	127
query20	79	76	80	76
query21	187	99	94	94
query22	5098	4465	4731	4465
query23	31953	31258	31329	31258
query24	12060	2828	2874	2828
query25	617	343	349	343
query26	1819	136	139	136
query27	2894	275	274	274
query28	6982	1860	1857	1857
query29	2202	401	385	385
query30	299	141	151	141
query31	988	764	784	764
query32	87	55	55	55
query33	735	267	245	245
query34	923	439	442	439
query35	882	767	781	767
query36	1341	1236	1253	1236
query37	180	67	65	65
query38	3412	3302	3266	3266
query39	1319	1301	1282	1282
query40	299	88	88	88
query41	34	35	40	35
query42	89	86	80	80
query43	519	495	487	487
query44	1066	695	693	693
query45	192	186	185	185
query46	1045	634	642	634
query47	1606	1529	1604	1529
query48	321	245	249	245
query49	1218	326	317	317
query50	798	373	365	365
query51	5388	5359	5222	5222
query52	93	83	75	75
query53	209	144	141	141
query54	1382	555	598	555
query55	99	77	78	77
query56	180	186	195	186
query57	1008	910	974	910
query58	231	198	190	190
query59	2797	2794	2708	2708
query60	228	230	217	217
query61	87	87	85	85
query62	639	454	460	454
query63	171	147	146	146
query64	5814	1772	1716	1716
query65	3329	3213	3242	3213
query66	1386	333	341	333
query67	15701	15069	15077	15069
query68	10145	500	519	500
query69	528	255	235	235
query70	1730	1542	1529	1529
query71	477	219	215	215
query72	5379	3598	3585	3585
query73	2043	312	305	305
query74	7004	6434	6494	6434
query75	4616	2281	2276	2276
query76	6237	1137	1111	1111
query77	658	247	254	247
query78	9149	8710	8664	8664
query79	1012	504	487	487
query80	545	365	350	350
query81	451	213	212	212
query82	222	98	92	92
query83	166	138	136	136
query84	250	53	54	53
query85	956	293	283	283
query86	386	396	406	396
query87	3586	3385	3322	3322
query88	2838	2272	2268	2268
query89	334	253	271	253
query90	1885	194	193	193
query91	122	93	102	93
query92	62	51	54	51
query93	1021	421	422	421
query94	812	180	174	174
query95	450	422	405	405
query96	617	320	319	319
query97	4291	4186	4152	4152
query98	198	198	188	188
query99	1141	815	831	815
Total cold run time: 287730 ms
Total hot run time: 178552 ms

doris-robot · 2024-01-04T13:55:05Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit dcb088ffeb3e2b6e52a0a144acb0c545dcb092db, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5496	5134	5168	5134
q2	381	182	159	159
q3	1452	1145	1119	1119
q4	1094	881	788	788
q5	3179	3110	3075	3075
q6	222	134	133	133
q7	956	557	531	531
q8	2146	2265	2241	2241
q9	6710	6647	6688	6647
q10	3169	3113	3148	3113
q11	337	231	215	215
q12	382	238	238	238
q13	4426	3643	3633	3633
q14	252	214	211	211
q15	593	534	542	534
q16	452	391	403	391
q17	1041	564	605	564
q18	7077	6730	6784	6730
q19	1636	1463	1355	1355
q20	594	324	353	324
q21	2867	2454	2512	2454
q22	389	345	329	329
Total cold run time: 44851 ms
Total hot run time: 39918 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5075	5063	5017	5017
q2	342	237	236	236
q3	3367	3334	3319	3319
q4	2173	2066	2027	2027
q5	5937	5916	5923	5916
q6	227	128	129	128
q7	2419	1893	1922	1893
q8	3552	3653	3681	3653
q9	8984	9056	8981	8981
q10	3880	3905	3923	3905
q11	572	487	491	487
q12	804	647	656	647
q13	3873	3230	3216	3216
q14	306	274	286	274
q15	600	543	528	528
q16	551	500	496	496
q17	2027	1787	1810	1787
q18	8807	8285	8329	8285
q19	1741	1631	1667	1631
q20	2250	1972	1966	1966
q21	5840	5327	5469	5327
q22	565	473	515	473
Total cold run time: 63892 ms
Total hot run time: 60192 ms

doris-robot · 2024-01-04T13:56:12Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.48 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17183415924 Bytes

HappenLee

LGTM

github-actions · 2024-01-05T03:08:51Z

PR approved by at least one committer and no changes requested.

…pache#28557)

wm1581066 added the dev/2.0.4 label Dec 21, 2023

BiteTheDDDDt previously approved these changes Jan 4, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 4, 2024

github-actions bot added the reviewed label Jan 4, 2024

HappenLee reviewed Jan 4, 2024

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/analysis/DeleteStmt.java Outdated Show resolved Hide resolved

zhangstar333 added 2 commits January 4, 2024 20:53

[enchancement](delete) fix delete stmt return error with fold on be

421f53a

update

dcb088f

zhangstar333 dismissed BiteTheDDDDt’s stale review via dcb088f January 4, 2024 12:59

zhangstar333 force-pushed the partition_dependency branch from f87408a to dcb088f Compare January 4, 2024 12:59

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 4, 2024

HappenLee approved these changes Jan 5, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 5, 2024

zclllyybb approved these changes Jan 5, 2024

View reviewed changes

HappenLee merged commit f3bbc7b into apache:master Jan 5, 2024

wm1581066 removed the dev/2.0.4 label Jan 5, 2024

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024

[enchancement](delete) fix delete stmt return error with fold on be (a…

b4f6042

…pache#28557)

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[enchancement](delete) fix delete stmt return error with fold on be #28557

[enchancement](delete) fix delete stmt return error with fold on be #28557

Uh oh!

Conversation

zhangstar333 commented Dec 18, 2023

Proposed changes

Further comments

Uh oh!

zhangstar333 commented Dec 18, 2023

Uh oh!

doris-robot commented Dec 18, 2023

Uh oh!

doris-robot commented Dec 18, 2023

Uh oh!

github-actions bot commented Jan 4, 2024

Uh oh!

github-actions bot commented Jan 4, 2024

Uh oh!

Uh oh!

zhangstar333 commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants