Skip to content

[fix](Nereids) decimalv3 cast in fe produce wrong data (#29808)#30522

Merged
morrySnow merged 1 commit intoapache:branch-2.0from
morrySnow:branch-2.0
Jan 30, 2024
Merged

[fix](Nereids) decimalv3 cast in fe produce wrong data (#29808)#30522
morrySnow merged 1 commit intoapache:branch-2.0from
morrySnow:branch-2.0

Conversation

@morrySnow
Copy link
Contributor

case:

MySQL root@127.0.0.1:test> select cast(12 as decimalv3(2,1))
+-----------------------------+
| cast(12 as DECIMALV3(2, 1)) |
+-----------------------------+
| 12.0                        |
+-----------------------------+

decimalv2 literal will generate wrong result too. But it is not only bugs in planner, but also have bugs in executor. So we need fix executor bug in another PR.

pick from master #29808
commi id 88ef595

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

case:
```
MySQL root@127.0.0.1:test> select cast(12 as decimalv3(2,1))
+-----------------------------+
| cast(12 as DECIMALV3(2, 1)) |
+-----------------------------+
| 12.0                        |
+-----------------------------+
```

decimalv2 literal will generate wrong result too. But it is not only
bugs in planner, but also have bugs in executor. So we need fix executor
bug in another PR.

pick from master apache#29808
commi id 88ef595
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49812 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 849c0fe07ff516fc0cb7ce718c49b06c886aeca0, data reload: false

------ Round 1 ----------------------------------
q1	17898	4404	4360	4360
q2	2054	148	139	139
q3	10457	1897	1900	1897
q4	10344	1240	1336	1240
q5	8473	3950	4004	3950
q6	227	122	122	122
q7	2025	1623	1612	1612
q8	9510	2686	2707	2686
q9	13307	10460	10413	10413
q10	8670	3523	3465	3465
q11	407	237	249	237
q12	474	290	310	290
q13	18360	3931	4048	3931
q14	352	315	325	315
q15	515	462	460	460
q16	679	596	588	588
q17	1127	943	978	943
q18	7276	6912	6857	6857
q19	1677	1531	1556	1531
q20	544	308	305	305
q21	4450	4080	4104	4080
q22	507	400	391	391
Total cold run time: 119333 ms
Total hot run time: 49812 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4336	4290	4282	4282
q2	319	217	223	217
q3	4142	4167	4131	4131
q4	2759	2744	2747	2744
q5	7244	7142	7108	7108
q6	236	120	117	117
q7	3293	2884	2836	2836
q8	4319	4427	4491	4427
q9	17025	17009	16950	16950
q10	4172	4257	4245	4245
q11	758	680	664	664
q12	1026	838	858	838
q13	7059	3746	3748	3746
q14	446	412	440	412
q15	499	459	454	454
q16	755	692	693	692
q17	3815	3830	3891	3830
q18	8722	8896	8930	8896
q19	1716	1652	1677	1652
q20	2359	2134	2077	2077
q21	8536	8466	8431	8431
q22	1019	957	946	946
Total cold run time: 84555 ms
Total hot run time: 79695 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 237203 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 849c0fe07ff516fc0cb7ce718c49b06c886aeca0, data reload: false

query1	945	387	378	378
query2	6520	2145	2302	2145
query3	6921	198	195	195
query4	20088	18089	18037	18037
query5	19725	6241	6225	6225
query6	279	216	226	216
query7	4159	303	294	294
query8	244	256	227	227
query9	3091	2674	2627	2627
query10	416	304	307	304
query11	11358	10645	10623	10623
query12	128	74	71	71
query13	5653	645	622	622
query14	17871	13039	13360	13039
query15	359	245	241	241
query16	6992	267	264	264
query17	3235	891	869	869
query18	2263	409	407	407
query19	215	148	149	148
query20	79	76	76	76
query21	188	97	98	97
query22	5118	4928	5048	4928
query23	32650	32420	32710	32420
query24	8902	6585	6569	6569
query25	562	437	427	427
query26	527	171	157	157
query27	1898	291	296	291
query28	6140	2259	2206	2206
query29	2960	2889	2893	2889
query30	248	162	160	160
query31	917	761	775	761
query32	63	58	57	57
query33	492	258	249	249
query34	839	453	471	453
query35	1104	895	836	836
query36	1885	1556	1508	1508
query37	84	57	56	56
query38	3057	2952	2951	2951
query39	1361	1337	1338	1337
query40	203	95	98	95
query41	34	32	32	32
query42	84	85	80	80
query43	593	521	604	521
query44	1149	710	704	704
query45	238	237	228	228
query46	1228	937	976	937
query47	1781	1758	1684	1684
query48	971	674	664	664
query49	602	375	364	364
query50	858	586	580	580
query51	5651	5543	5564	5543
query52	83	86	75	75
query53	437	321	314	314
query54	2427	2244	2248	2244
query55	79	79	74	74
query56	223	215	184	184
query57	1180	1127	1038	1038
query58	218	202	197	197
query59	3565	3240	3287	3240
query60	207	199	182	182
query61	89	91	87	87
query62	828	467	501	467
query63	462	336	341	336
query64	1999	1326	1313	1313
query65	3650	3603	3595	3595
query66	802	364	351	351
query67	16716	16313	15484	15484
query68	8495	637	647	637
query69	567	349	339	339
query70	1871	2011	1595	1595
query71	404	320	296	296
query72	4615	3383	3407	3383
query73	721	313	312	312
query74	6360	5852	5918	5852
query75	4224	3508	3500	3500
query76	4817	1192	1211	1192
query77	658	253	249	249
query78	30934	49104	46306	46306
query79	14687	630	636	630
query80	4499	376	381	376
query81	576	230	228	228
query82	703	95	94	94
query83	378	139	130	130
query84	257	69	68	68
query85	2392	285	293	285
query86	411	410	354	354
query87	3292	3046	2997	2997
query88	6712	2313	2306	2306
query89	371	289	298	289
query90	2521	210	215	210
query91	162	118	118	118
query92	70	53	52	52
query93	4264	594	531	531
query94	1621	205	213	205
query95	1123	1048	1070	1048
query96	648	324	315	315
query97	6462	6280	6415	6280
query98	184	176	172	172
query99	3873	881	926	881
Total cold run time: 344765 ms
Total hot run time: 237203 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 849c0fe07ff516fc0cb7ce718c49b06c886aeca0, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.05
query4	1.81	0.07	0.07
query5	0.53	0.53	0.52
query6	1.23	0.62	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.03
query9	0.54	0.46	0.48
query10	0.54	0.55	0.55
query11	0.11	0.08	0.08
query12	0.11	0.09	0.09
query13	0.63	0.61	0.61
query14	0.79	0.80	0.79
query15	0.79	0.75	0.76
query16	0.37	0.37	0.36
query17	0.99	1.00	1.01
query18	0.23	0.27	0.22
query19	1.91	1.90	1.84
query20	0.01	0.00	0.01
query21	15.48	0.54	0.56
query22	1.89	2.48	1.36
query23	16.64	1.16	1.03
query24	7.36	2.96	0.56
query25	2.24	0.15	0.14
query26	0.15	0.14	0.14
query27	0.15	0.15	0.14
query28	5.20	0.78	0.78
query29	12.63	2.20	2.38
query30	0.61	0.52	0.51
query31	2.80	0.39	0.37
query32	3.39	0.51	0.50
query33	3.06	3.08	3.06
query34	15.27	4.78	4.74
query35	4.84	4.83	4.86
query36	1.05	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.11 s
Total hot run time: 30.25 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 849c0fe07ff516fc0cb7ce718c49b06c886aeca0 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@morrySnow morrySnow merged commit 08db6b0 into apache:branch-2.0 Jan 30, 2024
@morrySnow morrySnow deleted the branch-2.0 branch January 30, 2024 02:13
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Apr 9, 2024
…apache#30522)

case:
```
MySQL root@127.0.0.1:test> select cast(12 as decimalv3(2,1))
+-----------------------------+
| cast(12 as DECIMALV3(2, 1)) |
+-----------------------------+
| 12.0                        |
+-----------------------------+
```

decimalv2 literal will generate wrong result too. But it is not only
bugs in planner, but also have bugs in executor. So we need fix executor
bug in another PR.

pick from master apache#29808
commi id 88ef595
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…apache#30522)

case:
```
MySQL root@127.0.0.1:test> select cast(12 as decimalv3(2,1))
+-----------------------------+
| cast(12 as DECIMALV3(2, 1)) |
+-----------------------------+
| 12.0                        |
+-----------------------------+
```

decimalv2 literal will generate wrong result too. But it is not only
bugs in planner, but also have bugs in executor. So we need fix executor
bug in another PR.

pick from master apache#29808
commi id 88ef595
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants