Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](mtmv) Fix duplicate column name not check when create materialized view #40658

Merged
merged 4 commits into from
Sep 20, 2024

Conversation

seawinde
Copy link
Contributor

Proposed changes

This is brought by #26146
If create materialized view as following, Should fail, because has the duplicated column name o_orderdatE and o_orderdate. But now can create materialized view successfully. the pr fix this.

        CREATE MATERIALIZED VIEW mv_1
        BUILD IMMEDIATE REFRESH AUTO ON MANUAL 
        partition by(o_orderdate) 
        DISTRIBUTED BY RANDOM BUCKETS 2 
        PROPERTIES ('replication_num' = '1') 
        AS  
        select o_orderdatE, o_shippriority, o_comment, o_orderdate, 
        sum(o_totalprice) as sum_total, 
        max(o_totalpricE) as max_total, 
        min(o_totalprice) as min_total, 
        count(*) as count_all, 
        bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) cnt_1, 
        bitmap_union(to_bitmap(case when o_shippriority > 2 and o_orderkey IN (2) then o_custkey else null end)) as cnt_2 
        from (select * from orders) as t1
        group by 
        o_orderdatE, 
        o_shippriority, 
        o_comment,
        o_orderdate;

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38377 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5f46338be73ee4c229dc9b37e43afd5d49f9e73, data reload: false

------ Round 1 ----------------------------------
q1	17610	4431	4319	4319
q2	2024	186	189	186
q3	11716	995	1057	995
q4	10411	709	703	703
q5	7731	2839	2788	2788
q6	225	138	137	137
q7	964	621	603	603
q8	9325	2063	2039	2039
q9	7151	6560	6610	6560
q10	7006	2285	2218	2218
q11	463	253	246	246
q12	402	228	230	228
q13	19004	3161	3163	3161
q14	290	234	254	234
q15	542	506	496	496
q16	517	431	440	431
q17	1020	696	721	696
q18	7437	6895	6988	6895
q19	1385	1152	991	991
q20	691	337	343	337
q21	4155	3262	3092	3092
q22	1133	1028	1022	1022
Total cold run time: 111202 ms
Total hot run time: 38377 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4401	4422	4954	4422
q2	381	272	274	272
q3	2951	2754	2728	2728
q4	2029	1702	1811	1702
q5	5788	5819	6034	5819
q6	225	138	138	138
q7	2391	1994	1933	1933
q8	3429	3622	3547	3547
q9	9127	9188	9048	9048
q10	3731	3408	3418	3408
q11	616	521	530	521
q12	835	692	678	678
q13	16219	3264	3308	3264
q14	324	293	289	289
q15	532	494	493	493
q16	563	498	506	498
q17	1828	1555	1474	1474
q18	8186	7839	7834	7834
q19	1802	1676	1698	1676
q20	2292	1907	1945	1907
q21	5913	5705	5349	5349
q22	1110	1060	1068	1060
Total cold run time: 74673 ms
Total hot run time: 58060 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197451 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5f46338be73ee4c229dc9b37e43afd5d49f9e73, data reload: false

query1	1260	903	902	902
query2	6434	1917	1911	1911
query3	10698	3987	4045	3987
query4	60554	28470	23253	23253
query5	4965	496	496	496
query6	398	177	166	166
query7	5627	299	293	293
query8	315	219	214	214
query9	7722	2486	2502	2486
query10	412	284	273	273
query11	16292	15085	15524	15085
query12	158	105	104	104
query13	1474	422	405	405
query14	10233	7002	7457	7002
query15	211	188	180	180
query16	6715	486	503	486
query17	1072	565	558	558
query18	1390	296	295	295
query19	203	148	149	148
query20	127	112	112	112
query21	229	106	106	106
query22	4860	4730	4781	4730
query23	34394	33589	33515	33515
query24	6009	2953	2817	2817
query25	487	392	396	392
query26	597	155	155	155
query27	1581	282	280	280
query28	3818	2058	2039	2039
query29	664	429	430	429
query30	250	156	152	152
query31	928	760	752	752
query32	77	57	53	53
query33	436	289	292	289
query34	859	491	478	478
query35	826	738	736	736
query36	1109	952	958	952
query37	143	95	91	91
query38	3968	3960	4059	3960
query39	1453	1421	1403	1403
query40	200	118	118	118
query41	49	50	48	48
query42	122	97	101	97
query43	508	464	466	464
query44	1086	757	740	740
query45	200	164	168	164
query46	1105	749	779	749
query47	1878	1795	1807	1795
query48	381	311	290	290
query49	794	463	466	463
query50	844	431	439	431
query51	7138	6798	7004	6798
query52	104	91	89	89
query53	254	185	186	185
query54	572	458	475	458
query55	78	80	76	76
query56	277	272	263	263
query57	1225	1080	1068	1068
query58	240	260	231	231
query59	3008	2820	2697	2697
query60	300	285	301	285
query61	183	100	102	100
query62	740	654	666	654
query63	213	188	186	186
query64	1399	688	699	688
query65	3234	3137	3168	3137
query66	662	337	333	333
query67	15923	15520	15370	15370
query68	1909	568	560	560
query69	395	259	276	259
query70	1208	1098	1144	1098
query71	326	277	271	271
query72	4684	4095	4023	4023
query73	739	327	328	327
query74	9317	8866	8926	8866
query75	3330	2721	2705	2705
query76	1345	1004	1024	1004
query77	513	329	312	312
query78	9777	9201	9723	9201
query79	1079	890	863	863
query80	1101	808	804	804
query81	520	267	264	264
query82	1382	272	264	264
query83	245	197	194	194
query84	265	108	110	108
query85	719	395	387	387
query86	335	319	325	319
query87	4439	4304	4465	4304
query88	4641	4098	4089	4089
query89	392	366	364	364
query90	1821	320	307	307
query91	122	174	124	124
query92	80	75	75	75
query93	953	927	926	926
query94	755	369	334	334
query95	440	415	422	415
query96	470	471	473	471
query97	3125	3059	3109	3059
query98	235	228	230	228
query99	1483	1264	1271	1264
Total cold run time: 299810 ms
Total hot run time: 197451 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5f46338be73ee4c229dc9b37e43afd5d49f9e73, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.52	0.51	0.50
query6	1.13	0.75	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.54	0.48	0.50
query10	0.54	0.55	0.56
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.60	0.59
query14	1.40	1.40	1.38
query15	0.85	0.84	0.82
query16	0.37	0.36	0.38
query17	1.08	0.99	0.99
query18	0.22	0.20	0.21
query19	1.86	1.73	1.78
query20	0.01	0.01	0.00
query21	15.39	0.69	0.66
query22	4.82	5.76	2.12
query23	18.23	1.37	1.24
query24	2.11	0.24	0.22
query25	0.16	0.09	0.08
query26	0.27	0.19	0.18
query27	0.08	0.07	0.07
query28	13.22	1.02	1.00
query29	12.62	3.37	3.40
query30	0.24	0.06	0.06
query31	2.87	0.40	0.39
query32	3.26	0.49	0.47
query33	3.00	3.01	3.02
query34	17.06	4.47	4.44
query35	4.55	4.50	4.52
query36	0.66	0.48	0.48
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.03	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.06
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.82 s
Total hot run time: 31.71 s

morrySnow
morrySnow previously approved these changes Sep 11, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 11, 2024
public void validateColumns(List<ColumnDefinition> columns) throws UserException {
Set<String> colSets = Sets.newTreeSet(String.CASE_INSENSITIVE_ORDER);
for (ColumnDefinition col : columns) {
if (!colSets.add(col.getName())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we check invalid character in column name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah,already have the logic to check invalid caracter in column name, and move the check invalid caracter logic to method validateColumns

Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 18, 2024
@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41634 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 297e3787077d9784d3cb85ef6ce7d5b3cba5a3ad, data reload: false

------ Round 1 ----------------------------------
q1	18160	7496	7314	7314
q2	2501	161	163	161
q3	11604	1180	1218	1180
q4	10662	743	737	737
q5	8393	3208	3146	3146
q6	239	156	155	155
q7	1031	635	615	615
q8	9818	2055	2035	2035
q9	6888	6394	6422	6394
q10	7020	2285	2300	2285
q11	429	261	263	261
q12	409	218	213	213
q13	17784	2987	2976	2976
q14	247	209	220	209
q15	573	525	517	517
q16	673	629	607	607
q17	979	816	809	809
q18	7217	6826	6662	6662
q19	1422	955	1075	955
q20	592	286	297	286
q21	4139	3191	3133	3133
q22	1125	994	984	984
Total cold run time: 111905 ms
Total hot run time: 41634 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7292	7257	7238	7238
q2	326	233	237	233
q3	2948	2749	2759	2749
q4	1943	1705	1698	1698
q5	5391	5446	5434	5434
q6	222	148	143	143
q7	2064	1742	1702	1702
q8	3255	3345	3338	3338
q9	8416	8411	8461	8411
q10	3384	3370	3393	3370
q11	568	461	470	461
q12	762	585	601	585
q13	5444	3026	3009	3009
q14	288	263	260	260
q15	568	518	519	518
q16	721	660	673	660
q17	1768	1569	1526	1526
q18	7669	7556	7531	7531
q19	1657	1518	1596	1518
q20	2038	1820	1836	1820
q21	5481	5083	5176	5083
q22	1133	1003	995	995
Total cold run time: 63338 ms
Total hot run time: 58282 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194119 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 297e3787077d9784d3cb85ef6ce7d5b3cba5a3ad, data reload: false

query1	946	384	380	380
query2	6535	2064	2065	2064
query3	6702	214	222	214
query4	34156	23333	23407	23333
query5	4371	499	481	481
query6	256	169	171	169
query7	4620	294	308	294
query8	279	223	221	221
query9	9889	2672	2678	2672
query10	475	286	276	276
query11	18166	15289	15237	15237
query12	153	101	102	101
query13	1642	420	407	407
query14	10922	6917	7652	6917
query15	309	178	175	175
query16	7337	458	474	458
query17	1736	555	556	555
query18	1984	295	288	288
query19	360	146	148	146
query20	116	109	109	109
query21	214	101	103	101
query22	4436	4360	4419	4360
query23	34404	33941	34086	33941
query24	11157	2964	2884	2884
query25	607	380	389	380
query26	1172	156	154	154
query27	2425	289	279	279
query28	7367	2453	2437	2437
query29	776	415	409	409
query30	322	162	154	154
query31	990	759	818	759
query32	97	53	57	53
query33	757	288	294	288
query34	958	489	488	488
query35	854	724	744	724
query36	1093	945	935	935
query37	150	89	87	87
query38	4095	3993	3864	3864
query39	1436	1443	1412	1412
query40	208	96	95	95
query41	50	45	47	45
query42	119	95	103	95
query43	537	478	474	474
query44	1284	820	799	799
query45	191	164	162	162
query46	1151	745	753	745
query47	1886	1776	1803	1776
query48	442	355	356	355
query49	1118	398	384	384
query50	832	398	401	398
query51	6992	6888	6909	6888
query52	101	88	90	88
query53	254	182	180	180
query54	1218	456	462	456
query55	79	77	84	77
query56	276	258	278	258
query57	1170	1092	1058	1058
query58	229	263	260	260
query59	3061	2821	2987	2821
query60	294	256	263	256
query61	103	100	103	100
query62	835	652	657	652
query63	219	193	186	186
query64	4123	645	635	635
query65	3257	3183	3181	3181
query66	1062	304	319	304
query67	16053	15622	15499	15499
query68	3074	577	577	577
query69	447	302	297	297
query70	1148	1126	1126	1126
query71	344	272	265	265
query72	6076	4016	4044	4016
query73	745	332	337	332
query74	9527	8903	8963	8903
query75	3358	2677	2623	2623
query76	1978	1001	967	967
query77	407	285	291	285
query78	9938	9294	10088	9294
query79	944	888	873	873
query80	616	583	587	583
query81	468	254	250	250
query82	253	234	231	231
query83	161	159	160	159
query84	241	110	101	101
query85	729	368	358	358
query86	319	330	315	315
query87	4337	4402	4514	4402
query88	5066	4068	4038	4038
query89	385	357	355	355
query90	1771	315	309	309
query91	164	188	166	166
query92	76	75	72	72
query93	894	900	886	886
query94	634	378	356	356
query95	438	416	417	416
query96	497	488	485	485
query97	3116	3112	3138	3112
query98	225	234	230	230
query99	1413	1304	1326	1304
Total cold run time: 291197 ms
Total hot run time: 194119 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 297e3787077d9784d3cb85ef6ce7d5b3cba5a3ad, data reload: false

query1	0.05	0.04	0.05
query2	0.06	0.02	0.03
query3	0.23	0.06	0.06
query4	1.67	0.10	0.10
query5	0.51	0.51	0.49
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.49
query10	0.55	0.58	0.55
query11	0.13	0.10	0.10
query12	0.14	0.10	0.11
query13	0.61	0.59	0.59
query14	3.17	2.98	3.02
query15	0.89	0.81	0.81
query16	0.39	0.38	0.38
query17	0.98	1.02	1.02
query18	0.22	0.20	0.21
query19	1.86	1.90	1.98
query20	0.02	0.01	0.01
query21	15.35	0.60	0.59
query22	2.91	1.87	1.63
query23	17.46	0.81	0.68
query24	2.59	1.60	0.39
query25	0.28	0.09	0.11
query26	0.44	0.14	0.13
query27	0.04	0.04	0.04
query28	11.00	1.08	1.07
query29	12.60	3.28	3.27
query30	0.25	0.06	0.06
query31	2.89	0.39	0.37
query32	3.27	0.46	0.46
query33	2.98	2.97	3.03
query34	16.70	4.37	4.43
query35	4.47	4.45	4.40
query36	0.68	0.48	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.59 s
Total hot run time: 31.91 s

@seawinde seawinde force-pushed the fix_create_mtmv_name_duplicate branch from 297e378 to 30db4dd Compare September 19, 2024 02:55
@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41480 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30db4dd380d7719a80b334d425bfe49dad7bac1b, data reload: false

------ Round 1 ----------------------------------
q1	18187	7455	7366	7366
q2	2384	159	177	159
q3	10556	1136	1149	1136
q4	10218	795	752	752
q5	7743	3070	3035	3035
q6	234	147	149	147
q7	1000	610	600	600
q8	9411	2012	2029	2012
q9	6816	6378	6381	6378
q10	6979	2289	2258	2258
q11	432	253	259	253
q12	402	215	210	210
q13	17784	2975	3027	2975
q14	235	220	232	220
q15	584	516	529	516
q16	665	619	626	619
q17	985	851	800	800
q18	7280	6656	6829	6656
q19	1389	972	1040	972
q20	579	291	283	283
q21	3866	3251	3135	3135
q22	1086	1006	998	998
Total cold run time: 108815 ms
Total hot run time: 41480 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7219	7229	7178	7178
q2	327	227	229	227
q3	2862	2744	2764	2744
q4	1910	1682	1677	1677
q5	5365	5380	5370	5370
q6	228	150	145	145
q7	2111	1706	1717	1706
q8	3148	3287	3299	3287
q9	8365	8358	8311	8311
q10	3376	3308	3311	3308
q11	567	472	468	468
q12	809	574	561	561
q13	6244	2988	2992	2988
q14	288	253	251	251
q15	555	518	526	518
q16	725	682	656	656
q17	1773	1520	1494	1494
q18	7724	7380	7339	7339
q19	1659	1622	1568	1568
q20	2019	1803	1809	1803
q21	5402	5061	5101	5061
q22	1118	1000	1021	1000
Total cold run time: 63794 ms
Total hot run time: 57660 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193722 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30db4dd380d7719a80b334d425bfe49dad7bac1b, data reload: false

query1	956	368	366	366
query2	6574	2089	2096	2089
query3	6708	209	225	209
query4	34372	23404	23405	23404
query5	4301	456	452	452
query6	266	169	165	165
query7	4628	289	305	289
query8	292	222	221	221
query9	9667	2662	2654	2654
query10	456	273	279	273
query11	18175	15080	15146	15080
query12	157	105	102	102
query13	1631	430	395	395
query14	10721	7482	7521	7482
query15	264	171	191	171
query16	8089	456	485	456
query17	1680	577	560	560
query18	2150	302	297	297
query19	352	141	146	141
query20	119	109	118	109
query21	207	103	102	102
query22	4626	4242	4271	4242
query23	34675	33788	34079	33788
query24	11112	3023	2893	2893
query25	649	406	407	406
query26	1210	161	159	159
query27	2345	279	280	279
query28	8134	2454	2426	2426
query29	866	418	424	418
query30	334	158	165	158
query31	990	762	831	762
query32	104	56	59	56
query33	787	301	303	301
query34	936	490	498	490
query35	924	743	719	719
query36	1067	949	948	948
query37	158	91	84	84
query38	4070	3970	3796	3796
query39	1455	1384	1381	1381
query40	215	99	98	98
query41	53	48	51	48
query42	120	95	97	95
query43	532	495	513	495
query44	1268	814	798	798
query45	198	166	169	166
query46	1131	745	768	745
query47	1875	1810	1812	1810
query48	439	355	363	355
query49	1157	416	401	401
query50	810	401	395	395
query51	6991	7017	6839	6839
query52	102	89	91	89
query53	254	186	182	182
query54	1266	469	462	462
query55	78	75	76	75
query56	286	265	262	262
query57	1215	1106	1078	1078
query58	255	260	256	256
query59	3177	3064	2857	2857
query60	330	269	264	264
query61	109	106	104	104
query62	855	663	667	663
query63	216	188	185	185
query64	4160	642	621	621
query65	3266	3158	3193	3158
query66	860	307	298	298
query67	15852	15466	15448	15448
query68	3103	563	557	557
query69	427	288	282	282
query70	1151	1147	1119	1119
query71	325	265	262	262
query72	6010	3772	3927	3772
query73	736	320	326	320
query74	9560	8892	9003	8892
query75	3364	2668	2646	2646
query76	1972	826	821	821
query77	413	285	292	285
query78	9970	9928	9305	9305
query79	1142	873	862	862
query80	943	596	572	572
query81	545	256	251	251
query82	809	233	230	230
query83	214	165	158	158
query84	243	110	109	109
query85	747	369	354	354
query86	378	332	323	323
query87	4377	4355	4363	4355
query88	4739	4054	4052	4052
query89	377	364	357	357
query90	1803	316	308	308
query91	177	166	170	166
query92	79	73	72	72
query93	929	909	900	900
query94	806	351	384	351
query95	453	415	408	408
query96	497	486	492	486
query97	3194	3081	3094	3081
query98	227	223	218	218
query99	1432	1300	1278	1278
Total cold run time: 294419 ms
Total hot run time: 193722 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 30db4dd380d7719a80b334d425bfe49dad7bac1b, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.03	0.03
query3	0.22	0.07	0.07
query4	1.64	0.10	0.09
query5	0.50	0.50	0.50
query6	1.13	0.74	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.02
query9	0.55	0.49	0.49
query10	0.55	0.56	0.53
query11	0.14	0.11	0.10
query12	0.14	0.10	0.10
query13	0.60	0.60	0.59
query14	2.99	2.98	3.00
query15	0.90	0.81	0.81
query16	0.39	0.38	0.39
query17	1.04	1.05	1.05
query18	0.21	0.21	0.21
query19	1.98	1.89	1.98
query20	0.01	0.00	0.01
query21	15.36	0.58	0.57
query22	2.58	2.40	1.56
query23	17.43	0.84	0.86
query24	3.54	1.88	0.17
query25	0.24	0.14	0.11
query26	0.45	0.14	0.13
query27	0.05	0.04	0.04
query28	10.46	1.10	1.07
query29	12.54	3.23	3.22
query30	0.26	0.06	0.06
query31	2.88	0.39	0.37
query32	3.28	0.47	0.46
query33	3.01	3.00	3.02
query34	16.73	4.36	4.36
query35	4.43	4.38	4.43
query36	0.68	0.48	0.50
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.54 s
Total hot run time: 31.74 s

@seawinde
Copy link
Contributor Author

run buildall

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 19, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 41620 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5559bd311ea41a26c880b7ee739439f938668d10, data reload: false

------ Round 1 ----------------------------------
q1	17908	7485	7315	7315
q2	2078	175	163	163
q3	10666	1116	1182	1116
q4	10333	728	720	720
q5	7768	3152	3110	3110
q6	235	149	148	148
q7	1003	612	604	604
q8	9420	2071	2021	2021
q9	6846	6399	6414	6399
q10	7027	2284	2332	2284
q11	428	244	241	241
q12	416	217	212	212
q13	17784	2990	3001	2990
q14	238	213	223	213
q15	590	536	538	536
q16	679	630	640	630
q17	995	815	793	793
q18	7457	6717	6775	6717
q19	1408	1106	1007	1007
q20	608	283	281	281
q21	4365	3116	3321	3116
q22	1129	1038	1004	1004
Total cold run time: 109381 ms
Total hot run time: 41620 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7273	7211	7210	7210
q2	326	237	228	228
q3	3038	2970	2979	2970
q4	2066	1898	1807	1807
q5	5663	5687	5659	5659
q6	238	147	145	145
q7	2207	1795	1795	1795
q8	3339	3435	3418	3418
q9	8792	8934	8840	8840
q10	3470	3453	3489	3453
q11	573	491	491	491
q12	830	612	639	612
q13	9362	3226	3196	3196
q14	300	280	277	277
q15	581	522	515	515
q16	716	665	687	665
q17	1856	1619	1565	1565
q18	8351	7667	7791	7667
q19	1725	1522	1689	1522
q20	2127	1914	1895	1895
q21	5471	5423	5495	5423
q22	1151	1088	1054	1054
Total cold run time: 69455 ms
Total hot run time: 60407 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196034 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5559bd311ea41a26c880b7ee739439f938668d10, data reload: false

query1	1280	895	868	868
query2	6312	2067	2159	2067
query3	10782	3930	3964	3930
query4	64651	27013	23594	23594
query5	5279	472	461	461
query6	406	165	161	161
query7	5478	303	322	303
query8	325	225	226	225
query9	8512	2664	2680	2664
query10	493	291	285	285
query11	17914	15295	15707	15295
query12	167	102	103	102
query13	1466	429	418	418
query14	10768	7447	7557	7447
query15	211	179	179	179
query16	6799	433	482	433
query17	1266	609	591	591
query18	1531	321	325	321
query19	222	171	170	170
query20	132	110	107	107
query21	210	110	110	110
query22	4878	4746	4981	4746
query23	34804	33821	34027	33821
query24	5987	2878	2840	2840
query25	506	413	394	394
query26	617	162	159	159
query27	1591	292	299	292
query28	4311	2462	2458	2458
query29	682	417	426	417
query30	239	154	153	153
query31	982	795	803	795
query32	76	53	54	53
query33	413	296	302	296
query34	917	494	484	484
query35	863	723	732	723
query36	1075	958	955	955
query37	145	92	88	88
query38	4069	3941	4016	3941
query39	1498	1463	1419	1419
query40	209	95	96	95
query41	50	47	49	47
query42	112	99	96	96
query43	515	475	485	475
query44	1159	818	797	797
query45	194	161	163	161
query46	1128	762	767	762
query47	1921	1811	1857	1811
query48	463	366	368	366
query49	702	410	411	410
query50	840	422	403	403
query51	7036	6933	6887	6887
query52	99	84	87	84
query53	245	178	177	177
query54	564	444	439	439
query55	76	78	75	75
query56	267	245	268	245
query57	1237	1085	1106	1085
query58	229	226	251	226
query59	3138	2953	3042	2953
query60	284	259	266	259
query61	105	107	105	105
query62	752	673	650	650
query63	214	182	179	179
query64	1270	649	631	631
query65	3259	3169	3170	3169
query66	648	288	312	288
query67	16066	15573	15593	15573
query68	4505	582	572	572
query69	466	296	299	296
query70	1166	1139	1089	1089
query71	367	270	264	264
query72	6377	3931	4143	3931
query73	775	324	334	324
query74	9532	8920	8973	8920
query75	3367	2647	2664	2647
query76	1747	987	872	872
query77	475	302	292	292
query78	10490	9305	9152	9152
query79	1488	545	542	542
query80	1297	444	430	430
query81	535	242	240	240
query82	1028	143	146	143
query83	160	134	139	134
query84	297	81	72	72
query85	976	294	272	272
query86	371	304	302	302
query87	4579	4312	4389	4312
query88	3065	2366	2362	2362
query89	406	285	288	285
query90	1962	195	189	189
query91	197	140	141	140
query92	60	46	52	46
query93	2314	554	586	554
query94	784	282	293	282
query95	342	257	253	253
query96	618	272	278	272
query97	3278	3138	3151	3138
query98	215	200	196	196
query99	1722	1307	1298	1298
Total cold run time: 315600 ms
Total hot run time: 196034 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5559bd311ea41a26c880b7ee739439f938668d10, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.53	0.51	0.52
query6	1.13	0.71	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.49	0.50
query10	0.54	0.57	0.56
query11	0.14	0.10	0.10
query12	0.14	0.10	0.10
query13	0.60	0.60	0.60
query14	3.11	3.00	3.05
query15	0.91	0.83	0.81
query16	0.39	0.38	0.38
query17	1.07	1.01	1.06
query18	0.18	0.19	0.19
query19	1.84	1.81	2.07
query20	0.01	0.01	0.02
query21	15.37	0.58	0.59
query22	2.36	2.47	1.63
query23	17.12	0.81	0.73
query24	3.12	0.98	1.76
query25	0.27	0.14	0.05
query26	0.41	0.13	0.14
query27	0.05	0.04	0.04
query28	10.31	1.10	1.07
query29	12.59	3.18	3.19
query30	0.24	0.06	0.05
query31	2.88	0.40	0.38
query32	3.26	0.46	0.46
query33	3.01	3.06	3.00
query34	17.31	4.34	4.36
query35	4.44	4.47	4.40
query36	0.69	0.48	0.49
query37	0.09	0.05	0.05
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 107.11 s
Total hot run time: 32.32 s

@starocean999 starocean999 merged commit 252aeeb into apache:master Sep 20, 2024
25 of 28 checks passed
dataroaring pushed a commit that referenced this pull request Sep 26, 2024
…zed view (#40658)

## Proposed changes
This is brought by #26146
If create materialized view as following, Should fail, because has the
duplicated column name `o_orderdatE` and `o_orderdate`. But now can
create materialized view successfully. the pr fix this.

```sql
        CREATE MATERIALIZED VIEW mv_1
        BUILD IMMEDIATE REFRESH AUTO ON MANUAL 
        partition by(o_orderdate) 
        DISTRIBUTED BY RANDOM BUCKETS 2 
        PROPERTIES ('replication_num' = '1') 
        AS  
        select o_orderdatE, o_shippriority, o_comment, o_orderdate, 
        sum(o_totalprice) as sum_total, 
        max(o_totalpricE) as max_total, 
        min(o_totalprice) as min_total, 
        count(*) as count_all, 
        bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) cnt_1, 
        bitmap_union(to_bitmap(case when o_shippriority > 2 and o_orderkey IN (2) then o_custkey else null end)) as cnt_2 
        from (select * from orders) as t1
        group by 
        o_orderdatE, 
        o_shippriority, 
        o_comment,
        o_orderdate;
```
seawinde added a commit to seawinde/doris that referenced this pull request Oct 14, 2024
…zed view (apache#40658)

This is brought by apache#26146
If create materialized view as following, Should fail, because has the
duplicated column name `o_orderdatE` and `o_orderdate`. But now can
create materialized view successfully. the pr fix this.

```sql
        CREATE MATERIALIZED VIEW mv_1
        BUILD IMMEDIATE REFRESH AUTO ON MANUAL
        partition by(o_orderdate)
        DISTRIBUTED BY RANDOM BUCKETS 2
        PROPERTIES ('replication_num' = '1')
        AS
        select o_orderdatE, o_shippriority, o_comment, o_orderdate,
        sum(o_totalprice) as sum_total,
        max(o_totalpricE) as max_total,
        min(o_totalprice) as min_total,
        count(*) as count_all,
        bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) cnt_1,
        bitmap_union(to_bitmap(case when o_shippriority > 2 and o_orderkey IN (2) then o_custkey else null end)) as cnt_2
        from (select * from orders) as t1
        group by
        o_orderdatE,
        o_shippriority,
        o_comment,
        o_orderdate;
```
yiguolei pushed a commit that referenced this pull request Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.2-merged p0_b reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants