Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](nereids)use mtmv to match legacy mv #33699

Merged
merged 14 commits into from
Jul 1, 2024
Merged

[feature](nereids)use mtmv to match legacy mv #33699

merged 14 commits into from
Jul 1, 2024

Conversation

starocean999
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41237 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a5e5ef0134d009abfd685642f43886b023cb338, data reload: false

------ Round 1 ----------------------------------
q1	17621	4336	4260	4260
q2	2034	189	187	187
q3	10453	1225	1131	1131
q4	10189	898	834	834
q5	7489	2703	2934	2703
q6	228	134	133	133
q7	982	622	623	622
q8	9218	2107	2091	2091
q9	9279	6729	6703	6703
q10	9353	3948	4093	3948
q11	420	229	233	229
q12	464	236	248	236
q13	17393	3164	3330	3164
q14	276	229	226	226
q15	508	474	475	474
q16	479	385	401	385
q17	994	630	695	630
q18	8405	7822	7808	7808
q19	2896	1407	1475	1407
q20	627	324	316	316
q21	5139	3410	3956	3410
q22	395	340	355	340
Total cold run time: 114842 ms
Total hot run time: 41237 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4685	4489	4404	4404
q2	403	281	266	266
q3	3222	3008	2938	2938
q4	1925	1669	1659	1659
q5	5499	5517	5499	5499
q6	221	126	125	125
q7	2168	1788	1792	1788
q8	3261	3424	3384	3384
q9	8647	8570	8687	8570
q10	4113	3882	3736	3736
q11	558	472	469	469
q12	797	635	628	628
q13	17133	3121	3112	3112
q14	314	250	277	250
q15	502	480	507	480
q16	473	431	439	431
q17	1800	1505	1480	1480
q18	8183	7675	7296	7296
q19	2641	1654	1610	1610
q20	3033	1778	1770	1770
q21	4844	4853	4675	4675
q22	736	515	553	515
Total cold run time: 75158 ms
Total hot run time: 55085 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172130 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a5e5ef0134d009abfd685642f43886b023cb338, data reload: false

query1	931	377	381	377
query2	6454	2655	2471	2471
query3	6647	217	214	214
query4	19658	17508	17168	17168
query5	4163	450	442	442
query6	239	157	150	150
query7	4590	297	296	296
query8	334	285	284	284
query9	8347	2443	2447	2443
query10	436	310	282	282
query11	10636	10042	10087	10042
query12	134	88	88	88
query13	1653	363	375	363
query14	10092	7419	6897	6897
query15	232	191	183	183
query16	7301	285	275	275
query17	1276	532	532	532
query18	1916	267	273	267
query19	200	152	154	152
query20	90	85	86	85
query21	210	143	127	127
query22	4362	4141	4251	4141
query23	33726	33145	33025	33025
query24	11196	2835	2856	2835
query25	648	352	385	352
query26	1419	167	157	157
query27	3010	323	321	321
query28	7350	2084	2093	2084
query29	979	626	612	612
query30	264	170	151	151
query31	947	730	718	718
query32	90	54	52	52
query33	753	280	276	276
query34	974	485	478	478
query35	729	601	609	601
query36	1120	929	935	929
query37	146	66	71	66
query38	2845	2698	2729	2698
query39	849	805	781	781
query40	208	124	120	120
query41	60	57	56	56
query42	120	99	98	98
query43	593	560	566	560
query44	1210	730	736	730
query45	198	162	174	162
query46	1077	744	722	722
query47	1887	1786	1793	1786
query48	380	311	305	305
query49	1002	417	422	417
query50	792	398	399	398
query51	6899	6840	6681	6681
query52	115	97	95	95
query53	360	295	291	291
query54	866	464	451	451
query55	79	78	76	76
query56	292	276	280	276
query57	1145	1053	1044	1044
query58	279	295	238	238
query59	3535	3209	3331	3209
query60	292	281	272	272
query61	88	87	89	87
query62	634	444	437	437
query63	311	286	287	286
query64	9822	2195	1777	1777
query65	3192	3097	3119	3097
query66	1374	324	325	324
query67	15394	14884	14930	14884
query68	4480	533	539	533
query69	476	356	374	356
query70	1184	1146	1097	1097
query71	390	286	280	280
query72	7497	5265	4521	4521
query73	746	331	332	331
query74	6006	5556	5479	5479
query75	3414	2597	2678	2597
query76	2387	854	899	854
query77	440	296	295	295
query78	10244	10110	9804	9804
query79	1752	519	525	519
query80	2062	471	459	459
query81	570	226	218	218
query82	1006	104	100	100
query83	309	172	171	171
query84	262	86	91	86
query85	1158	275	330	275
query86	375	281	302	281
query87	3274	3104	3074	3074
query88	3639	2478	2454	2454
query89	498	401	389	389
query90	1771	187	188	187
query91	130	97	99	97
query92	65	49	50	49
query93	1350	514	499	499
query94	1194	191	189	189
query95	414	316	307	307
query96	579	276	266	266
query97	3205	3029	3043	3029
query98	245	221	217	217
query99	1093	823	837	823
Total cold run time: 271365 ms
Total hot run time: 172130 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a5e5ef0134d009abfd685642f43886b023cb338, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.68	0.08	0.07
query5	0.49	0.48	0.49
query6	1.13	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.48	0.50
query10	0.55	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	0.80	0.78	0.78
query15	0.83	0.81	0.82
query16	0.37	0.38	0.37
query17	1.05	1.00	1.05
query18	0.22	0.25	0.23
query19	1.80	1.77	1.79
query20	0.02	0.01	0.01
query21	15.53	0.68	0.67
query22	4.21	6.10	2.21
query23	18.33	1.36	1.28
query24	1.96	0.23	0.21
query25	0.14	0.08	0.08
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.27	1.01	1.00
query29	13.58	3.38	3.27
query30	0.24	0.06	0.05
query31	2.89	0.40	0.37
query32	3.25	0.48	0.46
query33	2.85	2.95	2.86
query34	16.99	4.46	4.45
query35	4.46	4.48	4.54
query36	0.67	0.45	0.46
query37	0.17	0.16	0.15
query38	0.15	0.16	0.15
query39	0.04	0.04	0.03
query40	0.16	0.17	0.16
query41	0.08	0.04	0.04
query42	0.06	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 110.22 s
Total hot run time: 30.91 s

@starocean999
Copy link
Contributor Author

run buildall

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40948 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a1cc70e2918ba3163ad0b4bff32197d05c61f5ff, data reload: false

------ Round 1 ----------------------------------
q1	17597	4412	4332	4332
q2	2024	189	194	189
q3	10432	1316	1154	1154
q4	10190	834	814	814
q5	7501	2699	2747	2699
q6	218	133	141	133
q7	968	618	618	618
q8	9231	2116	2074	2074
q9	9119	6681	6702	6681
q10	9116	3894	3892	3892
q11	474	243	242	242
q12	506	230	239	230
q13	18424	3218	3129	3129
q14	282	235	234	234
q15	521	486	483	483
q16	516	414	409	409
q17	993	707	593	593
q18	8334	7794	7764	7764
q19	7250	1344	1319	1319
q20	646	321	329	321
q21	5173	3311	3935	3311
q22	387	327	335	327
Total cold run time: 119902 ms
Total hot run time: 40948 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4566	4469	4461	4461
q2	374	269	280	269
q3	3102	2900	2900	2900
q4	2020	1738	1650	1650
q5	5320	5480	5438	5438
q6	226	126	127	126
q7	2183	1813	1847	1813
q8	3212	3370	3318	3318
q9	8636	8674	8589	8589
q10	4071	3852	3798	3798
q11	586	499	480	480
q12	784	624	613	613
q13	17019	3183	3120	3120
q14	297	265	260	260
q15	519	475	473	473
q16	494	431	430	430
q17	1795	1482	1507	1482
q18	8106	7603	7376	7376
q19	1733	1636	1619	1619
q20	2530	1770	1789	1770
q21	8095	4642	4651	4642
q22	615	562	526	526
Total cold run time: 76283 ms
Total hot run time: 55153 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172845 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a1cc70e2918ba3163ad0b4bff32197d05c61f5ff, data reload: false

query1	909	375	363	363
query2	6447	2422	2184	2184
query3	6646	207	211	207
query4	19473	17303	17289	17289
query5	4207	451	454	451
query6	238	154	154	154
query7	4590	296	297	296
query8	333	267	281	267
query9	8669	2344	2370	2344
query10	444	301	283	283
query11	10661	9995	9959	9959
query12	139	83	90	83
query13	1668	364	358	358
query14	9532	7466	7620	7466
query15	230	185	183	183
query16	7801	267	265	265
query17	1389	546	530	530
query18	1956	266	267	266
query19	203	153	152	152
query20	87	85	78	78
query21	207	135	124	124
query22	4623	4219	4225	4219
query23	33600	32984	33085	32984
query24	11001	2813	2829	2813
query25	619	353	367	353
query26	1412	151	149	149
query27	3013	322	317	317
query28	7390	2018	2018	2018
query29	923	616	603	603
query30	243	151	151	151
query31	957	751	759	751
query32	87	52	53	52
query33	753	285	288	285
query34	942	470	481	470
query35	773	607	602	602
query36	1042	945	888	888
query37	151	68	69	68
query38	2852	2741	2722	2722
query39	871	788	792	788
query40	211	124	119	119
query41	56	50	51	50
query42	118	96	95	95
query43	573	564	541	541
query44	1215	726	749	726
query45	190	160	161	160
query46	1089	701	696	696
query47	1856	1771	1767	1767
query48	381	290	304	290
query49	909	393	404	393
query50	781	392	383	383
query51	6798	6652	6674	6652
query52	110	88	91	88
query53	360	288	288	288
query54	846	447	443	443
query55	79	74	76	74
query56	313	249	257	249
query57	1136	1034	1078	1034
query58	252	245	245	245
query59	3337	3082	3167	3082
query60	287	265	276	265
query61	91	91	90	90
query62	630	446	443	443
query63	314	290	283	283
query64	8973	2269	1755	1755
query65	3172	3076	3138	3076
query66	841	325	327	325
query67	15140	14969	14983	14969
query68	4581	553	554	553
query69	505	340	310	310
query70	1174	1138	1112	1112
query71	378	277	260	260
query72	7253	5628	5627	5627
query73	753	331	324	324
query74	5993	5545	5515	5515
query75	3452	2597	2702	2597
query76	2589	897	922	897
query77	617	293	294	293
query78	10247	9772	9653	9653
query79	1671	515	516	515
query80	854	456	461	456
query81	590	215	214	214
query82	1017	100	99	99
query83	236	171	168	168
query84	253	84	84	84
query85	1063	281	265	265
query86	407	300	326	300
query87	3282	3100	3064	3064
query88	3399	2445	2446	2445
query89	469	395	376	376
query90	1850	195	195	195
query91	139	111	110	110
query92	62	53	54	53
query93	1489	511	568	511
query94	1212	181	183	181
query95	408	307	314	307
query96	604	272	275	272
query97	3235	3010	2960	2960
query98	237	200	190	190
query99	1248	823	850	823
Total cold run time: 268406 ms
Total hot run time: 172845 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a1cc70e2918ba3163ad0b4bff32197d05c61f5ff, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.09	0.10
query5	0.50	0.49	0.50
query6	1.12	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.49
query10	0.56	0.53	0.55
query11	0.15	0.10	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	0.78	0.80	0.78
query15	0.83	0.80	0.82
query16	0.36	0.36	0.37
query17	1.02	1.00	1.02
query18	0.23	0.21	0.26
query19	1.82	1.73	1.83
query20	0.02	0.01	0.01
query21	15.55	0.66	0.65
query22	4.34	6.97	1.91
query23	18.20	1.36	1.30
query24	2.05	0.22	0.21
query25	0.16	0.07	0.08
query26	0.27	0.17	0.16
query27	0.07	0.08	0.08
query28	13.25	1.02	0.99
query29	13.80	3.33	3.26
query30	0.24	0.06	0.05
query31	2.88	0.40	0.39
query32	3.26	0.46	0.46
query33	2.93	2.90	2.86
query34	17.08	4.41	4.40
query35	4.48	4.47	4.60
query36	0.65	0.50	0.46
query37	0.17	0.15	0.14
query38	0.15	0.13	0.13
query39	0.04	0.04	0.03
query40	0.17	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.03	0.04	0.03
Total cold run time: 110.69 s
Total hot run time: 30.47 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40058 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e3027a273af2766bba71db319c8e27f0ee16e177, data reload: false

------ Round 1 ----------------------------------
q1	17986	5068	4392	4392
q2	2589	197	201	197
q3	11650	1154	1194	1154
q4	10326	853	817	817
q5	7628	2684	2656	2656
q6	229	138	136	136
q7	955	606	613	606
q8	9288	2090	2035	2035
q9	8938	6468	6460	6460
q10	8864	3723	3730	3723
q11	464	244	233	233
q12	442	229	224	224
q13	18845	2980	2992	2980
q14	262	225	220	220
q15	517	464	492	464
q16	509	375	376	375
q17	953	653	676	653
q18	8074	7470	7348	7348
q19	1661	1575	1529	1529
q20	650	323	313	313
q21	4864	3209	3244	3209
q22	390	341	334	334
Total cold run time: 116084 ms
Total hot run time: 40058 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4312	4224	4219	4219
q2	360	264	272	264
q3	2982	2742	2754	2742
q4	1897	1583	1619	1583
q5	5245	5242	5234	5234
q6	210	124	125	124
q7	2133	1705	1758	1705
q8	3172	3328	3294	3294
q9	8343	8314	8369	8314
q10	3883	3545	3638	3545
q11	605	479	472	472
q12	788	590	605	590
q13	17367	2973	2998	2973
q14	289	262	267	262
q15	511	470	470	470
q16	475	419	419	419
q17	1793	1504	1471	1471
q18	7623	7453	7258	7258
q19	1653	1525	1466	1466
q20	1970	1775	1776	1775
q21	4973	4627	4737	4627
q22	625	529	524	524
Total cold run time: 71209 ms
Total hot run time: 53331 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169660 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e3027a273af2766bba71db319c8e27f0ee16e177, data reload: false

query1	929	381	376	376
query2	6449	2531	2412	2412
query3	6689	207	208	207
query4	19090	17333	17048	17048
query5	4171	453	460	453
query6	254	159	155	155
query7	4592	308	294	294
query8	330	298	295	295
query9	8557	2446	2434	2434
query10	451	294	276	276
query11	10524	10073	10129	10073
query12	134	86	89	86
query13	1641	362	371	362
query14	10051	6889	7710	6889
query15	263	192	184	184
query16	8175	282	282	282
query17	1815	533	522	522
query18	2096	300	296	296
query19	203	146	152	146
query20	92	80	82	80
query21	201	130	122	122
query22	4348	4118	3997	3997
query23	33639	33038	33143	33038
query24	11946	2800	2821	2800
query25	652	356	358	356
query26	1738	158	159	158
query27	2969	313	325	313
query28	7405	2057	2051	2051
query29	1025	604	598	598
query30	272	146	147	146
query31	959	740	735	735
query32	100	53	56	53
query33	790	293	281	281
query34	955	474	479	474
query35	752	642	602	602
query36	1113	938	935	935
query37	204	71	72	71
query38	2899	2781	2735	2735
query39	846	788	785	785
query40	285	123	121	121
query41	54	55	52	52
query42	122	95	100	95
query43	591	561	550	550
query44	1291	731	747	731
query45	198	168	167	167
query46	1071	732	743	732
query47	1858	1824	1805	1805
query48	369	297	303	297
query49	1179	407	416	407
query50	784	391	400	391
query51	6755	6648	6591	6591
query52	110	92	97	92
query53	352	293	280	280
query54	989	453	447	447
query55	80	73	75	73
query56	287	263	259	259
query57	1177	1056	1083	1056
query58	240	242	239	239
query59	3475	3246	3155	3155
query60	287	269	268	268
query61	93	91	90	90
query62	665	454	427	427
query63	319	291	292	291
query64	9899	2286	1768	1768
query65	3162	3054	3089	3054
query66	1373	333	332	332
query67	15491	15008	14780	14780
query68	4561	538	540	538
query69	452	303	323	303
query70	1193	1143	1148	1143
query71	413	279	275	275
query72	7291	5310	2748	2748
query73	737	329	328	328
query74	5892	5490	5435	5435
query75	3515	2699	2620	2620
query76	2886	945	897	897
query77	456	305	358	305
query78	10263	9887	9668	9668
query79	2150	517	511	511
query80	1113	471	478	471
query81	585	236	219	219
query82	755	101	104	101
query83	234	168	170	168
query84	243	85	90	85
query85	2022	280	285	280
query86	490	328	306	306
query87	3281	3111	3094	3094
query88	4295	2470	2438	2438
query89	488	383	383	383
query90	1820	195	194	194
query91	128	99	101	99
query92	60	50	52	50
query93	2323	518	495	495
query94	1228	194	193	193
query95	419	387	331	331
query96	589	279	269	269
query97	3170	3034	3001	3001
query98	220	206	204	204
query99	1077	838	839	838
Total cold run time: 276224 ms
Total hot run time: 169660 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e3027a273af2766bba71db319c8e27f0ee16e177, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.05	0.04
query3	0.27	0.05	0.05
query4	2.13	0.08	0.06
query5	0.50	0.50	0.51
query6	1.40	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.54	0.54	0.54
query11	0.16	0.12	0.11
query12	0.16	0.13	0.13
query13	0.60	0.61	0.60
query14	0.80	0.79	0.78
query15	0.84	0.80	0.82
query16	0.36	0.37	0.35
query17	0.99	0.95	1.01
query18	0.22	0.24	0.24
query19	1.82	1.70	1.67
query20	0.01	0.01	0.02
query21	15.47	0.68	0.65
query22	4.36	7.41	1.84
query23	18.26	1.37	1.28
query24	2.11	0.23	0.22
query25	0.16	0.10	0.09
query26	0.27	0.17	0.17
query27	0.08	0.08	0.08
query28	13.22	1.03	1.00
query29	12.68	3.36	3.30
query30	0.25	0.06	0.06
query31	2.86	0.40	0.39
query32	3.25	0.48	0.49
query33	2.90	2.93	2.94
query34	17.13	4.38	4.39
query35	4.46	4.48	4.51
query36	0.66	0.49	0.46
query37	0.19	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.05	0.04
query40	0.17	0.14	0.14
query41	0.09	0.04	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.39 s
Total hot run time: 30.54 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39664 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 73d9d07420aab01f55b8e2f45bae5ec489185e2b, data reload: false

------ Round 1 ----------------------------------
q1	17620	4319	4227	4227
q2	2013	186	195	186
q3	10470	1135	1085	1085
q4	10207	704	734	704
q5	7501	2631	2628	2628
q6	215	132	134	132
q7	941	617	589	589
q8	9221	2046	2078	2046
q9	8934	6463	6459	6459
q10	8843	3722	3728	3722
q11	461	247	230	230
q12	536	229	219	219
q13	17762	2958	2957	2957
q14	261	228	221	221
q15	520	467	468	467
q16	516	380	374	374
q17	959	718	692	692
q18	7956	7381	7392	7381
q19	4409	1449	1478	1449
q20	656	326	332	326
q21	5177	3240	3815	3240
q22	375	330	338	330
Total cold run time: 115553 ms
Total hot run time: 39664 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4345	4244	4216	4216
q2	383	269	259	259
q3	2982	2711	2935	2711
q4	2009	1725	1729	1725
q5	5565	5503	5521	5503
q6	223	130	126	126
q7	2159	1824	1879	1824
q8	3250	3416	3428	3416
q9	8733	8611	8752	8611
q10	4058	3986	3741	3741
q11	581	474	487	474
q12	782	635	637	635
q13	15928	3195	3145	3145
q14	314	278	284	278
q15	517	479	483	479
q16	497	425	437	425
q17	1804	1477	1496	1477
q18	7938	7962	7683	7683
q19	2239	1628	1594	1594
q20	3057	1877	1890	1877
q21	5109	5044	4913	4913
q22	594	522	540	522
Total cold run time: 73067 ms
Total hot run time: 55634 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173003 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 73d9d07420aab01f55b8e2f45bae5ec489185e2b, data reload: false

query1	927	370	361	361
query2	6460	2331	2302	2302
query3	6638	205	202	202
query4	19158	17474	17371	17371
query5	3587	470	462	462
query6	244	155	160	155
query7	4583	302	294	294
query8	325	283	280	280
query9	8580	2381	2418	2381
query10	572	299	289	289
query11	10385	10081	9905	9905
query12	124	85	83	83
query13	1633	361	365	361
query14	9413	6902	7578	6902
query15	234	188	187	187
query16	7741	260	257	257
query17	1900	525	505	505
query18	1898	272	281	272
query19	201	155	160	155
query20	89	85	83	83
query21	215	133	132	132
query22	4440	4219	3901	3901
query23	33702	33666	33796	33666
query24	10998	2951	2835	2835
query25	622	379	363	363
query26	1011	153	154	153
query27	2413	323	322	322
query28	6621	2103	2083	2083
query29	870	653	628	628
query30	240	156	153	153
query31	988	777	777	777
query32	96	56	58	56
query33	777	340	298	298
query34	969	506	468	468
query35	751	630	638	630
query36	1135	961	982	961
query37	153	72	74	72
query38	2942	2845	2822	2822
query39	889	844	820	820
query40	220	127	123	123
query41	54	53	49	49
query42	111	109	102	102
query43	581	566	547	547
query44	1202	724	729	724
query45	201	156	160	156
query46	1086	704	721	704
query47	1844	1782	1803	1782
query48	387	294	310	294
query49	859	397	400	397
query50	759	388	390	388
query51	6752	6738	6650	6650
query52	102	87	93	87
query53	362	294	282	282
query54	882	435	443	435
query55	78	72	75	72
query56	274	259	263	259
query57	1137	1030	1070	1030
query58	258	263	243	243
query59	3308	3278	3052	3052
query60	295	274	272	272
query61	92	90	93	90
query62	596	440	436	436
query63	314	291	295	291
query64	8873	2196	1741	1741
query65	3184	3088	3099	3088
query66	745	327	349	327
query67	15362	15005	14966	14966
query68	5282	537	535	535
query69	612	399	372	372
query70	1146	1148	1093	1093
query71	475	272	279	272
query72	7120	5372	5413	5372
query73	784	315	325	315
query74	5791	5458	5543	5458
query75	3654	2673	2721	2673
query76	3123	950	839	839
query77	607	302	296	296
query78	10406	9796	9735	9735
query79	2751	512	496	496
query80	1420	477	472	472
query81	538	216	224	216
query82	852	103	100	100
query83	193	167	165	165
query84	269	86	93	86
query85	1299	273	277	273
query86	456	312	298	298
query87	3315	3120	3066	3066
query88	3974	2318	2299	2299
query89	472	384	378	378
query90	1733	185	195	185
query91	129	98	98	98
query92	64	48	48	48
query93	4132	516	502	502
query94	1033	194	184	184
query95	408	320	323	320
query96	594	264	261	261
query97	3141	3022	3021	3021
query98	225	201	202	201
query99	1315	840	858	840
Total cold run time: 271755 ms
Total hot run time: 173003 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 73d9d07420aab01f55b8e2f45bae5ec489185e2b, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.06	0.05
query4	1.69	0.07	0.08
query5	0.50	0.47	0.49
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.53	0.47	0.49
query10	0.54	0.53	0.56
query11	0.16	0.11	0.11
query12	0.14	0.13	0.11
query13	0.59	0.58	0.58
query14	0.80	0.79	0.79
query15	0.83	0.82	0.81
query16	0.36	0.37	0.37
query17	1.06	1.04	1.03
query18	0.21	0.27	0.21
query19	1.79	1.71	1.68
query20	0.02	0.01	0.00
query21	15.49	0.64	0.65
query22	3.96	5.98	2.59
query23	18.34	1.40	1.38
query24	2.07	0.22	0.25
query25	0.16	0.09	0.07
query26	0.28	0.18	0.18
query27	0.07	0.08	0.10
query28	13.19	1.01	0.99
query29	12.66	3.30	3.24
query30	0.26	0.07	0.06
query31	2.88	0.39	0.38
query32	3.27	0.47	0.47
query33	2.92	2.83	2.96
query34	17.09	4.68	4.64
query35	4.83	4.71	4.48
query36	0.65	0.48	0.46
query37	0.17	0.15	0.15
query38	0.16	0.14	0.16
query39	0.04	0.04	0.03
query40	0.17	0.16	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.62 s
Total hot run time: 31.4 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40218 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f7aa3e11d66501af261d0a6d945f3827f92f87e5, data reload: false

------ Round 1 ----------------------------------
q1	17620	4314	4235	4235
q2	2038	197	196	196
q3	10438	1171	1222	1171
q4	10179	836	675	675
q5	7465	2647	2705	2647
q6	219	140	132	132
q7	950	610	582	582
q8	9235	2063	2062	2062
q9	8841	6488	6422	6422
q10	8876	3741	3699	3699
q11	445	236	238	236
q12	460	231	229	229
q13	17779	2930	2979	2930
q14	266	213	219	213
q15	515	477	480	477
q16	530	372	380	372
q17	966	684	670	670
q18	8035	7484	7401	7401
q19	3739	1454	1488	1454
q20	636	326	315	315
q21	4935	3768	3777	3768
q22	386	332	337	332
Total cold run time: 114553 ms
Total hot run time: 40218 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4350	4206	4238	4206
q2	363	272	268	268
q3	2966	2775	2910	2775
q4	1939	1695	1738	1695
q5	5531	5539	5480	5480
q6	213	126	129	126
q7	2207	1811	1866	1811
q8	3289	3419	3403	3403
q9	8646	8612	8780	8612
q10	4168	3886	3733	3733
q11	554	502	482	482
q12	774	617	656	617
q13	17143	3177	3163	3163
q14	294	270	275	270
q15	522	506	482	482
q16	496	460	417	417
q17	1796	1513	1503	1503
q18	8010	7864	7727	7727
q19	3331	1743	1673	1673
q20	2171	1885	1896	1885
q21	5061	4921	4767	4767
q22	616	536	550	536
Total cold run time: 74440 ms
Total hot run time: 55631 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174348 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f7aa3e11d66501af261d0a6d945f3827f92f87e5, data reload: false

query1	920	388	375	375
query2	6447	2442	2413	2413
query3	6634	213	220	213
query4	18958	17501	17306	17306
query5	3601	474	463	463
query6	237	158	162	158
query7	4578	317	307	307
query8	312	312	276	276
query9	8401	2326	2334	2326
query10	558	313	278	278
query11	10523	9990	10002	9990
query12	123	84	84	84
query13	1627	374	367	367
query14	9614	7680	7050	7050
query15	227	189	180	180
query16	7677	273	265	265
query17	1792	545	527	527
query18	1941	275	272	272
query19	194	152	154	152
query20	93	86	87	86
query21	208	130	126	126
query22	4269	4169	4001	4001
query23	33701	33669	33491	33491
query24	11102	2888	2843	2843
query25	593	378	369	369
query26	708	158	164	158
query27	2394	328	347	328
query28	6295	2091	2076	2076
query29	890	614	618	614
query30	258	155	163	155
query31	1010	758	773	758
query32	94	52	60	52
query33	762	284	304	284
query34	1090	501	483	483
query35	775	637	645	637
query36	1127	961	969	961
query37	142	81	77	77
query38	3070	2846	2844	2844
query39	925	851	843	843
query40	222	146	135	135
query41	59	58	61	58
query42	111	109	106	106
query43	608	574	546	546
query44	1205	740	742	740
query45	206	162	167	162
query46	1082	705	711	705
query47	1848	1772	1775	1772
query48	372	306	313	306
query49	864	411	417	411
query50	765	392	392	392
query51	6812	6755	6704	6704
query52	108	88	97	88
query53	361	298	301	298
query54	939	453	457	453
query55	81	75	76	75
query56	295	272	274	272
query57	1128	1035	1028	1028
query58	265	253	260	253
query59	3422	3222	3249	3222
query60	305	291	292	291
query61	110	106	107	106
query62	625	435	441	435
query63	322	294	290	290
query64	8601	2314	1925	1925
query65	3194	3103	3111	3103
query66	827	325	325	325
query67	15684	15029	15027	15027
query68	8229	555	566	555
query69	677	492	424	424
query70	1245	1153	1148	1148
query71	501	277	275	275
query72	9234	5444	5813	5444
query73	1233	329	327	327
query74	5977	5494	5433	5433
query75	4858	2666	2670	2666
query76	4511	966	892	892
query77	761	295	307	295
query78	10479	10029	9800	9800
query79	4507	530	512	512
query80	1951	470	470	470
query81	546	221	232	221
query82	411	109	99	99
query83	298	179	175	175
query84	263	88	88	88
query85	873	318	267	267
query86	350	336	316	316
query87	3259	3132	3094	3094
query88	4168	2434	2441	2434
query89	493	387	391	387
query90	1840	188	188	188
query91	131	99	101	99
query92	60	47	48	47
query93	5129	516	506	506
query94	1207	194	186	186
query95	402	313	309	309
query96	582	272	267	267
query97	3196	3006	3002	3002
query98	215	206	192	192
query99	1179	814	844	814
Total cold run time: 283056 ms
Total hot run time: 174348 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f7aa3e11d66501af261d0a6d945f3827f92f87e5, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.05
query3	0.23	0.05	0.04
query4	1.71	0.08	0.09
query5	0.50	0.49	0.48
query6	1.15	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.49
query10	0.54	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.58
query14	0.77	0.82	0.77
query15	0.84	0.82	0.81
query16	0.39	0.37	0.37
query17	1.04	1.04	0.99
query18	0.21	0.24	0.23
query19	1.90	1.74	1.77
query20	0.02	0.00	0.01
query21	15.40	0.66	0.66
query22	4.64	7.40	1.72
query23	18.29	1.35	1.27
query24	2.04	0.22	0.21
query25	0.17	0.08	0.08
query26	0.25	0.18	0.17
query27	0.08	0.07	0.08
query28	13.27	1.01	0.98
query29	12.60	3.28	3.27
query30	0.27	0.06	0.07
query31	2.83	0.40	0.38
query32	3.28	0.47	0.46
query33	2.93	2.85	2.95
query34	17.00	4.45	4.37
query35	4.50	4.49	4.44
query36	0.65	0.48	0.46
query37	0.18	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.06
query43	0.03	0.03	0.04
Total cold run time: 109.84 s
Total hot run time: 30.23 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39684 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0d8c3d8ad08a02d65cc30b15efabd7fa23773449, data reload: false

------ Round 1 ----------------------------------
q1	17590	4801	4235	4235
q2	2033	209	187	187
q3	10448	1230	1069	1069
q4	10191	755	917	755
q5	7537	2699	2658	2658
q6	221	140	140	140
q7	959	610	613	610
q8	9308	2139	2085	2085
q9	8826	6515	6435	6435
q10	8966	3692	3711	3692
q11	454	235	239	235
q12	464	230	226	226
q13	17909	2979	2951	2951
q14	277	220	227	220
q15	514	494	477	477
q16	522	379	369	369
q17	968	647	769	647
q18	8031	7423	7521	7423
q19	6656	1466	1475	1466
q20	661	319	321	319
q21	4904	3145	3908	3145
q22	394	340	342	340
Total cold run time: 117833 ms
Total hot run time: 39684 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4336	4245	4207	4207
q2	386	268	273	268
q3	3016	2750	2774	2750
q4	1821	1579	1565	1565
q5	5250	5271	5314	5271
q6	220	135	133	133
q7	2119	1779	1788	1779
q8	3227	3355	3315	3315
q9	8362	8336	8388	8336
q10	3882	3638	3683	3638
q11	582	494	486	486
q12	774	612	585	585
q13	17591	2986	2950	2950
q14	298	268	278	268
q15	530	472	478	472
q16	468	420	417	417
q17	1797	1495	1456	1456
q18	7618	7487	7344	7344
q19	2804	1571	1559	1559
q20	1980	1771	1741	1741
q21	4933	4759	4600	4600
q22	605	542	537	537
Total cold run time: 72599 ms
Total hot run time: 53677 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172168 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0d8c3d8ad08a02d65cc30b15efabd7fa23773449, data reload: false

query1	915	382	374	374
query2	6455	2437	2347	2347
query3	6654	207	212	207
query4	19286	17340	17157	17157
query5	4228	464	471	464
query6	285	167	167	167
query7	4599	290	291	290
query8	306	273	281	273
query9	8433	2373	2375	2373
query10	610	306	286	286
query11	10621	10094	9970	9970
query12	137	90	84	84
query13	1664	368	376	368
query14	10104	7041	6252	6252
query15	237	186	187	186
query16	7845	275	265	265
query17	1694	574	543	543
query18	1950	289	275	275
query19	206	152	160	152
query20	91	82	87	82
query21	214	139	131	131
query22	4334	4044	3933	3933
query23	33793	33122	33092	33092
query24	12139	2896	2801	2801
query25	695	387	388	387
query26	1800	154	154	154
query27	3045	319	327	319
query28	7646	2067	2059	2059
query29	1157	624	627	624
query30	292	155	152	152
query31	967	742	770	742
query32	96	55	57	55
query33	790	307	304	304
query34	927	470	470	470
query35	747	632	614	614
query36	1068	941	907	907
query37	293	76	81	76
query38	2876	2785	2730	2730
query39	850	886	798	798
query40	280	126	125	125
query41	52	51	55	51
query42	114	96	98	96
query43	582	541	534	534
query44	1210	714	716	714
query45	190	162	167	162
query46	1095	718	712	712
query47	1859	1768	1782	1768
query48	368	294	298	294
query49	1200	417	403	403
query50	768	378	378	378
query51	6907	6727	6766	6727
query52	106	88	95	88
query53	356	292	293	292
query54	971	455	431	431
query55	76	75	74	74
query56	278	282	265	265
query57	1188	1055	1028	1028
query58	259	250	260	250
query59	3307	3208	3089	3089
query60	324	300	273	273
query61	91	93	88	88
query62	644	459	446	446
query63	319	288	288	288
query64	9883	2261	1718	1718
query65	3150	3094	3103	3094
query66	1541	337	333	333
query67	15657	14892	14888	14888
query68	6447	535	540	535
query69	684	447	322	322
query70	1203	1144	1121	1121
query71	482	279	280	279
query72	8529	5566	5629	5566
query73	771	321	321	321
query74	5881	5485	5475	5475
query75	4856	2684	2637	2637
query76	4348	933	942	933
query77	741	305	297	297
query78	11047	10159	9877	9877
query79	6366	531	528	528
query80	913	477	466	466
query81	553	217	216	216
query82	1477	108	109	108
query83	361	177	171	171
query84	268	86	87	86
query85	765	314	303	303
query86	438	311	311	311
query87	3294	3072	3080	3072
query88	4955	2372	2377	2372
query89	506	393	396	393
query90	2032	189	194	189
query91	130	97	100	97
query92	67	49	50	49
query93	5144	501	507	501
query94	1142	196	188	188
query95	417	325	320	320
query96	597	263	265	263
query97	3263	3070	3081	3070
query98	224	206	188	188
query99	1245	842	859	842
Total cold run time: 291970 ms
Total hot run time: 172168 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.68 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0d8c3d8ad08a02d65cc30b15efabd7fa23773449, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.22	0.06	0.05
query4	1.67	0.07	0.08
query5	0.49	0.49	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.55	0.48	0.49
query10	0.55	0.54	0.53
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.60	0.58
query14	0.76	0.78	0.78
query15	0.86	0.82	0.83
query16	0.36	0.36	0.37
query17	1.03	0.97	0.96
query18	0.21	0.23	0.25
query19	1.78	1.71	1.67
query20	0.01	0.02	0.01
query21	15.46	0.77	0.66
query22	3.83	7.51	2.06
query23	18.29	1.37	1.26
query24	2.11	0.23	0.22
query25	0.15	0.08	0.09
query26	0.26	0.18	0.18
query27	0.09	0.08	0.08
query28	13.22	1.04	1.02
query29	12.62	3.27	3.27
query30	0.25	0.07	0.05
query31	2.86	0.39	0.39
query32	3.27	0.49	0.46
query33	2.86	2.93	2.91
query34	17.09	4.42	4.45
query35	4.50	4.48	4.48
query36	0.65	0.47	0.46
query37	0.18	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.14
query41	0.08	0.04	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.95 s
Total hot run time: 30.68 s

@starocean999 starocean999 marked this pull request as ready for review July 1, 2024 09:35
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 1, 2024

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit 707cb40 into apache:master Jul 1, 2024
27 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Jul 2, 2024
@starocean999 @dataroaring
[feature](nereids)use mtmv to match legacy mv (#33699)
f5eb1db
@liutang123 liutang123 mentioned this pull request Jul 4, 2024
Open
3 tasks
@seawinde seawinde mentioned this pull request Jul 4, 2024
Merged
starocean999 pushed a commit that referenced this pull request Jul 8, 2024
@seawinde
[fix](mtmv) Fix rewrite wrongly when sync and async materialized view…
5c03890 
… name is same (#37311)

## Proposed changes

This is brought by #33699

`AbstractMaterializedViewRule#isMaterializationValid` check would fail
when sync and async materialized view name is same, because the
Materialization valid check result is cached by name。this will cause the
async materialized view check fail wrongly. This fix this by make sync
materialized view name unique in every place.
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@seawinde @dataroaring
[fix](mtmv) Fix rewrite wrongly when sync and async materialized view…
bf24d42 
… name is same (#37311)

## Proposed changes

This is brought by #33699

`AbstractMaterializedViewRule#isMaterializationValid` check would fail
when sync and async materialized view name is same, because the
Materialization valid check result is cached by name。this will cause the
async materialized view check fail wrongly. This fix this by make sync
materialized view name unique in every place.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@englefly englefly englefly approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged not-merge/2.1 reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@starocean999 @doris-robot @englefly @morrySnow @dataroaring