Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat](Nereids) add use mv hint #40167

Merged
merged 7 commits into from
Sep 3, 2024
Merged

Conversation

LiBinfeng-01
Copy link
Collaborator

support hint like:
/*+ no_use_mv(tablename mvname) / which forbid tablename(indexname) to be choose
or /+ use_mv(tablename mvname) */ which choose specific materialize view

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37978 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a4ee6e775137b13bb7892331c22f3d8662e83466, data reload: false

------ Round 1 ----------------------------------
q1	17617	4460	4278	4278
q2	2013	188	177	177
q3	11909	972	1155	972
q4	10517	762	830	762
q5	7738	2840	2818	2818
q6	222	137	141	137
q7	972	607	596	596
q8	9328	2087	2068	2068
q9	7237	6527	6543	6527
q10	7009	2244	2234	2234
q11	463	242	243	242
q12	396	221	222	221
q13	18911	3040	3065	3040
q14	283	241	232	232
q15	519	492	491	491
q16	587	508	521	508
q17	973	684	686	684
q18	7300	6891	6758	6758
q19	1392	1043	989	989
q20	661	340	324	324
q21	3826	3020	2929	2929
q22	1124	991	1014	991
Total cold run time: 110997 ms
Total hot run time: 37978 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4335	4311	4379	4311
q2	381	273	278	273
q3	2843	2630	2617	2617
q4	1945	1655	1609	1609
q5	5608	5684	5729	5684
q6	229	144	142	142
q7	2231	1857	1798	1798
q8	3288	3426	3453	3426
q9	8813	8893	8805	8805
q10	3594	3393	3368	3368
q11	604	510	504	504
q12	875	674	656	656
q13	15261	3196	3266	3196
q14	331	300	276	276
q15	532	488	510	488
q16	639	579	573	573
q17	1845	1555	1533	1533
q18	8132	7919	7706	7706
q19	1725	1559	1520	1520
q20	2129	1911	1892	1892
q21	5679	5494	5525	5494
q22	1123	1021	1047	1021
Total cold run time: 72142 ms
Total hot run time: 56892 ms

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38328 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34fb6091609080355a72ff39962f8e557a06d92e, data reload: false

------ Round 1 ----------------------------------
q1	17668	4629	4306	4306
q2	2020	182	187	182
q3	10534	1199	1179	1179
q4	10342	755	728	728
q5	7758	2902	2824	2824
q6	228	138	140	138
q7	971	636	611	611
q8	9522	2081	2059	2059
q9	8089	6549	6533	6533
q10	7087	2239	2198	2198
q11	459	255	250	250
q12	490	224	231	224
q13	19003	3032	3029	3029
q14	284	245	246	245
q15	525	482	489	482
q16	607	520	515	515
q17	983	730	759	730
q18	7406	6844	6912	6844
q19	1384	1111	1074	1074
q20	696	337	337	337
q21	3952	3112	2845	2845
q22	1098	995	1011	995
Total cold run time: 111106 ms
Total hot run time: 38328 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4346	4269	4239	4239
q2	379	280	277	277
q3	2929	2641	2617	2617
q4	1913	1643	1677	1643
q5	5375	5401	5397	5397
q6	221	134	134	134
q7	2124	1756	1770	1756
q8	3449	3368	3380	3368
q9	8479	8474	8480	8474
q10	3439	3181	3211	3181
q11	615	506	513	506
q12	805	627	626	626
q13	11737	3005	3042	3005
q14	322	288	267	267
q15	529	479	484	479
q16	596	541	551	541
q17	1803	1503	1478	1478
q18	7696	7475	7533	7475
q19	1682	1612	1497	1497
q20	2095	1823	1805	1805
q21	5503	5328	5188	5188
q22	1145	1019	1003	1003
Total cold run time: 67182 ms
Total hot run time: 54956 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188674 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34fb6091609080355a72ff39962f8e557a06d92e, data reload: false

query1	916	381	368	368
query2	6436	1984	1897	1897
query3	6428	209	222	209
query4	28078	23332	23148	23148
query5	4113	522	528	522
query6	258	163	164	163
query7	4467	303	293	293
query8	255	225	219	219
query9	8150	2484	2476	2476
query10	441	289	283	283
query11	17752	15157	15054	15054
query12	156	103	99	99
query13	1628	398	374	374
query14	9370	7554	7197	7197
query15	258	170	179	170
query16	7852	456	398	398
query17	1532	560	544	544
query18	2049	305	281	281
query19	253	148	147	147
query20	118	127	110	110
query21	212	102	101	101
query22	4590	4253	4150	4150
query23	34420	33342	33422	33342
query24	10562	2966	2895	2895
query25	641	382	400	382
query26	1189	160	161	160
query27	2821	277	276	276
query28	7615	2123	2114	2114
query29	817	420	411	411
query30	289	159	147	147
query31	1040	739	772	739
query32	93	54	59	54
query33	743	300	290	290
query34	968	500	500	500
query35	868	750	723	723
query36	1081	949	956	949
query37	167	98	93	93
query38	4061	3899	3911	3899
query39	1447	1378	1358	1358
query40	205	120	117	117
query41	49	46	48	46
query42	118	97	101	97
query43	526	483	470	470
query44	1265	763	759	759
query45	198	178	173	173
query46	1111	798	746	746
query47	1895	1810	1813	1810
query48	391	298	303	298
query49	1104	457	481	457
query50	838	418	421	418
query51	7219	7014	7004	7004
query52	100	88	94	88
query53	261	188	188	188
query54	945	476	457	457
query55	81	75	80	75
query56	288	267	266	266
query57	1201	1078	1098	1078
query58	243	223	267	223
query59	3152	2847	2902	2847
query60	327	279	270	270
query61	103	104	101	101
query62	839	667	670	667
query63	223	192	186	186
query64	5309	703	670	670
query65	3245	3164	3186	3164
query66	1409	338	336	336
query67	15716	15543	15279	15279
query68	3084	581	591	581
query69	398	285	284	284
query70	1193	1155	1128	1128
query71	347	278	271	271
query72	6180	4006	3960	3960
query73	773	327	332	327
query74	9069	8979	8879	8879
query75	3378	2741	2791	2741
query76	1851	1031	975	975
query77	470	344	329	329
query78	10311	9044	9065	9044
query79	1086	553	548	548
query80	738	526	523	523
query81	548	240	234	234
query82	244	143	163	143
query83	178	154	153	153
query84	223	76	79	76
query85	709	306	345	306
query86	313	303	292	292
query87	4517	4217	4383	4217
query88	2945	2324	2313	2313
query89	388	299	298	298
query90	1824	203	202	202
query91	128	105	108	105
query92	60	56	52	52
query93	1040	546	539	539
query94	645	288	306	288
query95	361	276	273	273
query96	587	271	275	271
query97	3179	3065	3131	3065
query98	238	210	200	200
query99	1472	1259	1232	1232
Total cold run time: 280972 ms
Total hot run time: 188674 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34fb6091609080355a72ff39962f8e557a06d92e, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.03
query3	0.22	0.06	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.51
query10	0.54	0.53	0.52
query11	0.16	0.12	0.12
query12	0.14	0.12	0.13
query13	0.60	0.59	0.59
query14	2.04	2.08	2.03
query15	0.90	0.82	0.80
query16	0.37	0.38	0.37
query17	1.06	1.04	1.03
query18	0.21	0.20	0.20
query19	1.84	1.83	1.84
query20	0.02	0.01	0.01
query21	15.39	0.68	0.67
query22	4.01	6.87	1.89
query23	18.31	1.35	1.27
query24	2.08	0.21	0.22
query25	0.16	0.07	0.08
query26	0.26	0.18	0.17
query27	0.08	0.08	0.07
query28	13.28	1.01	1.00
query29	12.59	3.28	3.26
query30	0.24	0.06	0.06
query31	2.88	0.40	0.40
query32	3.26	0.49	0.48
query33	2.97	3.02	3.01
query34	17.18	4.40	4.40
query35	4.39	4.47	4.39
query36	0.65	0.48	0.47
query37	0.19	0.16	0.16
query38	0.16	0.17	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.06
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.63 s
Total hot run time: 32.02 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38101 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4ab99f8f778697bafc31fece93c0e2f0a3837e74, data reload: false

------ Round 1 ----------------------------------
q1	17604	4405	4298	4298
q2	2005	183	176	176
q3	11760	949	1081	949
q4	10513	709	781	709
q5	7756	2864	2825	2825
q6	228	142	144	142
q7	963	623	613	613
q8	9353	2088	2093	2088
q9	6954	6518	6523	6518
q10	6999	2240	2147	2147
q11	449	256	249	249
q12	390	218	225	218
q13	17769	3063	3089	3063
q14	289	245	234	234
q15	528	493	489	489
q16	595	518	508	508
q17	990	731	717	717
q18	7372	6877	6850	6850
q19	1390	1045	1092	1045
q20	668	352	333	333
q21	3938	2915	2934	2915
q22	1101	1044	1015	1015
Total cold run time: 109614 ms
Total hot run time: 38101 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4424	4297	4313	4297
q2	392	278	278	278
q3	2867	2717	2690	2690
q4	1941	1658	1641	1641
q5	5607	5700	5690	5690
q6	248	148	143	143
q7	2301	1818	1875	1818
q8	3312	3493	3454	3454
q9	8916	8863	8856	8856
q10	3533	3337	3366	3337
q11	633	518	538	518
q12	822	691	651	651
q13	11983	3221	3270	3221
q14	326	293	289	289
q15	535	511	492	492
q16	621	584	560	560
q17	1843	1547	1553	1547
q18	8115	7736	7744	7736
q19	1756	1579	1608	1579
q20	2113	1905	1912	1905
q21	5868	5576	5580	5576
q22	1177	1039	1048	1039
Total cold run time: 69333 ms
Total hot run time: 57317 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193607 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4ab99f8f778697bafc31fece93c0e2f0a3837e74, data reload: false

query1	1247	886	867	867
query2	6288	2030	2000	2000
query3	10651	3966	3903	3903
query4	59918	24973	23195	23195
query5	5491	498	507	498
query6	423	159	159	159
query7	5892	297	291	291
query8	293	219	225	219
query9	9162	2473	2487	2473
query10	505	268	260	260
query11	18179	15074	15442	15074
query12	155	100	101	100
query13	1573	406	383	383
query14	11148	7554	7732	7554
query15	229	175	180	175
query16	7134	466	489	466
query17	1156	572	596	572
query18	1440	314	319	314
query19	294	166	158	158
query20	131	116	112	112
query21	206	104	108	104
query22	4542	4435	4553	4435
query23	34608	33741	33591	33591
query24	5895	2797	2864	2797
query25	550	396	409	396
query26	695	161	158	158
query27	1766	280	278	278
query28	3669	2120	2108	2108
query29	653	421	435	421
query30	235	156	149	149
query31	910	783	776	776
query32	93	56	57	56
query33	443	299	293	293
query34	851	492	478	478
query35	833	730	718	718
query36	1071	934	952	934
query37	151	92	89	89
query38	3941	3969	3911	3911
query39	1427	1405	1398	1398
query40	200	118	118	118
query41	48	49	50	49
query42	117	99	97	97
query43	510	488	490	488
query44	1094	752	746	746
query45	196	170	173	170
query46	1103	739	727	727
query47	1882	1784	1773	1773
query48	366	303	299	299
query49	779	445	448	445
query50	817	426	425	425
query51	7117	7031	6986	6986
query52	104	87	89	87
query53	256	184	185	184
query54	587	455	464	455
query55	77	76	79	76
query56	284	262	272	262
query57	1179	1096	1098	1096
query58	232	255	240	240
query59	3025	2833	2845	2833
query60	309	280	279	279
query61	124	119	118	118
query62	768	658	653	653
query63	212	190	184	184
query64	2948	798	681	681
query65	3237	3175	3137	3137
query66	649	337	339	337
query67	15564	15191	15426	15191
query68	4385	554	562	554
query69	431	277	279	277
query70	1103	1068	1063	1063
query71	369	274	276	274
query72	6569	3988	4055	3988
query73	754	327	331	327
query74	9265	8810	8968	8810
query75	3393	2698	2711	2698
query76	1852	1005	1022	1005
query77	558	311	308	308
query78	9846	9090	9878	9090
query79	1315	543	537	537
query80	816	516	506	506
query81	573	230	231	230
query82	283	146	146	146
query83	223	151	152	151
query84	263	73	76	73
query85	884	302	286	286
query86	380	315	297	297
query87	4351	4299	4229	4229
query88	3088	2330	2307	2307
query89	389	279	294	279
query90	2021	197	194	194
query91	129	99	103	99
query92	61	53	53	53
query93	1522	538	534	534
query94	789	285	290	285
query95	349	266	257	257
query96	592	275	270	270
query97	3194	3030	3053	3030
query98	218	205	200	200
query99	1758	1251	1280	1251
Total cold run time: 309355 ms
Total hot run time: 193607 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4ab99f8f778697bafc31fece93c0e2f0a3837e74, data reload: false

query1	0.04	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.50	0.50	0.49
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.56	0.49	0.50
query10	0.54	0.54	0.55
query11	0.16	0.12	0.12
query12	0.14	0.13	0.12
query13	0.61	0.59	0.58
query14	2.06	2.12	2.07
query15	0.84	0.81	0.82
query16	0.37	0.38	0.38
query17	1.00	1.01	1.02
query18	0.21	0.20	0.20
query19	1.79	1.78	1.81
query20	0.01	0.01	0.02
query21	15.38	0.65	0.65
query22	4.44	6.24	2.32
query23	18.32	1.41	1.38
query24	2.14	0.22	0.22
query25	0.15	0.09	0.09
query26	0.27	0.18	0.19
query27	0.07	0.07	0.07
query28	13.21	1.04	0.99
query29	12.57	3.36	3.30
query30	0.25	0.06	0.05
query31	2.87	0.40	0.39
query32	3.27	0.49	0.47
query33	2.96	2.99	3.00
query34	16.91	4.35	4.42
query35	4.44	4.43	4.42
query36	0.66	0.46	0.48
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.14	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.7 s
Total hot run time: 32.51 s

@LiBinfeng-01
Copy link
Collaborator Author

run external

1 similar comment
@LiBinfeng-01
Copy link
Collaborator Author

run external

englefly
englefly previously approved these changes Sep 2, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 2, 2024
Copy link
Contributor

github-actions bot commented Sep 2, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Sep 2, 2024

PR approved by anyone and no changes requested.

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 2, 2024
@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41271 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b50f3a38e2b97c57917b6dfb7c3ec64a72f4165, data reload: false

------ Round 1 ----------------------------------
q1	5488	4315	4277	4277
q2	1683	180	180	180
q3	10425	4250	4180	4180
q4	11324	691	780	691
q5	7749	2851	2842	2842
q6	228	139	137	137
q7	959	614	604	604
q8	9340	2113	2102	2102
q9	7499	6590	6605	6590
q10	7035	2284	2193	2193
q11	450	244	239	239
q12	399	219	222	219
q13	18316	3085	3056	3056
q14	287	230	229	229
q15	518	504	497	497
q16	604	522	497	497
q17	987	631	672	631
q18	7329	6883	6913	6883
q19	1369	1055	1058	1055
q20	666	320	333	320
q21	3916	2828	3127	2828
q22	1126	1038	1021	1021
Total cold run time: 97697 ms
Total hot run time: 41271 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4356	4327	4268	4268
q2	371	280	262	262
q3	2864	2681	2696	2681
q4	1920	1664	1609	1609
q5	5402	5585	5704	5585
q6	221	138	139	138
q7	2243	1866	1851	1851
q8	3306	3451	3429	3429
q9	8733	8936	8880	8880
q10	3609	3335	3295	3295
q11	605	498	513	498
q12	873	680	693	680
q13	9592	3304	3246	3246
q14	303	268	264	264
q15	532	480	487	480
q16	614	594	584	584
q17	1835	1522	1552	1522
q18	8170	7848	7871	7848
q19	1729	1651	1614	1614
q20	2134	1925	1920	1920
q21	5675	5537	5430	5430
q22	1133	1076	1078	1076
Total cold run time: 66220 ms
Total hot run time: 57160 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193153 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b50f3a38e2b97c57917b6dfb7c3ec64a72f4165, data reload: false

query1	1243	884	881	881
query2	6203	1995	2002	1995
query3	10611	3851	3861	3851
query4	59405	27650	23207	23207
query5	5371	509	492	492
query6	407	173	155	155
query7	5745	298	295	295
query8	282	197	202	197
query9	8791	2460	2452	2452
query10	442	268	263	263
query11	16836	15021	15236	15021
query12	158	99	103	99
query13	1514	396	373	373
query14	11032	7439	7214	7214
query15	229	182	169	169
query16	7608	457	484	457
query17	1153	570	552	552
query18	2054	311	295	295
query19	293	139	142	139
query20	124	105	109	105
query21	205	100	101	100
query22	4965	4512	4883	4512
query23	34396	33627	33588	33588
query24	5919	2867	2827	2827
query25	520	385	390	385
query26	689	173	147	147
query27	1770	283	279	279
query28	3948	2054	2042	2042
query29	695	399	404	399
query30	225	149	156	149
query31	948	770	761	761
query32	72	52	54	52
query33	468	278	285	278
query34	878	476	472	472
query35	850	716	697	697
query36	1054	943	925	925
query37	142	93	84	84
query38	4043	4024	3975	3975
query39	1431	1400	1407	1400
query40	196	112	114	112
query41	48	46	48	46
query42	114	95	91	91
query43	516	502	485	485
query44	1095	743	748	743
query45	204	165	171	165
query46	1090	736	745	736
query47	1918	1782	1816	1782
query48	370	298	302	298
query49	774	440	475	440
query50	827	436	425	425
query51	6911	6896	6899	6896
query52	100	90	87	87
query53	253	176	179	176
query54	572	465	462	462
query55	76	77	82	77
query56	276	271	261	261
query57	1222	1072	1034	1034
query58	229	237	242	237
query59	3121	2868	2875	2868
query60	305	276	282	276
query61	125	117	116	116
query62	759	659	665	659
query63	215	187	184	184
query64	2868	755	738	738
query65	3226	3203	3157	3157
query66	687	341	346	341
query67	15472	15414	15390	15390
query68	3227	597	585	585
query69	407	288	282	282
query70	1098	1128	1129	1128
query71	349	285	280	280
query72	6484	4038	4045	4038
query73	755	326	328	326
query74	9095	8740	8903	8740
query75	3363	2662	2686	2662
query76	1427	917	1014	917
query77	516	316	314	314
query78	10003	9085	9169	9085
query79	1023	519	517	517
query80	764	523	526	523
query81	548	235	235	235
query82	250	151	148	148
query83	167	145	145	145
query84	252	74	82	74
query85	727	284	281	281
query86	299	278	298	278
query87	4377	4265	4241	4241
query88	3606	2294	2317	2294
query89	386	283	283	283
query90	2009	186	187	186
query91	124	95	98	95
query92	60	48	49	48
query93	1088	533	538	533
query94	712	295	295	295
query95	384	253	252	252
query96	584	264	261	261
query97	3252	3094	3085	3085
query98	233	204	195	195
query99	1544	1261	1272	1261
Total cold run time: 305401 ms
Total hot run time: 193153 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b50f3a38e2b97c57917b6dfb7c3ec64a72f4165, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.04	0.05
query4	1.67	0.08	0.07
query5	0.52	0.50	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.05	0.04
query9	0.55	0.50	0.50
query10	0.55	0.54	0.55
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.59	0.60
query14	2.06	2.06	2.11
query15	0.90	0.87	0.82
query16	0.37	0.38	0.37
query17	1.08	0.95	1.04
query18	0.20	0.19	0.20
query19	1.94	1.81	1.79
query20	0.02	0.01	0.01
query21	15.40	0.67	0.66
query22	4.33	6.93	1.93
query23	18.25	1.28	1.24
query24	2.08	0.24	0.23
query25	0.14	0.09	0.08
query26	0.28	0.18	0.17
query27	0.08	0.08	0.09
query28	13.16	1.01	1.00
query29	12.66	3.27	3.26
query30	0.24	0.05	0.05
query31	2.89	0.39	0.40
query32	3.29	0.47	0.47
query33	2.95	2.96	3.06
query34	17.02	4.43	4.36
query35	4.42	4.49	4.45
query36	0.65	0.49	0.48
query37	0.19	0.17	0.15
query38	0.16	0.14	0.14
query39	0.04	0.04	0.03
query40	0.15	0.13	0.13
query41	0.10	0.05	0.06
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.9 s
Total hot run time: 31.92 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38125 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f7afc9bb8b1a06f2579a5d24627bd08720229af, data reload: false

------ Round 1 ----------------------------------
q1	17623	4574	4339	4339
q2	2042	185	175	175
q3	11384	993	1123	993
q4	10492	792	681	681
q5	7757	2937	2810	2810
q6	228	144	140	140
q7	980	607	606	606
q8	9340	2084	2082	2082
q9	7104	6570	6575	6570
q10	7064	2298	2181	2181
q11	467	252	246	246
q12	394	223	226	223
q13	18999	3111	3066	3066
q14	301	230	231	230
q15	525	497	491	491
q16	624	542	518	518
q17	988	686	740	686
q18	7434	6965	6756	6756
q19	1386	1106	1076	1076
q20	652	344	320	320
q21	4056	2947	3034	2947
q22	1128	1019	989	989
Total cold run time: 110968 ms
Total hot run time: 38125 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4576	4359	4315	4315
q2	381	280	273	273
q3	2910	2665	2643	2643
q4	1931	1645	1649	1645
q5	5685	5666	5814	5666
q6	249	143	134	134
q7	2199	1879	1787	1787
q8	3295	3463	3518	3463
q9	8969	8916	8800	8800
q10	3598	3438	3407	3407
q11	617	520	502	502
q12	810	678	660	660
q13	14528	3207	3270	3207
q14	332	308	279	279
q15	541	493	486	486
q16	637	577	597	577
q17	1868	1569	1552	1552
q18	8179	7987	7922	7922
q19	1755	1553	1517	1517
q20	2175	1930	1959	1930
q21	5780	5536	5453	5453
q22	1133	1079	1077	1077
Total cold run time: 72148 ms
Total hot run time: 57295 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193135 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8f7afc9bb8b1a06f2579a5d24627bd08720229af, data reload: false

query1	1268	882	866	866
query2	6431	2005	1950	1950
query3	10638	4149	4021	4021
query4	59119	23983	23189	23189
query5	5430	512	497	497
query6	405	169	170	169
query7	5767	296	305	296
query8	302	214	210	210
query9	8745	2510	2497	2497
query10	492	274	262	262
query11	18102	15091	15272	15091
query12	167	115	118	115
query13	1565	394	391	391
query14	11267	7435	7350	7350
query15	228	190	185	185
query16	7457	508	475	475
query17	1125	605	582	582
query18	2015	321	317	317
query19	288	162	155	155
query20	124	111	111	111
query21	206	105	106	105
query22	4629	4536	4720	4536
query23	34396	33306	34244	33306
query24	5969	2912	2824	2824
query25	514	410	398	398
query26	681	159	157	157
query27	1769	281	280	280
query28	3768	2067	2048	2048
query29	648	428	423	423
query30	235	158	160	158
query31	945	749	774	749
query32	140	51	52	51
query33	436	285	266	266
query34	863	494	472	472
query35	828	718	748	718
query36	1070	903	968	903
query37	149	94	92	92
query38	4023	3905	3940	3905
query39	1458	1397	1403	1397
query40	198	113	110	110
query41	47	47	44	44
query42	118	95	93	93
query43	504	471	457	457
query44	1074	752	740	740
query45	198	165	164	164
query46	1102	748	726	726
query47	1926	1810	1824	1810
query48	381	294	302	294
query49	767	439	442	439
query50	810	427	418	418
query51	6948	6822	6879	6822
query52	99	84	86	84
query53	251	173	177	173
query54	568	461	449	449
query55	74	70	74	70
query56	269	252	269	252
query57	1171	1107	1044	1044
query58	230	221	224	221
query59	3049	2798	2816	2798
query60	288	271	262	262
query61	100	101	118	101
query62	734	649	655	649
query63	211	185	185	185
query64	2798	665	655	655
query65	3192	3117	3170	3117
query66	681	334	358	334
query67	15577	15369	15493	15369
query68	4357	558	545	545
query69	415	287	283	283
query70	1186	1106	1073	1073
query71	363	282	286	282
query72	6243	3996	4024	3996
query73	778	330	331	330
query74	9270	8840	8926	8840
query75	3395	2687	2664	2664
query76	1666	935	986	935
query77	548	312	317	312
query78	9740	9151	9136	9136
query79	1073	532	525	525
query80	787	509	509	509
query81	550	236	233	233
query82	280	151	142	142
query83	171	144	149	144
query84	266	74	79	74
query85	821	279	285	279
query86	324	317	309	309
query87	4502	4280	4359	4280
query88	3077	2391	2394	2391
query89	439	286	289	286
query90	1862	190	185	185
query91	123	98	98	98
query92	68	52	55	52
query93	1082	511	522	511
query94	736	280	287	280
query95	348	250	252	250
query96	598	280	271	271
query97	3231	3120	3055	3055
query98	212	205	203	203
query99	1577	1273	1265	1265
Total cold run time: 307115 ms
Total hot run time: 193135 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8f7afc9bb8b1a06f2579a5d24627bd08720229af, data reload: false

query1	0.04	0.04	0.05
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.69	0.08	0.08
query5	0.52	0.49	0.49
query6	1.12	0.72	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.54	0.54
query11	0.14	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.58	0.58
query14	2.06	2.02	2.12
query15	0.84	0.82	0.82
query16	0.39	0.38	0.37
query17	1.00	1.03	1.09
query18	0.22	0.21	0.21
query19	1.95	1.76	1.78
query20	0.00	0.01	0.01
query21	15.38	0.66	0.66
query22	4.45	7.24	1.57
query23	18.23	1.37	1.34
query24	2.08	0.24	0.22
query25	0.15	0.08	0.08
query26	0.28	0.18	0.19
query27	0.07	0.08	0.08
query28	13.25	1.03	1.01
query29	12.62	3.33	3.28
query30	0.25	0.06	0.06
query31	2.85	0.40	0.41
query32	3.24	0.47	0.48
query33	3.00	3.01	2.95
query34	17.01	4.38	4.42
query35	4.48	4.46	4.49
query36	0.64	0.47	0.50
query37	0.19	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.93 s
Total hot run time: 31.72 s

@LiBinfeng-01
Copy link
Collaborator Author

run p0

@LiBinfeng-01
Copy link
Collaborator Author

run cloud_p0

1 similar comment
@LiBinfeng-01
Copy link
Collaborator Author

run cloud_p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 3, 2024
Copy link
Contributor

github-actions bot commented Sep 3, 2024

PR approved by at least one committer and no changes requested.

@englefly englefly merged commit d0654cd into apache:master Sep 3, 2024
25 of 27 checks passed
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request Sep 3, 2024
support hint like:
/*+ no_use_mv(tablename mvname) / which forbid tablename(indexname) to
be choose
or /+ use_mv(tablename mvname) */ which choose specific materialize view
morrySnow pushed a commit that referenced this pull request Sep 13, 2024
remove unstable case which can not be sure in choise of mv
intro by: #40167
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request Oct 17, 2024
support hint like:
/*+ no_use_mv(tablename mvname) / which forbid tablename(indexname) to
be choose
or /+ use_mv(tablename mvname) */ which choose specific materialize view
morrySnow pushed a commit that referenced this pull request Oct 17, 2024
…#41869)

pick: #35925 #39715 #40167 #40958
Add feat of force use/nouse cbo rule hint and fix pr

introduce
when not using this hint, cbo rules like INFER_SET_OPERATOR_DISTINCT
would generate two plans and compare their cost
and nereids optimizer would decide which is better. But when we want to
control the behavior of cbo rules we could use this force cbo rule hint
usage example
explain shape plan
select /*+ USE_CBO_RULE(INFER_SET_OPERATOR_DISTINCT) */
*
from t1
union
select * from t2;
the USE_CBO_RULE(INFER_SET_OPERATOR_DISTINCT) hint would force rule
INFER_SET_OPERATOR_DISTINCT to be used
and generate plan like, which hashAgg below union is generated by this
rule:

-- !with_hint_union_distinct --
----hashAgg[GLOBAL]
--------hashAgg[LOCAL]
----------PhysicalUnion
--------------hashAgg[LOCAL]
----------------PhysicalOlapScan[t1]
--------------hashAgg[LOCAL]
----------------PhysicalOlapScan[t2]
Hint log:
Used: INFER_SET_OPERATOR_DISTINCT
UnUsed:
SyntaxError:
When we want to force disable this rule, we could use
explain shape plan select /*+
NO_USE_CBO_RULE(INFER_SET_OPERATOR_DISTINCT) */ * from t1 union select *
from t2;
which would generate plan with this rule:

-- !with_hint_no_union_distinct --
----hashAgg[GLOBAL]
--------hashAgg[LOCAL]
----------PhysicalUnion
--------------PhysicalOlapScan[t1]
--------------PhysicalOlapScan[t2]
Hint log:
Used: NO_INFER_SET_OPERATOR_DISTINCT
UnUsed:
SyntaxError:
change sessionvariable enableNereidsRules to varType.remove
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants