Skip to content

[fix](nereids) scale broadcast distribution cost by backend count#65424

Open
foxtail463 wants to merge 1 commit into
apache:masterfrom
foxtail463:fix/cbo-be-count
Open

[fix](nereids) scale broadcast distribution cost by backend count#65424
foxtail463 wants to merge 1 commit into
apache:masterfrom
foxtail463:fix/cbo-be-count

Conversation

@foxtail463

Copy link
Copy Markdown
Contributor

No description provided.

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@foxtail463

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29374 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 904d445566d79479abbac53fc456624808fcc90b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17622	4140	4000	4000
q2	2056	333	203	203
q3	10274	1415	883	883
q4	4685	481	341	341
q5	7514	851	570	570
q6	185	173	138	138
q7	785	833	625	625
q8	9354	1662	1469	1469
q9	5652	4397	4367	4367
q10	6833	1786	1529	1529
q11	501	346	318	318
q12	708	545	434	434
q13	18092	3431	2733	2733
q14	264	257	247	247
q15	q16	783	783	723	723
q17	972	944	940	940
q18	7279	5667	5599	5599
q19	1303	1271	1024	1024
q20	729	648	528	528
q21	5919	2653	2406	2406
q22	434	348	297	297
Total cold run time: 101944 ms
Total hot run time: 29374 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4341	4267	4334	4267
q2	286	315	215	215
q3	4592	4978	4424	4424
q4	2058	2165	1380	1380
q5	4455	4330	4319	4319
q6	231	178	131	131
q7	1759	1971	1839	1839
q8	2656	2238	2212	2212
q9	7975	8152	7821	7821
q10	4721	4719	4249	4249
q11	568	421	379	379
q12	776	764	556	556
q13	3226	3603	2920	2920
q14	313	292	286	286
q15	q16	723	751	645	645
q17	1388	1347	1361	1347
q18	8034	7348	7379	7348
q19	1142	1090	1053	1053
q20	2212	2212	1916	1916
q21	5261	4640	4494	4494
q22	523	473	391	391
Total cold run time: 57240 ms
Total hot run time: 52192 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 180092 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 904d445566d79479abbac53fc456624808fcc90b, data reload: false

query5	4318	634	484	484
query6	470	252	236	236
query7	4864	597	352	352
query8	343	194	172	172
query9	8764	4021	4037	4021
query10	447	352	289	289
query11	5911	2344	2125	2125
query12	160	105	104	104
query13	1285	584	443	443
query14	6268	5292	4957	4957
query14_1	4323	4300	4288	4288
query15	228	213	182	182
query16	1026	495	446	446
query17	951	738	602	602
query18	2455	493	374	374
query19	218	194	162	162
query20	109	111	111	111
query21	233	155	136	136
query22	13781	13716	13377	13377
query23	17441	16605	16210	16210
query23_1	16328	16346	16410	16346
query24	7982	1825	1287	1287
query24_1	1294	1308	1307	1307
query25	572	470	403	403
query26	1316	381	205	205
query27	2612	575	378	378
query28	4505	2024	2014	2014
query29	1054	610	468	468
query30	332	257	221	221
query31	1122	1090	989	989
query32	101	59	59	59
query33	517	316	265	265
query34	1195	1169	655	655
query35	775	785	702	702
query36	1386	1344	1192	1192
query37	153	109	90	90
query38	1872	1700	1615	1615
query39	928	908	920	908
query39_1	879	901	889	889
query40	242	170	137	137
query41	65	62	63	62
query42	95	93	89	89
query43	325	327	281	281
query44	1430	792	770	770
query45	194	185	190	185
query46	1062	1213	738	738
query47	2344	2310	2220	2220
query48	384	392	306	306
query49	573	434	336	336
query50	1103	424	329	329
query51	10998	10729	10525	10525
query52	87	89	73	73
query53	260	276	202	202
query54	279	255	213	213
query55	74	74	69	69
query56	306	295	270	270
query57	1434	1420	1328	1328
query58	267	265	264	264
query59	1590	1621	1454	1454
query60	293	267	246	246
query61	151	151	145	145
query62	691	652	590	590
query63	235	210	206	206
query64	2843	1048	882	882
query65	4869	4767	4791	4767
query66	1839	514	385	385
query67	29477	29422	29188	29188
query68	3282	1485	967	967
query69	420	297	252	252
query70	1086	988	957	957
query71	345	313	299	299
query72	3112	2741	2390	2390
query73	829	742	468	468
query74	5091	4943	4740	4740
query75	2598	2574	2214	2214
query76	2304	1188	771	771
query77	349	367	299	299
query78	12242	12310	11722	11722
query79	1293	1103	734	734
query80	586	545	475	475
query81	472	323	277	277
query82	235	157	124	124
query83	312	319	292	292
query84	301	158	129	129
query85	890	627	502	502
query86	343	298	269	269
query87	1817	1814	1750	1750
query88	3688	2808	2788	2788
query89	434	404	345	345
query90	2143	207	197	197
query91	220	206	178	178
query92	62	58	56	56
query93	1511	1530	1063	1063
query94	543	340	312	312
query95	798	483	555	483
query96	1068	779	361	361
query97	2674	2671	2544	2544
query98	220	204	198	198
query99	1182	1135	1036	1036
Total cold run time: 264977 ms
Total hot run time: 180092 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 904d445566d79479abbac53fc456624808fcc90b, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.04
query3	0.25	0.14	0.13
query4	1.60	0.14	0.14
query5	0.27	0.24	0.22
query6	1.25	1.08	1.10
query7	0.03	0.01	0.01
query8	0.06	0.04	0.04
query9	0.39	0.30	0.31
query10	0.55	0.56	0.54
query11	0.20	0.13	0.14
query12	0.18	0.15	0.14
query13	0.47	0.47	0.48
query14	1.00	1.02	1.02
query15	0.61	0.60	0.59
query16	0.31	0.34	0.32
query17	1.10	1.14	1.08
query18	0.23	0.22	0.21
query19	2.04	1.93	2.00
query20	0.01	0.02	0.01
query21	15.43	0.19	0.14
query22	4.99	0.05	0.05
query23	16.13	0.31	0.11
query24	2.94	0.41	0.32
query25	0.10	0.05	0.05
query26	0.74	0.21	0.16
query27	0.05	0.04	0.03
query28	3.54	0.90	0.57
query29	12.59	4.06	3.23
query30	0.27	0.15	0.15
query31	2.77	0.61	0.31
query32	3.21	0.59	0.48
query33	3.16	3.15	3.21
query34	15.47	4.23	3.54
query35	3.54	3.50	3.54
query36	0.55	0.43	0.43
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.16
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 96.66 s
Total hot run time: 25.01 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants