Skip to content

[Performance](exec) Fix sin performance problem#36358

Merged
HappenLee merged 1 commit intoapache:masterfrom
HappenLee:simd
Jun 17, 2024
Merged

[Performance](exec) Fix sin performance problem#36358
HappenLee merged 1 commit intoapache:masterfrom
HappenLee:simd

Conversation

@HappenLee
Copy link
Copy Markdown
Contributor

Proposed changes

Before:

mysql [rqg]>select count(sin(f64)) from nums;
+-----------------+
| count(sin(f64)) |
+-----------------+
|        40000000 |
+-----------------+
1 row in set (10.73 sec)

After:

mysql [rqg]>select count(sin(f64)) from nums;
+-----------------+
| count(sin(f64)) |
+-----------------+
|        40000000 |
+-----------------+
1 row in set (0.57 sec)

revert the patch after we support the new toolchain in Doris:https://github.com/amosbird/ldb_toolchain_gen/releases/tag/v0.22

Thanks @amosbird

@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@HappenLee
Copy link
Copy Markdown
Contributor Author

run buildall

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 39879 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 950b74e895581f669924459d23dfcc31e4cb594d, data reload: false

------ Round 1 ----------------------------------
q1	18155	4466	4408	4408
q2	2984	193	194	193
q3	11821	1132	1149	1132
q4	10511	820	837	820
q5	7545	2694	2653	2653
q6	225	141	143	141
q7	963	622	613	613
q8	9258	2061	2057	2057
q9	8852	6453	6458	6453
q10	8978	3740	3747	3740
q11	448	236	242	236
q12	403	232	226	226
q13	18712	2983	2944	2944
q14	263	210	224	210
q15	525	478	475	475
q16	500	388	377	377
q17	970	729	741	729
q18	7989	7590	7277	7277
q19	6437	1528	1416	1416
q20	664	326	347	326
q21	4896	3114	3970	3114
q22	382	339	340	339
Total cold run time: 121481 ms
Total hot run time: 39879 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4441	4414	4194	4194
q2	379	274	275	274
q3	2930	2769	2709	2709
q4	1858	1610	1627	1610
q5	5246	5266	5276	5266
q6	220	127	130	127
q7	2097	1752	1755	1752
q8	3209	3347	3291	3291
q9	8309	8364	8305	8305
q10	3864	3652	3668	3652
q11	581	468	465	465
q12	749	622	600	600
q13	17298	2944	3000	2944
q14	293	258	254	254
q15	520	478	472	472
q16	474	422	403	403
q17	1768	1504	1496	1496
q18	7727	7455	7433	7433
q19	1684	1552	1482	1482
q20	1963	1803	1756	1756
q21	4795	4753	4653	4653
q22	638	536	541	536
Total cold run time: 71043 ms
Total hot run time: 53674 ms

@doris-robot
Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.48% (9003/24679)
Line Coverage: 28.02% (73735/263135)
Region Coverage: 27.49% (38304/139316)
Branch Coverage: 24.19% (19523/80708)
Coverage Report: http://coverage.selectdb-in.cc/coverage/950b74e895581f669924459d23dfcc31e4cb594d_950b74e895581f669924459d23dfcc31e4cb594d/report/index.html

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 170904 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 950b74e895581f669924459d23dfcc31e4cb594d, data reload: false

query1	936	379	373	373
query2	6453	2608	2368	2368
query3	6650	206	224	206
query4	19340	17198	17210	17198
query5	4156	459	471	459
query6	242	157	153	153
query7	4587	302	289	289
query8	350	297	284	284
query9	8704	2360	2364	2360
query10	596	292	298	292
query11	10458	10045	9895	9895
query12	137	86	84	84
query13	1639	359	365	359
query14	9401	7310	6063	6063
query15	220	190	185	185
query16	7831	275	271	271
query17	1401	547	549	547
query18	1964	298	262	262
query19	197	152	153	152
query20	91	79	84	79
query21	209	132	122	122
query22	4312	3988	3909	3909
query23	33813	33022	32952	32952
query24	12141	2875	2837	2837
query25	673	358	359	358
query26	1790	149	147	147
query27	3098	316	313	313
query28	7583	2037	2025	2025
query29	1125	615	604	604
query30	286	146	150	146
query31	937	719	747	719
query32	96	55	62	55
query33	795	283	272	272
query34	992	469	456	456
query35	753	603	636	603
query36	1076	958	939	939
query37	184	70	68	68
query38	2871	2721	2710	2710
query39	859	810	806	806
query40	281	126	125	125
query41	56	53	53	53
query42	123	94	102	94
query43	596	585	537	537
query44	1228	734	730	730
query45	199	167	170	167
query46	1090	703	725	703
query47	1818	1740	1780	1740
query48	369	299	292	292
query49	1182	420	407	407
query50	768	402	389	389
query51	6796	6658	6648	6648
query52	96	93	91	91
query53	352	293	285	285
query54	991	446	441	441
query55	73	74	75	74
query56	279	256	267	256
query57	1108	1059	1020	1020
query58	258	241	247	241
query59	3423	3057	3143	3057
query60	287	270	273	270
query61	92	88	107	88
query62	643	455	448	448
query63	325	294	294	294
query64	9879	2223	1741	1741
query65	3190	3068	3074	3068
query66	1369	342	336	336
query67	15259	14882	15003	14882
query68	4597	539	537	537
query69	470	301	309	301
query70	1088	1136	1089	1089
query71	395	275	266	266
query72	7070	5514	5767	5514
query73	742	322	319	319
query74	5864	5554	5426	5426
query75	3403	2668	2638	2638
query76	2500	854	852	852
query77	453	299	291	291
query78	10289	9853	9657	9657
query79	2544	503	510	503
query80	1843	458	453	453
query81	535	223	220	220
query82	823	103	100	100
query83	277	166	167	166
query84	280	93	86	86
query85	2089	289	276	276
query86	498	308	318	308
query87	3308	3077	3068	3068
query88	3983	2368	2378	2368
query89	477	377	386	377
query90	1850	191	198	191
query91	145	109	111	109
query92	65	54	50	50
query93	2552	514	498	498
query94	1294	194	200	194
query95	419	319	343	319
query96	618	270	270	270
query97	3296	3063	3022	3022
query98	218	202	196	196
query99	1249	868	843	843
Total cold run time: 276310 ms
Total hot run time: 170904 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 950b74e895581f669924459d23dfcc31e4cb594d, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.06	0.05
query4	1.66	0.06	0.06
query5	0.48	0.46	0.47
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.53	0.49	0.51
query10	0.54	0.56	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.13
query13	0.59	0.59	0.60
query14	0.77	0.79	0.81
query15	0.83	0.80	0.82
query16	0.35	0.34	0.36
query17	0.96	1.00	1.00
query18	0.22	0.27	0.23
query19	1.84	1.71	1.70
query20	0.01	0.01	0.01
query21	15.42	0.66	0.65
query22	4.19	7.52	2.16
query23	18.28	1.38	1.23
query24	2.19	0.21	0.21
query25	0.15	0.08	0.09
query26	0.27	0.18	0.18
query27	0.08	0.09	0.07
query28	13.20	1.02	1.00
query29	12.58	3.22	3.23
query30	0.26	0.07	0.05
query31	2.86	0.39	0.38
query32	3.27	0.47	0.47
query33	2.87	2.89	2.94
query34	17.13	4.41	4.42
query35	4.49	4.46	4.49
query36	0.65	0.46	0.47
query37	0.17	0.15	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.25 s
Total hot run time: 30.67 s

@github-actions
Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@HappenLee
Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot
Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.48% (9003/24679)
Line Coverage: 28.02% (73726/263134)
Region Coverage: 27.49% (38301/139316)
Branch Coverage: 24.19% (19522/80708)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a89004113ef0c6328730dddc0756a96df226b2ab_a89004113ef0c6328730dddc0756a96df226b2ab/report/index.html

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 39713 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a89004113ef0c6328730dddc0756a96df226b2ab, data reload: false

------ Round 1 ----------------------------------
q1	17610	4716	4312	4312
q2	2012	193	193	193
q3	10448	1160	1059	1059
q4	10187	819	782	782
q5	7526	2702	2626	2626
q6	227	134	138	134
q7	949	603	598	598
q8	9217	2071	2078	2071
q9	8999	6471	6433	6433
q10	8952	3713	3739	3713
q11	446	232	236	232
q12	435	240	234	234
q13	17765	2985	2999	2985
q14	268	219	228	219
q15	527	501	489	489
q16	521	381	382	381
q17	969	731	621	621
q18	7996	7305	7365	7305
q19	1943	1560	1492	1492
q20	642	310	324	310
q21	5032	3181	3317	3181
q22	389	350	343	343
Total cold run time: 113060 ms
Total hot run time: 39713 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4413	4263	4242	4242
q2	363	267	261	261
q3	2986	2744	2761	2744
q4	2027	1764	1686	1686
q5	5582	5615	5548	5548
q6	231	130	131	130
q7	2232	1835	1818	1818
q8	3346	3401	3410	3401
q9	8741	8683	8742	8683
q10	4164	3901	3860	3860
q11	574	496	495	495
q12	765	596	622	596
q13	16965	3196	3194	3194
q14	305	278	291	278
q15	533	481	514	481
q16	503	445	429	429
q17	1809	1525	1507	1507
q18	8054	7713	8001	7713
q19	1835	1613	1605	1605
q20	2172	1870	1904	1870
q21	5120	5029	5026	5026
q22	640	581	556	556
Total cold run time: 73360 ms
Total hot run time: 56123 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 174579 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a89004113ef0c6328730dddc0756a96df226b2ab, data reload: false

query1	917	382	366	366
query2	6468	2463	2453	2453
query3	6641	218	207	207
query4	19352	17418	17431	17418
query5	3633	479	473	473
query6	239	161	159	159
query7	4580	297	297	297
query8	331	287	286	286
query9	8349	2371	2336	2336
query10	552	304	278	278
query11	10578	9917	10043	9917
query12	134	86	86	86
query13	1637	368	371	368
query14	9958	7985	7868	7868
query15	245	185	197	185
query16	7696	269	275	269
query17	1712	567	545	545
query18	1874	289	313	289
query19	200	148	158	148
query20	92	82	85	82
query21	214	124	125	124
query22	4207	3938	4021	3938
query23	33986	33777	33689	33689
query24	11021	2864	2894	2864
query25	619	398	372	372
query26	977	158	162	158
query27	2367	320	329	320
query28	6532	2097	2125	2097
query29	875	635	666	635
query30	236	149	153	149
query31	1002	781	738	738
query32	95	51	58	51
query33	758	289	283	283
query34	1048	489	490	489
query35	751	686	635	635
query36	1150	955	1010	955
query37	146	73	72	72
query38	2934	2790	2836	2790
query39	913	851	827	827
query40	212	134	133	133
query41	61	59	54	54
query42	113	102	111	102
query43	619	574	571	571
query44	1271	731	734	731
query45	201	166	169	166
query46	1087	721	718	718
query47	1867	1760	1786	1760
query48	378	299	295	295
query49	864	437	431	431
query50	766	405	396	396
query51	6724	6792	6700	6700
query52	110	95	98	95
query53	361	302	295	295
query54	967	442	457	442
query55	78	81	74	74
query56	293	279	286	279
query57	1118	1079	1066	1066
query58	282	257	285	257
query59	3288	3431	3164	3164
query60	355	284	277	277
query61	97	93	86	86
query62	608	436	445	436
query63	322	305	298	298
query64	8746	2288	1743	1743
query65	3201	3132	3114	3114
query66	744	336	319	319
query67	15463	15045	14827	14827
query68	4552	541	551	541
query69	580	431	379	379
query70	1148	1125	1152	1125
query71	429	290	277	277
query72	7156	5240	5499	5240
query73	767	326	324	324
query74	5850	5528	5419	5419
query75	3401	2659	2611	2611
query76	2560	934	903	903
query77	641	303	305	303
query78	10423	9770	9770	9770
query79	2296	513	522	513
query80	1288	482	477	477
query81	606	232	229	229
query82	832	102	107	102
query83	265	170	170	170
query84	238	87	84	84
query85	1450	279	283	279
query86	467	318	335	318
query87	3276	3040	3086	3040
query88	3863	2359	2349	2349
query89	483	389	391	389
query90	1745	198	196	196
query91	130	107	99	99
query92	60	50	50	50
query93	2304	508	512	508
query94	1086	186	188	186
query95	402	312	315	312
query96	597	272	273	272
query97	3235	3050	3092	3050
query98	220	196	195	195
query99	1160	864	824	824
Total cold run time: 268725 ms
Total hot run time: 174579 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 31.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a89004113ef0c6328730dddc0756a96df226b2ab, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.03	0.04
query3	0.22	0.04	0.04
query4	1.69	0.06	0.07
query5	0.49	0.49	0.48
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.05
query9	0.56	0.46	0.49
query10	0.54	0.53	0.52
query11	0.16	0.11	0.12
query12	0.15	0.13	0.12
query13	0.58	0.58	0.59
query14	0.76	0.80	0.78
query15	0.83	0.82	0.82
query16	0.37	0.37	0.38
query17	1.04	1.01	1.03
query18	0.24	0.24	0.25
query19	1.86	1.71	1.76
query20	0.01	0.01	0.01
query21	15.40	0.66	0.66
query22	4.00	6.91	2.40
query23	18.31	1.40	1.31
query24	2.04	0.22	0.23
query25	0.15	0.08	0.09
query26	0.28	0.18	0.18
query27	0.08	0.08	0.08
query28	13.24	1.02	0.99
query29	12.65	3.30	3.32
query30	0.26	0.06	0.06
query31	2.96	0.39	0.38
query32	3.22	0.48	0.47
query33	2.91	2.82	2.87
query34	17.03	4.39	4.44
query35	4.51	4.52	4.45
query36	0.66	0.50	0.49
query37	0.18	0.16	0.15
query38	0.14	0.14	0.15
query39	0.04	0.04	0.04
query40	0.17	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.25 s
Total hot run time: 31.02 s

Copy link
Copy Markdown
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 17, 2024
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@HappenLee HappenLee merged commit 17ee30c into apache:master Jun 17, 2024
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.x dev/2.1.x dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants