Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Opt](exec) Reduce unnecessary find function calls #32165

Merged
merged 1 commit into from
Mar 13, 2024
Merged

[Opt](exec) Reduce unnecessary find function calls #32165

merged 1 commit into from
Mar 13, 2024

Conversation

xingyingone
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@liaoxin01
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 36812 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f73a572fa0eaac439486ca8f8df1d571a07e3731, data reload: false

------ Round 1 ----------------------------------
q1	17632	4245	4142	4142
q2	2018	149	149	149
q3	10826	954	928	928
q4	4769	694	723	694
q5	8168	2906	2814	2814
q6	185	129	124	124
q7	1236	858	841	841
q8	10106	2122	2077	2077
q9	8305	6614	6442	6442
q10	8157	2520	2489	2489
q11	420	224	220	220
q12	705	295	299	295
q13	17983	3021	2981	2981
q14	276	249	266	249
q15	507	460	455	455
q16	483	409	395	395
q17	948	566	571	566
q18	6835	5829	5804	5804
q19	1575	1399	1464	1399
q20	571	285	295	285
q21	7350	3277	3169	3169
q22	793	305	294	294
Total cold run time: 109848 ms
Total hot run time: 36812 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4132	4179	4122	4122
q2	322	215	230	215
q3	3000	2828	2829	2828
q4	1820	1542	1531	1531
q5	5215	5238	5235	5235
q6	192	121	117	117
q7	2236	1847	1842	1842
q8	3145	3272	3246	3246
q9	8512	8521	8513	8513
q10	6151	3668	3619	3619
q11	546	451	437	437
q12	729	574	567	567
q13	9090	2872	2857	2857
q14	269	248	244	244
q15	482	469	434	434
q16	433	417	411	411
q17	1731	1508	1472	1472
q18	7640	7400	7035	7035
q19	6251	1554	1479	1479
q20	1944	1711	1681	1681
q21	4732	4705	4759	4705
q22	562	443	444	443
Total cold run time: 69134 ms
Total hot run time: 53033 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.01% (8576/24497)
Line Coverage: 26.76% (69473/259572)
Region Coverage: 26.03% (36076/138581)
Branch Coverage: 22.99% (18419/80132)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f73a572fa0eaac439486ca8f8df1d571a07e3731_f73a572fa0eaac439486ca8f8df1d571a07e3731/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 187156 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f73a572fa0eaac439486ca8f8df1d571a07e3731, data reload: false

query1	922	370	346	346
query2	7752	1928	1971	1928
query3	6728	214	218	214
query4	31509	20689	20562	20562
query5	4440	431	404	404
query6	262	174	173	173
query7	4614	287	284	284
query8	231	175	175	175
query9	9754	2296	2266	2266
query10	453	241	259	241
query11	17125	14278	14069	14069
query12	131	89	81	81
query13	1626	418	423	418
query14	10675	10813	10949	10813
query15	269	209	196	196
query16	7903	262	253	253
query17	1947	587	529	529
query18	2034	293	276	276
query19	219	164	159	159
query20	92	84	83	83
query21	204	133	123	123
query22	4603	4409	4416	4409
query23	31702	31208	31294	31208
query24	11728	2855	2839	2839
query25	672	382	382	382
query26	1890	157	154	154
query27	3251	351	359	351
query28	8279	1860	1879	1860
query29	1068	675	619	619
query30	325	148	148	148
query31	966	724	739	724
query32	95	61	63	61
query33	770	273	273	273
query34	1149	475	479	475
query35	847	630	611	611
query36	1006	861	872	861
query37	283	77	77	77
query38	3296	3168	3110	3110
query39	1408	1391	1367	1367
query40	287	108	109	108
query41	50	46	45	45
query42	99	95	96	95
query43	488	468	471	468
query44	1397	737	713	713
query45	279	240	247	240
query46	1057	683	677	677
query47	1671	1609	1609	1609
query48	429	366	356	356
query49	1233	320	321	320
query50	754	363	359	359
query51	6837	6708	6753	6708
query52	104	93	87	87
query53	338	271	287	271
query54	310	225	228	225
query55	83	84	80	80
query56	233	217	213	213
query57	1083	1000	987	987
query58	221	191	198	191
query59	2879	2619	2599	2599
query60	267	233	250	233
query61	98	91	91	91
query62	574	423	389	389
query63	300	268	273	268
query64	6655	3870	3564	3564
query65	3123	3035	2999	2999
query66	1384	314	313	313
query67	14840	14322	14301	14301
query68	6949	531	537	531
query69	601	382	371	371
query70	1212	1200	1191	1191
query71	500	291	279	279
query72	6549	2727	2597	2597
query73	734	323	320	320
query74	8266	6602	6607	6602
query75	9331	7996	8027	7996
query76	4804	931	978	931
query77	642	273	267	267
query78	10336	9618	9678	9618
query79	8036	530	516	516
query80	1408	456	403	403
query81	509	212	216	212
query82	978	201	204	201
query83	212	159	151	151
query84	287	82	82	82
query85	1541	361	322	322
query86	472	299	298	298
query87	3443	3223	3176	3176
query88	4827	2310	2301	2301
query89	531	357	351	351
query90	1962	177	171	171
query91	167	126	127	126
query92	56	47	45	45
query93	5935	514	508	508
query94	1156	171	169	169
query95	414	324	319	319
query96	613	274	267	267
query97	4431	4259	4292	4259
query98	225	223	202	202
query99	1217	743	748	743
Total cold run time: 317645 ms
Total hot run time: 187156 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.72 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f73a572fa0eaac439486ca8f8df1d571a07e3731, data reload: false

query1	0.03	0.04	0.03
query2	0.05	0.03	0.03
query3	0.22	0.06	0.06
query4	1.68	0.10	0.10
query5	0.49	0.53	0.49
query6	1.21	0.62	0.62
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.55	0.49	0.49
query10	0.54	0.55	0.54
query11	0.12	0.08	0.08
query12	0.12	0.09	0.10
query13	0.56	0.56	0.57
query14	0.75	0.74	0.74
query15	0.80	0.80	0.78
query16	0.41	0.39	0.40
query17	1.01	1.03	0.98
query18	0.20	0.27	0.24
query19	1.84	1.81	1.82
query20	0.02	0.01	0.01
query21	15.44	0.58	0.58
query22	2.79	1.92	1.90
query23	17.13	0.72	0.71
query24	3.14	1.85	0.20
query25	0.14	0.47	0.10
query26	0.43	0.12	0.12
query27	0.04	0.03	0.03
query28	11.79	0.86	0.84
query29	12.59	3.14	3.16
query30	0.64	0.57	0.61
query31	2.78	0.33	0.34
query32	3.40	0.44	0.43
query33	2.77	2.87	2.79
query34	15.98	4.29	4.32
query35	4.31	4.33	4.36
query36	1.10	1.00	0.99
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.02	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 105.53 s
Total hot run time: 29.72 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f73a572fa0eaac439486ca8f8df1d571a07e3731 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.6 seconds inserted 10000000 Rows, about 684K ops/s

@liaoxin01 liaoxin01 merged commit ae7d0d2 into apache:master Mar 13, 2024
25 of 30 checks passed
yiguolei pushed a commit that referenced this pull request Mar 15, 2024
@xingyingone @Doris-Extras
[Opt](exec) Reduce unnecessary find function calls (#32165)
e6f9bd4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@xingyingone @doris-robot @liaoxin01 @yiguolei @alexxing662