Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](executor) improve readability and efficiency of the checks #29696

Merged
merged 1 commit into from
Jan 9, 2024

Conversation

xingyingone
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@xingyingone
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jan 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.66% (8639/23566)
Line Coverage: 28.68% (70226/244842)
Region Coverage: 27.63% (36322/131448)
Branch Coverage: 24.31% (18557/76320)
Coverage Report: http://coverage.selectdb-in.cc/coverage/89887da04e9321a61548abc5c67064507a00d434_89887da04e9321a61548abc5c67064507a00d434/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38430 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false

------ Round 1 ----------------------------------
q1	17780	4943	4904	4904
q2	2015	156	141	141
q3	10586	1118	1200	1118
q4	10457	819	830	819
q5	7798	2967	2917	2917
q6	202	122	125	122
q7	899	514	496	496
q8	9301	1968	1963	1963
q9	6429	6361	6394	6361
q10	8224	3050	2998	2998
q11	429	209	210	209
q12	380	229	231	229
q13	18003	3374	3381	3374
q14	239	213	207	207
q15	544	504	508	504
q16	434	393	396	393
q17	948	706	588	588
q18	7259	6632	6616	6616
q19	1545	1507	1509	1507
q20	704	297	291	291
q21	2707	2415	2364	2364
q22	359	309	323	309
Total cold run time: 107242 ms
Total hot run time: 38430 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4891	4930	4912	4912
q2	331	250	243	243
q3	3265	3283	3246	3246
q4	2096	2002	1996	1996
q5	5769	5746	5742	5742
q6	204	113	117	113
q7	2304	1884	1869	1869
q8	3360	3425	3452	3425
q9	8787	8716	8743	8716
q10	3779	3835	3842	3835
q11	544	447	458	447
q12	798	626	628	626
q13	6147	3176	3178	3176
q14	295	278	255	255
q15	544	504	494	494
q16	554	496	494	494
q17	1841	1848	1856	1848
q18	8553	8369	8217	8217
q19	1579	1637	1611	1611
q20	2176	1942	1934	1934
q21	5622	5187	5245	5187
q22	502	460	477	460
Total cold run time: 63941 ms
Total hot run time: 58846 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184763 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	937	339	338	338
query2	6567	2011	1923	1923
query3	6716	234	225	225
query4	26830	22558	22555	22555
query5	5769	603	583	583
query6	334	230	211	211
query7	4655	307	288	288
query8	273	241	231	231
query9	8743	2947	2948	2947
query10	554	336	313	313
query11	16320	15789	15757	15757
query12	170	114	98	98
query13	1693	351	343	343
query14	13543	7757	7529	7529
query15	268	245	216	216
query16	6437	340	334	334
query17	1701	522	535	522
query18	1950	309	300	300
query19	248	165	165	165
query20	105	97	105	97
query21	188	95	95	95
query22	4908	5012	4917	4917
query23	32567	31601	31516	31516
query24	12237	2994	2992	2992
query25	600	368	360	360
query26	1843	183	171	171
query27	3027	312	302	302
query28	6942	2135	2123	2123
query29	2063	421	419	419
query30	297	145	150	145
query31	1021	827	820	820
query32	130	86	88	86
query33	840	344	343	343
query34	846	480	471	471
query35	1001	918	866	866
query36	1377	1245	1295	1245
query37	239	93	88	88
query38	3441	3362	3359	3359
query39	1341	1319	1459	1319
query40	302	92	89	89
query41	38	35	36	35
query42	114	111	110	110
query43	583	532	530	530
query44	1207	808	815	808
query45	215	206	212	206
query46	1063	722	743	722
query47	1767	1683	1610	1610
query48	367	320	284	284
query49	1207	322	322	322
query50	740	339	328	328
query51	5452	5270	5309	5270
query52	123	97	99	97
query53	242	175	170	170
query54	1419	687	699	687
query55	116	106	102	102
query56	304	245	288	245
query57	1033	1024	972	972
query58	328	294	290	290
query59	3087	2777	2793	2777
query60	348	302	305	302
query61	133	146	149	146
query62	571	500	484	484
query63	199	183	188	183
query64	6033	1710	1718	1710
query65	3428	3358	3364	3358
query66	1372	362	415	362
query67	15699	15569	15349	15349
query68	13012	541	547	541
query69	597	302	325	302
query70	1796	1605	1654	1605
query71	606	296	258	258
query72	5304	3400	3412	3400
query73	3045	339	344	339
query74	6963	6439	6560	6439
query75	5020	2328	2292	2292
query76	6303	1059	1178	1059
query77	749	303	311	303
query78	9020	8603	8507	8507
query79	1093	535	531	531
query80	574	347	347	347
query81	462	213	213	213
query82	241	112	106	106
query83	188	143	140	140
query84	250	57	57	57
query85	954	281	259	259
query86	436	445	413	413
query87	3563	3449	3430	3430
query88	3085	2513	2532	2513
query89	375	283	269	269
query90	1890	267	253	253
query91	122	95	97	95
query92	81	89	69	69
query93	1139	503	503	503
query94	913	249	224	224
query95	586	510	468	468
query96	637	341	335	335
query97	4336	4231	4338	4231
query98	238	224	220	220
query99	1171	898	882	882
Total cold run time: 300960 ms
Total hot run time: 184763 ms

Copy link
Contributor

@adonis0147 adonis0147 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.13 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.3 seconds inserted 10000000 Rows, about 366K ops/s
storage size: 17184131717 Bytes

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 9, 2024
Copy link
Contributor

github-actions bot commented Jan 9, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Jan 9, 2024

PR approved by anyone and no changes requested.

@apache apache deleted a comment from xingyingone Jan 9, 2024
@yiguolei yiguolei merged commit 5230478 into apache:master Jan 9, 2024
31 of 35 checks passed
yiguolei pushed a commit that referenced this pull request Jan 12, 2024
…cks (#29696)

Co-authored-by: xingying01 <xingying01@corp.netease.com>
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
…cks (apache#29696)

Co-authored-by: xingying01 <xingying01@corp.netease.com>
wsjz pushed a commit to wsjz/incubator-doris that referenced this pull request Feb 4, 2024
…cks (apache#29696)

Co-authored-by: xingying01 <xingying01@corp.netease.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants