[feature](executor)Make workload group property not required by wangbo · Pull Request #27229 · apache/doris

wangbo · 2023-11-18T07:41:34Z

Proposed changes

Make workload group property not required.

github-actions

clang-tidy made some suggestions

be/src/runtime/task_group/task_group.h

fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/WorkloadGroup.java

github-actions · 2023-11-18T13:34:17Z

clang-tidy review says "All clean, LGTM! 👍"

yiguolei

LGTM

yiguolei · 2023-11-18T13:46:50Z

run buildall

github-actions · 2023-11-18T13:47:40Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-11-18T13:47:42Z

PR approved by anyone and no changes requested.

doris-robot · 2023-11-18T15:11:23Z

TeamCity be ut coverage result:
Function Coverage: 36.75% (8434/22952)
Line Coverage: 29.25% (68617/234572)
Region Coverage: 27.86% (35475/127342)
Branch Coverage: 24.59% (18104/73612)
Coverage Report: http://coverage.selectdb-in.cc/coverage/603c07445f9f4fcc819a25fe5fe70cbf455ee5d3_603c07445f9f4fcc819a25fe5fe70cbf455ee5d3/report/index.html

doris-robot · 2023-11-18T15:42:17Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 603c07445f9f4fcc819a25fe5fe70cbf455ee5d3, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4933	4684	4624	4624
q2	360	156	170	156
q3	2054	1943	1929	1929
q4	1392	1248	1237	1237
q5	3955	3907	4019	3907
q6	252	126	124	124
q7	1394	886	894	886
q8	2791	2814	2766	2766
q9	58614	9597	9599	9597
q10	12715	3530	3530	3530
q11	376	251	235	235
q12	1459	292	296	292
q13	18841	3762	3783	3762
q14	325	290	285	285
q15	570	535	516	516
q16	670	578	582	578
q17	1153	993	904	904
q18	7837	7377	7419	7377
q19	1730	1691	1677	1677
q20	580	281	295	281
q21	7611	4033	4040	4033
q22	475	371	370	370
Total cold run time: 130087 ms
Total hot run time: 49066 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4634	4590	4549	4549
q2	340	234	262	234
q3	4022	4003	4021	4003
q4	2714	2702	2706	2702
q5	9688	9589	9671	9589
q6	250	119	116	116
q7	3026	2499	2500	2499
q8	4444	4461	4429	4429
q9	12997	12821	12853	12821
q10	4053	4160	4137	4137
q11	804	655	649	649
q12	963	818	810	810
q13	4288	3580	3558	3558
q14	374	346	344	344
q15	564	518	518	518
q16	726	658	682	658
q17	3856	3828	3842	3828
q18	9630	9214	9080	9080
q19	1811	1823	1773	1773
q20	2397	2062	2048	2048
q21	8796	8472	8667	8472
q22	863	847	751	751
Total cold run time: 81240 ms
Total hot run time: 77568 ms

doris-robot · 2023-11-18T15:52:09Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.05 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.0 seconds inserted 10000000 Rows, about 357K ops/s
storage size: 17097874580 Bytes

wangbo · 2023-11-19T04:30:20Z

run buildall

doris-robot · 2023-11-19T05:55:40Z

TeamCity be ut coverage result:
Function Coverage: 36.75% (8434/22952)
Line Coverage: 29.25% (68611/234572)
Region Coverage: 27.86% (35473/127342)
Branch Coverage: 24.59% (18099/73612)
Coverage Report: http://coverage.selectdb-in.cc/coverage/76376f577fc1212d2b733be636609424240aa6e9_76376f577fc1212d2b733be636609424240aa6e9/report/index.html

doris-robot · 2023-11-19T06:28:43Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 76376f577fc1212d2b733be636609424240aa6e9, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4912	4677	4634	4634
q2	359	172	150	150
q3	2024	1914	1858	1858
q4	1398	1281	1247	1247
q5	3928	3942	4020	3942
q6	249	130	131	130
q7	1386	870	904	870
q8	2788	2788	2768	2768
q9	9774	9413	9439	9413
q10	3430	3525	3538	3525
q11	379	247	249	247
q12	433	290	296	290
q13	4569	3796	3751	3751
q14	316	280	281	280
q15	582	522	510	510
q16	665	591	586	586
q17	1128	955	940	940
q18	7952	7434	7481	7434
q19	1659	1686	1678	1678
q20	565	315	298	298
q21	4442	4014	3995	3995
q22	475	362	377	362
Total cold run time: 53413 ms
Total hot run time: 48908 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4614	4573	4571	4571
q2	336	218	276	218
q3	4039	4010	4018	4010
q4	2709	2705	2715	2705
q5	9832	9897	9839	9839
q6	243	123	121	121
q7	3006	2492	2500	2492
q8	4479	4446	4484	4446
q9	12972	12826	12904	12826
q10	4056	4127	4183	4127
q11	805	642	726	642
q12	971	822	795	795
q13	4323	3583	3590	3583
q14	380	339	351	339
q15	571	515	505	505
q16	733	679	665	665
q17	3895	3893	3862	3862
q18	9658	9154	9197	9154
q19	1820	1780	1776	1776
q20	2403	2069	2057	2057
q21	8994	8632	8613	8613
q22	865	799	765	765
Total cold run time: 81704 ms
Total hot run time: 78111 ms

doris-robot · 2023-11-19T06:39:50Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.71 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17098759667 Bytes

yiguolei

LGTM

github-actions · 2023-11-19T07:04:32Z

PR approved by at least one committer and no changes requested.

xinyiZzz

LGTM

…27229) * Make workload group property not required * remove useless UT

Make workload group property not required

188b6cc

wangbo added the workload-group label Nov 18, 2023

github-actions bot reviewed Nov 18, 2023

View reviewed changes

be/src/runtime/task_group/task_group.h Show resolved Hide resolved

yiguolei reviewed Nov 18, 2023

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/WorkloadGroup.java Show resolved Hide resolved

yiguolei previously approved these changes Nov 18, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 18, 2023

github-actions bot added the reviewed label Nov 18, 2023

remove useless UT

76376f5

wangbo dismissed yiguolei’s stale review via 76376f5 November 19, 2023 04:30

wangbo force-pushed the 1118_wg_req_property branch from 603c074 to 76376f5 Compare November 19, 2023 04:30

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 19, 2023

yiguolei approved these changes Nov 19, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 19, 2023

xinyiZzz approved these changes Nov 19, 2023

View reviewed changes

yiguolei merged commit 70e0701 into apache:master Nov 19, 2023

superdiaodiao pushed a commit to superdiaodiao/doris that referenced this pull request Nov 21, 2023

[feature](executor)Make workload group property not required (apache#…

1a3e422

…27229) * Make workload group property not required * remove useless UT

seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023

[feature](executor)Make workload group property not required (apache#…

349f3fc

…27229) * Make workload group property not required * remove useless UT

XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023

[feature](executor)Make workload group property not required (apache#…

72d57fc

…27229) * Make workload group property not required * remove useless UT

Conversation

wangbo commented Nov 18, 2023

Proposed changes

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 18, 2023

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

yiguolei commented Nov 18, 2023

Uh oh!

github-actions bot commented Nov 18, 2023

Uh oh!

github-actions bot commented Nov 18, 2023

Uh oh!

doris-robot commented Nov 18, 2023

Uh oh!

doris-robot commented Nov 18, 2023

Uh oh!

doris-robot commented Nov 18, 2023

Uh oh!

wangbo commented Nov 19, 2023

Uh oh!

doris-robot commented Nov 19, 2023

Uh oh!

doris-robot commented Nov 19, 2023

Uh oh!

doris-robot commented Nov 19, 2023

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 19, 2023

Uh oh!

xinyiZzz left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants