Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](clone) dead be will abort sched task #36795

Merged
merged 3 commits into from
Jun 26, 2024
Merged

[improvement](clone) dead be will abort sched task #36795

merged 3 commits into from
Jun 26, 2024

Conversation

yujun777
Copy link
Collaborator

@yujun777 yujun777 commented Jun 25, 2024 •
edited
Loading

When be is down, its related clone task need to abort. Otherwise this task need to wait until timeout.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Collaborator Author

run buildall

deardeng
deardeng approved these changes Jun 25, 2024
View reviewed changes
Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yujun777
Copy link
Collaborator Author

run buildall

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39739 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit be4962e0ebfaff3806e0114f52f3bdb91cfa8084, data reload: false

------ Round 1 ----------------------------------
q1	17623	4294	4274	4274
q2	2005	185	190	185
q3	10477	1122	1121	1121
q4	10206	763	813	763
q5	7542	2657	2656	2656
q6	213	132	132	132
q7	948	589	592	589
q8	9225	2061	2062	2061
q9	8851	6524	6483	6483
q10	8951	3683	3705	3683
q11	447	232	241	232
q12	421	220	227	220
q13	17764	2961	2976	2961
q14	259	221	219	219
q15	510	485	479	479
q16	512	383	392	383
q17	965	722	666	666
q18	8008	7406	7394	7394
q19	8055	1501	1436	1436
q20	663	325	348	325
q21	4860	3139	4012	3139
q22	379	341	338	338
Total cold run time: 118884 ms
Total hot run time: 39739 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4487	4239	4247	4239
q2	383	259	271	259
q3	3037	2925	2969	2925
q4	1954	1792	1803	1792
q5	5591	5535	5491	5491
q6	220	127	134	127
q7	2268	1861	1840	1840
q8	3293	3455	3489	3455
q9	8823	8852	8823	8823
q10	4180	3780	3972	3780
q11	631	525	525	525
q12	854	666	667	666
q13	16138	3362	3385	3362
q14	336	309	291	291
q15	541	482	515	482
q16	532	469	472	469
q17	1884	1568	1526	1526
q18	9111	8390	7810	7810
q19	1861	1667	1677	1667
q20	3150	1887	1851	1851
q21	5209	4937	4865	4865
q22	681	571	566	566
Total cold run time: 75164 ms
Total hot run time: 56811 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173828 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit be4962e0ebfaff3806e0114f52f3bdb91cfa8084, data reload: false

query1	910	426	356	356
query2	6419	2293	2193	2193
query3	6620	203	216	203
query4	21375	17459	17185	17185
query5	3593	458	463	458
query6	243	158	161	158
query7	4589	297	289	289
query8	298	268	271	268
query9	8452	2376	2376	2376
query10	563	276	299	276
query11	10578	10075	10099	10075
query12	113	79	83	79
query13	1631	363	367	363
query14	9955	7711	7520	7520
query15	244	188	188	188
query16	7454	274	272	272
query17	1902	519	515	515
query18	1436	270	268	268
query19	203	150	144	144
query20	86	80	79	79
query21	209	128	123	123
query22	4358	4154	4052	4052
query23	33889	33777	33564	33564
query24	11050	2885	2963	2885
query25	594	380	373	373
query26	1123	152	176	152
query27	2685	327	317	317
query28	7314	2122	2154	2122
query29	871	630	627	627
query30	247	155	151	151
query31	975	768	750	750
query32	100	51	53	51
query33	753	281	288	281
query34	1039	475	492	475
query35	764	666	634	634
query36	1134	997	969	969
query37	157	70	79	70
query38	2955	2935	2816	2816
query39	922	859	847	847
query40	221	123	126	123
query41	53	51	52	51
query42	111	100	95	95
query43	596	518	552	518
query44	1239	748	727	727
query45	194	158	162	158
query46	1069	719	725	719
query47	1870	1778	1788	1778
query48	377	298	287	287
query49	852	409	406	406
query50	780	398	390	390
query51	6941	6838	6744	6744
query52	100	92	91	91
query53	360	281	291	281
query54	939	432	430	430
query55	76	70	71	70
query56	293	261	268	261
query57	1136	1011	1060	1011
query58	245	243	260	243
query59	3333	3075	3186	3075
query60	290	266	264	264
query61	91	87	90	87
query62	627	444	431	431
query63	329	295	293	293
query64	8814	2262	1798	1798
query65	3142	3077	3093	3077
query66	753	331	323	323
query67	15588	15183	14939	14939
query68	4476	541	542	541
query69	623	422	367	367
query70	1172	1129	1153	1129
query71	466	276	275	275
query72	6940	5364	6174	5364
query73	766	332	325	325
query74	5834	5547	5528	5528
query75	3428	2670	2700	2670
query76	3055	914	959	914
query77	641	316	294	294
query78	10481	10003	9693	9693
query79	2233	516	519	516
query80	1228	465	457	457
query81	611	217	217	217
query82	753	104	98	98
query83	273	165	166	165
query84	249	87	83	83
query85	1731	294	270	270
query86	491	279	305	279
query87	3329	3095	3118	3095
query88	3913	2370	2347	2347
query89	474	370	380	370
query90	1740	187	187	187
query91	126	98	97	97
query92	65	48	50	48
query93	2320	518	508	508
query94	1059	188	179	179
query95	397	312	304	304
query96	583	265	269	265
query97	3256	3041	3013	3013
query98	217	206	200	200
query99	1195	844	836	836
Total cold run time: 272347 ms
Total hot run time: 173828 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit be4962e0ebfaff3806e0114f52f3bdb91cfa8084, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.04
query4	1.66	0.08	0.07
query5	0.49	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.48	0.50
query10	0.53	0.55	0.52
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.75	0.77	0.79
query15	0.83	0.81	0.81
query16	0.35	0.38	0.37
query17	0.98	0.96	0.97
query18	0.24	0.23	0.26
query19	1.77	1.76	1.72
query20	0.01	0.00	0.00
query21	15.42	0.66	0.66
query22	4.98	6.66	1.68
query23	18.22	1.35	1.28
query24	2.19	0.22	0.22
query25	0.16	0.08	0.08
query26	0.27	0.17	0.18
query27	0.07	0.08	0.08
query28	13.18	1.01	0.99
query29	12.63	3.33	3.23
query30	0.27	0.06	0.05
query31	2.88	0.39	0.39
query32	3.27	0.47	0.46
query33	2.89	2.89	2.95
query34	17.07	4.46	4.45
query35	4.48	4.53	4.64
query36	0.65	0.48	0.47
query37	0.19	0.15	0.17
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.18	0.15	0.15
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.99 s
Total hot run time: 30.36 s

@yujun777
Copy link
Collaborator Author

run p0

@yujun777
Copy link
Collaborator Author

run external

dataroaring
dataroaring approved these changes Jun 26, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 26, 2024
@dataroaring dataroaring merged commit cfb8788 into apache:master Jun 26, 2024
25 of 27 checks passed
@yujun777 yujun777 mentioned this pull request Jun 27, 2024
Merged
yujun777 added a commit to yujun777/doris that referenced this pull request Jun 27, 2024
@yujun777
[improvement](clone) dead be will abort sched task (apache#36795)
403b6ec 
When be is down, its related clone task need to abort. Otherwise this
task need to wait until timeout.
@yujun777 yujun777 mentioned this pull request Jun 27, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jun 27, 2024
@yujun777
[improvement](clone) dead be will abort sched task #36795 (#36897)
f80750f 
cherry pick from #36795
dataroaring pushed a commit that referenced this pull request Jun 27, 2024
@yujun777
[improvement](clone) dead be will abort sched task #36795 (#36895)
02b9d7a 
cherry pick from #36795
dataroaring pushed a commit that referenced this pull request Jun 28, 2024
@yujun777 @dataroaring
[improvement](clone) dead be will abort sched task (#36795)
49df269 
When be is down, its related clone task need to abort. Otherwise this
task need to wait until timeout.
@xiaokang xiaokang mentioned this pull request Jul 14, 2024
Closed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@yujun777
[improvement](clone) dead be will abort sched task apache#36795 (apac…
be0832c 
…he#36895)

cherry pick from apache#36795
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@deardeng deardeng deardeng approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.13-merged dev/2.1.5-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yujun777 @doris-robot @deardeng @dataroaring