Skip to content

[queryretry](errormsg) should not send retry request to BE again if the query is timeout#33984

Merged
yiguolei merged 2 commits intoapache:masterfrom
yiguolei:fix_msg
Apr 24, 2024
Merged

[queryretry](errormsg) should not send retry request to BE again if the query is timeout#33984
yiguolei merged 2 commits intoapache:masterfrom
yiguolei:fix_msg

Conversation

@yiguolei
Copy link
Contributor

Proposed changes

If retry, then FE will send fragments to BE, but there is a timeout checker and will cancel the timeout query. But BE will run the fragment, it is useless and will cost CPU.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei
Copy link
Contributor Author

run buildall

Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38857 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ce4c857ff1d27b0d0bbaeccd99cdfffb35d9ce4e, data reload: false

------ Round 1 ----------------------------------
q1	17615	4631	4288	4288
q2	2023	191	193	191
q3	10661	1191	1214	1191
q4	10643	817	742	742
q5	7503	2676	2670	2670
q6	212	132	135	132
q7	1030	615	587	587
q8	9245	2127	2074	2074
q9	7783	6691	6548	6548
q10	8661	3572	3501	3501
q11	434	233	235	233
q12	444	225	218	218
q13	18705	2931	2952	2931
q14	276	236	233	233
q15	534	488	495	488
q16	528	396	378	378
q17	972	621	714	621
q18	7530	6914	6931	6914
q19	6989	1542	1494	1494
q20	633	309	299	299
q21	3411	2832	2816	2816
q22	377	308	316	308
Total cold run time: 116209 ms
Total hot run time: 38857 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4318	4197	4217	4197
q2	363	267	257	257
q3	2962	2769	2752	2752
q4	1863	1639	1578	1578
q5	5333	5354	5277	5277
q6	207	125	126	125
q7	2224	1878	1875	1875
q8	3213	3349	3366	3349
q9	8593	8573	8563	8563
q10	3867	3755	3705	3705
q11	587	477	471	471
q12	749	554	573	554
q13	16597	2979	2968	2968
q14	293	268	285	268
q15	512	475	465	465
q16	486	427	434	427
q17	1816	1471	1468	1468
q18	7690	7469	7415	7415
q19	1625	1586	1515	1515
q20	1970	1775	1749	1749
q21	5086	4802	4793	4793
q22	547	479	466	466
Total cold run time: 70901 ms
Total hot run time: 54237 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185751 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ce4c857ff1d27b0d0bbaeccd99cdfffb35d9ce4e, data reload: false

query1	908	382	357	357
query2	6467	2585	2423	2423
query3	6656	215	216	215
query4	24833	21433	21305	21305
query5	4160	415	410	410
query6	261	176	170	170
query7	4581	286	283	283
query8	245	200	190	190
query9	8607	2280	2256	2256
query10	577	247	243	243
query11	14794	14312	14141	14141
query12	151	93	88	88
query13	1641	373	369	369
query14	10175	7788	7926	7788
query15	258	186	183	183
query16	8156	265	256	256
query17	1927	563	544	544
query18	2105	275	267	267
query19	345	146	156	146
query20	94	87	84	84
query21	197	119	121	119
query22	5038	4835	4802	4802
query23	33849	33312	33017	33017
query24	11889	2962	2940	2940
query25	652	364	385	364
query26	1760	151	150	150
query27	3024	312	317	312
query28	7641	1956	1932	1932
query29	1012	599	592	592
query30	307	166	170	166
query31	990	717	729	717
query32	83	53	52	52
query33	748	243	240	240
query34	1133	466	471	466
query35	840	681	683	681
query36	1047	904	873	873
query37	281	69	69	69
query38	3401	3179	3172	3172
query39	1590	1517	1566	1517
query40	273	122	123	122
query41	45	43	44	43
query42	108	101	121	101
query43	593	523	521	521
query44	1168	722	739	722
query45	273	260	274	260
query46	1081	700	735	700
query47	1943	1843	1865	1843
query48	365	290	294	290
query49	1197	396	382	382
query50	760	375	385	375
query51	6727	6601	6591	6591
query52	105	89	87	87
query53	349	280	277	277
query54	311	229	235	229
query55	79	72	73	72
query56	236	214	227	214
query57	1212	1120	1126	1120
query58	229	194	192	192
query59	3301	3322	3314	3314
query60	258	224	231	224
query61	91	90	90	90
query62	654	444	426	426
query63	300	272	275	272
query64	6198	4089	4113	4089
query65	3131	3035	3048	3035
query66	1386	354	322	322
query67	15221	14883	14903	14883
query68	5098	530	530	530
query69	531	305	300	300
query70	1236	1202	1177	1177
query71	1408	1264	1272	1264
query72	6438	2621	2437	2437
query73	717	321	326	321
query74	6889	6434	6495	6434
query75	3682	2643	2644	2643
query76	3531	1023	965	965
query77	623	260	259	259
query78	10850	10175	10468	10175
query79	3072	527	519	519
query80	1901	462	492	462
query81	540	240	245	240
query82	842	101	97	97
query83	356	178	182	178
query84	268	88	93	88
query85	1614	259	262	259
query86	515	314	305	305
query87	3474	3262	3316	3262
query88	4466	2398	2393	2393
query89	497	370	375	370
query90	1978	186	181	181
query91	129	98	100	98
query92	60	48	49	48
query93	4777	508	495	495
query94	1210	180	178	178
query95	1098	1097	1102	1097
query96	612	264	263	263
query97	3150	2964	2946	2946
query98	256	227	215	215
query99	1339	878	857	857
Total cold run time: 293174 ms
Total hot run time: 185751 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ce4c857ff1d27b0d0bbaeccd99cdfffb35d9ce4e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.65	0.10	0.08
query5	0.53	0.51	0.51
query6	1.45	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.56	0.56	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.76	0.78	0.77
query15	0.83	0.81	0.81
query16	0.36	0.34	0.38
query17	0.93	0.94	0.94
query18	0.20	0.24	0.24
query19	1.82	1.68	1.79
query20	0.01	0.01	0.02
query21	15.43	0.65	0.65
query22	4.88	6.63	2.20
query23	18.31	1.40	1.30
query24	1.68	0.24	0.30
query25	0.14	0.08	0.08
query26	0.27	0.15	0.16
query27	0.08	0.08	0.08
query28	13.22	1.01	0.99
query29	12.74	3.27	3.26
query30	0.26	0.07	0.05
query31	2.84	0.41	0.37
query32	3.28	0.46	0.46
query33	2.80	2.86	2.80
query34	17.17	4.39	4.46
query35	4.46	4.48	4.49
query36	0.65	0.45	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.18	0.14	0.14
query41	0.09	0.04	0.04
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.94 s
Total hot run time: 30.66 s

Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

ClickBench: Total hot run time: 31.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0cfffff8be287f684dec960e1b5284a2d5e6ace9, data reload: false

query1	0.03	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.50	0.49	0.48
query6	1.25	0.88	0.81
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.49	0.48	0.43
query10	0.51	0.50	0.49
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.64	0.63	0.65
query14	0.96	1.02	0.96
query15	0.85	0.84	0.86
query16	0.38	0.38	0.36
query17	0.97	1.05	0.98
query18	0.22	0.22	0.23
query19	1.86	1.84	1.84
query20	0.01	0.01	0.01
query21	15.41	0.66	0.65
query22	4.78	6.56	2.10
query23	18.34	1.32	1.28
query24	1.95	0.30	0.25
query25	0.15	0.10	0.10
query26	0.26	0.18	0.17
query27	0.10	0.10	0.09
query28	13.32	1.03	1.01
query29	12.74	3.39	3.37
query30	0.26	0.07	0.05
query31	2.84	0.40	0.40
query32	3.25	0.49	0.49
query33	2.76	2.85	2.78
query34	17.39	4.54	4.56
query35	4.53	4.43	4.63
query36	0.65	0.47	0.47
query37	0.21	0.18	0.18
query38	0.19	0.20	0.18
query39	0.05	0.04	0.04
query40	0.18	0.16	0.14
query41	0.11	0.06	0.06
query42	0.06	0.06	0.06
query43	0.05	0.05	0.05
Total cold run time: 110.58 s
Total hot run time: 31.38 s

Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 24, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@yiguolei yiguolei merged commit 92e2f1c into apache:master Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants