Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature](show) show backend config using ShowStmt #36525

Merged
merged 6 commits into from
Aug 16, 2024

Conversation

DarvenDuan
Copy link
Contributor

@DarvenDuan DarvenDuan commented Jun 19, 2024

Proposed changes

Issue Number: close #36445

Currently, Doris only supports accessing backend configurations via HTTP APIs. To enhance this functionality, I have added some syntax extensions to allow displaying backend configurations using SQL commands.

The SQL syntax for displaying backend configurations is similar to that for displaying frontend configurations:

  1. Display backend configurations matching a pattern:
    This statement will display all active backend configurations that match the specified pattern.
show backend config like ${pattern}
  1. Display backend configurations for a specific BE using the backend ID
show backend config from ${beId}
  1. Use both a pattern and a backend ID:
show backend config like ${pattern} from ${backendID}

The output of these statements consists of 6 columns: BE ID, BE host IP, configuration key, configuration value, configuration type, and whether it is mutable or not.

+-----------+---------------+---------+-------+---------+-----------+
| BackendId | Host          | Key     | Value | Type    | IsMutable |
+-----------+---------------+---------+-------+---------+-----------+
| 10001     | xx.xx.xxx.xxx | be_port | 9060  | int32_t | false     |
+-----------+---------------+---------+-------+---------+-----------+

This provides an additional way to access backend configurations within Doris.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@DarvenDuan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39959 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b6dcf47bd77a7c3073bdfa17bb28a738a971a8c7, data reload: false

------ Round 1 ----------------------------------
q1	17591	4409	4255	4255
q2	2033	189	209	189
q3	10517	1201	1151	1151
q4	10186	803	786	786
q5	7468	2679	2727	2679
q6	222	134	146	134
q7	955	601	608	601
q8	9219	2084	2075	2075
q9	9134	6545	6482	6482
q10	9053	3731	3720	3720
q11	435	242	241	241
q12	446	243	227	227
q13	18955	2942	2990	2942
q14	272	226	219	219
q15	515	476	475	475
q16	527	370	396	370
q17	973	697	709	697
q18	7996	7354	7456	7354
q19	4651	1453	1519	1453
q20	645	323	336	323
q21	5014	3289	3265	3265
q22	390	328	321	321
Total cold run time: 117197 ms
Total hot run time: 39959 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4374	4205	4253	4205
q2	374	275	263	263
q3	2969	2940	2839	2839
q4	1998	1696	1786	1696
q5	5595	5520	5451	5451
q6	223	137	129	129
q7	2229	1875	1853	1853
q8	3275	3409	3429	3409
q9	8713	8697	8817	8697
q10	4101	3905	3744	3744
q11	608	500	507	500
q12	821	635	647	635
q13	17090	3128	3164	3128
q14	307	281	284	281
q15	521	472	499	472
q16	509	436	440	436
q17	1814	1511	1532	1511
q18	8082	7899	7728	7728
q19	1829	1584	1649	1584
q20	3042	1899	1872	1872
q21	5208	5051	4817	4817
q22	628	538	577	538
Total cold run time: 74310 ms
Total hot run time: 55788 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174356 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b6dcf47bd77a7c3073bdfa17bb28a738a971a8c7, data reload: false

query1	924	388	380	380
query2	6434	2594	2278	2278
query3	6635	211	205	205
query4	18831	17594	17326	17326
query5	3657	483	486	483
query6	236	166	158	158
query7	4579	297	286	286
query8	304	278	271	271
query9	8596	2399	2394	2394
query10	560	320	270	270
query11	10666	10158	10117	10117
query12	119	94	88	88
query13	1629	357	355	355
query14	10150	7128	7601	7128
query15	241	185	183	183
query16	7772	262	256	256
query17	1721	532	505	505
query18	1959	265	268	265
query19	194	147	146	146
query20	92	87	82	82
query21	210	132	123	123
query22	4472	4302	4210	4210
query23	33916	33596	33608	33596
query24	10744	2941	2834	2834
query25	594	372	363	363
query26	716	155	155	155
query27	2242	335	322	322
query28	6328	2115	2117	2115
query29	897	646	647	646
query30	249	155	156	155
query31	993	780	766	766
query32	97	57	57	57
query33	648	299	299	299
query34	899	487	481	481
query35	773	658	649	649
query36	1165	963	974	963
query37	149	82	72	72
query38	3010	2821	2836	2821
query39	894	826	807	807
query40	205	129	125	125
query41	55	58	54	54
query42	118	97	102	97
query43	576	553	547	547
query44	1065	722	736	722
query45	191	166	165	165
query46	1078	715	748	715
query47	1893	1842	1801	1801
query48	367	299	295	295
query49	851	414	420	414
query50	757	390	390	390
query51	6818	6719	6635	6635
query52	107	94	90	90
query53	363	291	289	289
query54	915	436	431	431
query55	74	72	75	72
query56	297	279	272	272
query57	1138	1039	1073	1039
query58	268	253	255	253
query59	3502	3246	3143	3143
query60	299	280	291	280
query61	115	112	109	109
query62	613	440	459	440
query63	315	286	291	286
query64	8737	2348	1838	1838
query65	3259	3126	3094	3094
query66	771	335	340	335
query67	15593	14945	15357	14945
query68	7974	529	540	529
query69	731	458	411	411
query70	1188	1052	1141	1052
query71	558	281	279	279
query72	9258	5595	5658	5595
query73	839	322	325	322
query74	6063	5594	5616	5594
query75	4908	2686	2668	2668
query76	4289	965	881	881
query77	783	289	292	289
query78	10577	9823	9718	9718
query79	8675	506	515	506
query80	1070	461	454	454
query81	558	222	223	222
query82	750	101	101	101
query83	356	165	161	161
query84	276	86	83	83
query85	1257	280	316	280
query86	412	284	299	284
query87	3315	3129	3104	3104
query88	4987	2376	2340	2340
query89	532	381	382	381
query90	1974	183	189	183
query91	125	100	95	95
query92	59	46	49	46
query93	6882	505	496	496
query94	1204	179	180	179
query95	404	315	308	308
query96	601	262	266	262
query97	3216	3020	3022	3020
query98	220	203	194	194
query99	1166	844	833	833
Total cold run time: 289842 ms
Total hot run time: 174356 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b6dcf47bd77a7c3073bdfa17bb28a738a971a8c7, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.07	0.08
query5	0.50	0.47	0.50
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.56	0.55	0.53
query11	0.16	0.12	0.12
query12	0.14	0.12	0.12
query13	0.59	0.59	0.61
query14	0.76	0.77	0.77
query15	0.84	0.81	0.82
query16	0.37	0.36	0.34
query17	0.97	1.02	1.00
query18	0.21	0.22	0.22
query19	1.89	1.76	1.83
query20	0.02	0.01	0.01
query21	15.42	0.66	0.65
query22	4.69	6.43	2.16
query23	18.26	1.37	1.32
query24	2.04	0.25	0.22
query25	0.17	0.09	0.08
query26	0.28	0.18	0.18
query27	0.09	0.08	0.09
query28	13.27	1.02	1.01
query29	12.62	3.29	3.28
query30	0.29	0.08	0.07
query31	2.83	0.40	0.38
query32	3.28	0.48	0.47
query33	2.91	2.89	2.87
query34	17.07	4.47	4.46
query35	4.50	4.49	4.47
query36	0.64	0.48	0.47
query37	0.18	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.16	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.03	0.04
Total cold run time: 109.9 s
Total hot run time: 30.98 s

caiconghui
caiconghui previously approved these changes Jun 20, 2024
Copy link
Contributor

@caiconghui caiconghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@DarvenDuan
Copy link
Contributor Author

run p0

Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this feature(show config) supported in Nereids?

@DarvenDuan
Copy link
Contributor Author

Is this feature(show config) supported in Nereids?

ShowStmt does not differentiate between old and new optimizers,so it works with both.

@zclllyybb
Copy link
Contributor

Is this feature(show config) supported in Nereids?

ShowStmt does not differentiate between old and new optimizers,so it works with both.

should what fe/fe-core/src/main/cup/sql_parser.cup implements also be implemented in nereids/DorisParser.g4?
AFAIK, now it can't be parsed by nereids? then when we deprecate legacy planner in near future, should we also imp it in nereids?

@DarvenDuan
Copy link
Contributor Author

Is this feature(show config) supported in Nereids?

ShowStmt does not differentiate between old and new optimizers,so it works with both.

should what fe/fe-core/src/main/cup/sql_parser.cup implements also be implemented in nereids/DorisParser.g4? AFAIK, now it can't be parsed by nereids? then when we deprecate legacy planner in near future, should we also imp it in nereids?

The NereidsParser can't parse all ShowStmt (e.g., show frontends stmt) and it will rollback to old SqlParser now, is there a plan to deprecate the old SqlParser and remove sql_parser.cup when we deprecate legacy planner in near future?

@zclllyybb
Copy link
Contributor

Is this feature(show config) supported in Nereids?

ShowStmt does not differentiate between old and new optimizers,so it works with both.

should what fe/fe-core/src/main/cup/sql_parser.cup implements also be implemented in nereids/DorisParser.g4? AFAIK, now it can't be parsed by nereids? then when we deprecate legacy planner in near future, should we also imp it in nereids?

The NereidsParser can't parse all ShowStmt (e.g., show frontends stmt) and it will rollback to old SqlParser now, is there a plan to deprecate the old SqlParser and remove sql_parser.cup when we deprecate legacy planner in near future?

I think so. But if all ShowStmt are not supported now for nereids, it's also good for this pr. The optimizer group may look forward for nereids compatible in the future

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 26, 2024
@DarvenDuan DarvenDuan closed this Jun 27, 2024
@DarvenDuan DarvenDuan reopened this Jun 27, 2024
@DarvenDuan
Copy link
Contributor Author

Is this feature(show config) supported in Nereids?

ShowStmt does not differentiate between old and new optimizers,so it works with both.

should what fe/fe-core/src/main/cup/sql_parser.cup implements also be implemented in nereids/DorisParser.g4? AFAIK, now it can't be parsed by nereids? then when we deprecate legacy planner in near future, should we also imp it in nereids?

The NereidsParser can't parse all ShowStmt (e.g., show frontends stmt) and it will rollback to old SqlParser now, is there a plan to deprecate the old SqlParser and remove sql_parser.cup when we deprecate legacy planner in near future?

I think so. But if all ShowStmt are not supported now for nereids, it's also good for this pr. The optimizer group may look forward for nereids compatible in the future

I have optimized the implementation and now it supports nereids.

@DarvenDuan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39823 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 75b56023f67ed47c252cd01cd1f48ed94b6ff9c0, data reload: false

------ Round 1 ----------------------------------
q1	18192	4608	4270	4270
q2	2028	190	196	190
q3	10467	1233	1122	1122
q4	10193	790	793	790
q5	7487	2647	2628	2628
q6	219	137	135	135
q7	945	597	610	597
q8	9228	2075	2072	2072
q9	8934	6488	6469	6469
q10	8881	3695	3771	3695
q11	456	243	229	229
q12	402	227	230	227
q13	18993	2977	2990	2977
q14	266	220	213	213
q15	516	484	487	484
q16	520	384	374	374
q17	959	663	750	663
q18	8057	7490	7517	7490
q19	5500	1426	1513	1426
q20	649	322	333	322
q21	4855	3117	3208	3117
q22	401	339	333	333
Total cold run time: 118148 ms
Total hot run time: 39823 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4205	4253	4205
q2	377	261	270	261
q3	2952	2745	2745	2745
q4	1812	1581	1589	1581
q5	5221	5265	5235	5235
q6	216	126	129	126
q7	2080	1712	1722	1712
q8	3203	3296	3299	3296
q9	8350	8292	8327	8292
q10	3853	3741	3699	3699
q11	589	494	509	494
q12	777	626	629	626
q13	17317	2974	2994	2974
q14	282	276	272	272
q15	530	468	478	468
q16	470	421	428	421
q17	1754	1491	1467	1467
q18	7732	7581	7301	7301
q19	1730	1620	1657	1620
q20	1983	1765	1779	1765
q21	4708	4648	4711	4648
q22	622	533	561	533
Total cold run time: 70886 ms
Total hot run time: 53741 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173010 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 75b56023f67ed47c252cd01cd1f48ed94b6ff9c0, data reload: false

query1	912	381	378	378
query2	6488	2558	2373	2373
query3	6646	204	210	204
query4	18738	17551	17242	17242
query5	4185	493	477	477
query6	254	171	170	170
query7	4593	299	309	299
query8	299	305	306	305
query9	8487	2441	2411	2411
query10	594	288	292	288
query11	10614	10032	10006	10006
query12	135	84	85	84
query13	1646	390	377	377
query14	9751	7901	6306	6306
query15	220	183	186	183
query16	7916	281	272	272
query17	1879	566	538	538
query18	1791	283	279	279
query19	199	153	156	153
query20	90	82	80	80
query21	214	134	130	130
query22	4382	4013	3976	3976
query23	33932	33395	33269	33269
query24	12032	2802	2880	2802
query25	672	375	387	375
query26	1760	154	154	154
query27	2953	318	322	318
query28	7328	2085	2091	2085
query29	1071	640	636	636
query30	283	154	156	154
query31	970	749	747	747
query32	94	55	62	55
query33	796	309	296	296
query34	929	474	469	469
query35	752	618	614	614
query36	1127	926	941	926
query37	210	69	71	69
query38	2872	2724	2745	2724
query39	877	824	806	806
query40	273	132	138	132
query41	59	53	54	53
query42	128	97	101	97
query43	590	576	541	541
query44	1178	739	734	734
query45	197	166	163	163
query46	1068	711	697	697
query47	1856	1782	1775	1775
query48	366	300	311	300
query49	1194	439	429	429
query50	772	381	385	381
query51	6881	6790	6808	6790
query52	104	93	94	93
query53	365	295	297	295
query54	1018	459	449	449
query55	77	76	75	75
query56	308	288	282	282
query57	1185	1066	1034	1034
query58	264	252	246	246
query59	3401	3277	3383	3277
query60	296	265	285	265
query61	113	97	95	95
query62	667	460	427	427
query63	314	294	301	294
query64	9816	2233	1800	1800
query65	3200	3088	3112	3088
query66	1342	337	335	335
query67	15727	14855	14974	14855
query68	8433	558	577	558
query69	674	478	391	391
query70	1098	1166	1158	1158
query71	527	278	280	278
query72	8579	5934	5440	5440
query73	876	324	326	324
query74	6094	5492	5525	5492
query75	5050	2673	2640	2640
query76	4884	965	1012	965
query77	812	315	297	297
query78	10579	9845	9777	9777
query79	7664	525	535	525
query80	2426	480	480	480
query81	564	221	229	221
query82	772	107	104	104
query83	317	167	167	167
query84	268	86	82	82
query85	1404	297	277	277
query86	419	314	301	301
query87	3275	3096	3087	3087
query88	4513	2381	2353	2353
query89	510	404	402	402
query90	1951	191	194	191
query91	133	104	100	100
query92	64	54	50	50
query93	5593	531	514	514
query94	1185	196	186	186
query95	404	315	325	315
query96	611	272	268	268
query97	3202	3021	3083	3021
query98	217	212	196	196
query99	1235	856	818	818
Total cold run time: 295783 ms
Total hot run time: 173010 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 75b56023f67ed47c252cd01cd1f48ed94b6ff9c0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.49
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.54	0.52	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.78	0.77
query15	0.85	0.81	0.82
query16	0.37	0.36	0.38
query17	1.04	0.98	1.04
query18	0.23	0.23	0.24
query19	1.80	1.75	1.81
query20	0.02	0.01	0.01
query21	15.46	0.76	0.67
query22	3.97	7.27	2.36
query23	18.28	1.41	1.25
query24	2.06	0.25	0.22
query25	0.16	0.09	0.09
query26	0.27	0.17	0.18
query27	0.08	0.08	0.08
query28	13.28	1.01	1.00
query29	12.60	3.34	3.30
query30	0.25	0.06	0.05
query31	2.89	0.40	0.39
query32	3.27	0.48	0.47
query33	2.88	2.86	2.86
query34	17.24	4.38	4.39
query35	4.45	4.57	4.45
query36	0.65	0.47	0.46
query37	0.17	0.16	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.04
query40	0.18	0.15	0.15
query41	0.09	0.04	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.34 s
Total hot run time: 31 s

@doris-robot
Copy link

TPC-DS: Total hot run time: 174697 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6163b54861223f888fef012c34437a43df5bae5, data reload: false

query1	926	385	370	370
query2	6440	2519	2416	2416
query3	6625	214	215	214
query4	20983	17694	17480	17480
query5	3564	471	462	462
query6	274	185	164	164
query7	4595	292	290	290
query8	321	300	303	300
query9	8770	2346	2317	2317
query10	574	300	281	281
query11	10779	10344	10204	10204
query12	117	86	86	86
query13	1670	367	366	366
query14	10107	7864	7426	7426
query15	223	197	188	188
query16	7898	277	308	277
query17	1854	536	523	523
query18	1989	276	268	268
query19	199	151	150	150
query20	94	81	85	81
query21	206	135	125	125
query22	4462	4368	4167	4167
query23	33969	33854	33710	33710
query24	11147	2843	2919	2843
query25	599	375	371	371
query26	740	152	158	152
query27	2278	327	334	327
query28	6559	2087	2075	2075
query29	880	640	609	609
query30	260	165	155	155
query31	972	744	757	744
query32	91	56	52	52
query33	768	290	304	290
query34	926	480	491	480
query35	751	678	648	648
query36	1126	981	983	981
query37	142	75	72	72
query38	2984	2814	2788	2788
query39	849	774	789	774
query40	214	128	124	124
query41	55	51	55	51
query42	128	103	104	103
query43	587	543	572	543
query44	1218	744	724	724
query45	198	164	168	164
query46	1084	747	688	688
query47	1881	1817	1802	1802
query48	369	308	299	299
query49	857	396	420	396
query50	768	388	386	386
query51	6958	6772	6905	6772
query52	108	91	96	91
query53	365	297	307	297
query54	912	441	436	436
query55	76	75	73	73
query56	275	263	259	259
query57	1137	1090	1092	1090
query58	250	301	257	257
query59	3427	3255	3183	3183
query60	296	271	274	271
query61	91	88	89	88
query62	591	450	428	428
query63	315	285	286	285
query64	8578	2257	1749	1749
query65	3167	3086	3077	3077
query66	746	326	326	326
query67	15543	15133	14908	14908
query68	4484	537	532	532
query69	520	409	388	388
query70	1128	1138	1063	1063
query71	390	270	272	270
query72	6984	4873	5392	4873
query73	767	333	324	324
query74	5879	5612	5566	5566
query75	3346	2680	2660	2660
query76	2303	984	916	916
query77	451	303	294	294
query78	10441	10107	9893	9893
query79	2566	511	520	511
query80	2002	467	456	456
query81	597	217	223	217
query82	791	104	102	102
query83	300	173	171	171
query84	257	91	85	85
query85	1524	277	292	277
query86	462	339	341	339
query87	3327	3150	3168	3150
query88	4293	2506	2430	2430
query89	466	393	392	392
query90	1697	189	190	189
query91	125	106	100	100
query92	60	51	51	51
query93	1893	489	490	489
query94	1054	187	184	184
query95	405	321	311	311
query96	587	266	266	266
query97	3267	3107	3085	3085
query98	212	193	201	193
query99	1193	824	833	824
Total cold run time: 271676 ms
Total hot run time: 174697 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6163b54861223f888fef012c34437a43df5bae5, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.08	0.07
query5	0.52	0.50	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.55	0.48	0.48
query10	0.55	0.54	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.61
query14	0.79	0.77	0.79
query15	0.83	0.80	0.81
query16	0.36	0.35	0.36
query17	1.03	1.04	1.05
query18	0.22	0.24	0.25
query19	1.72	1.67	1.79
query20	0.01	0.00	0.01
query21	15.44	0.76	0.66
query22	4.38	5.92	2.69
query23	18.30	1.36	1.31
query24	2.08	0.24	0.23
query25	0.17	0.09	0.09
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.18	1.02	0.99
query29	12.57	3.25	3.26
query30	0.26	0.06	0.05
query31	2.87	0.39	0.39
query32	3.27	0.47	0.47
query33	2.83	2.85	2.92
query34	17.18	4.48	4.43
query35	4.46	4.50	4.48
query36	0.65	0.46	0.48
query37	0.18	0.16	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.18	0.14	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.44 s
Total hot run time: 31.37 s

@DarvenDuan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38551 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80b39e9082df408a6db8bb249f8a97420b44ed81, data reload: false

------ Round 1 ----------------------------------
q1	18289	4471	4442	4442
q2	2051	224	212	212
q3	10394	1298	1099	1099
q4	10167	794	765	765
q5	7779	2842	2782	2782
q6	271	156	160	156
q7	990	649	660	649
q8	9396	2071	2082	2071
q9	7360	6579	6565	6565
q10	7079	2293	2284	2284
q11	476	275	280	275
q12	433	263	257	257
q13	17802	3006	2992	2992
q14	295	264	255	255
q15	567	521	525	521
q16	515	419	408	408
q17	987	712	719	712
q18	7585	6825	6787	6787
q19	6302	1083	1074	1074
q20	760	369	374	369
q21	4034	3064	2834	2834
q22	1162	1045	1042	1042
Total cold run time: 114694 ms
Total hot run time: 38551 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4881	4357	4309	4309
q2	426	335	312	312
q3	2881	2667	2631	2631
q4	1960	1716	1572	1572
q5	5397	5394	5434	5394
q6	250	152	151	151
q7	2116	1715	1705	1705
q8	3232	3407	3349	3349
q9	8475	8350	8368	8350
q10	3399	3185	3172	3172
q11	617	517	503	503
q12	850	617	642	617
q13	17495	3020	2987	2987
q14	319	309	289	289
q15	560	504	506	504
q16	487	453	470	453
q17	1783	1496	1491	1491
q18	8061	7584	7431	7431
q19	9850	1569	1715	1569
q20	3167	1820	1812	1812
q21	7786	5226	5178	5178
q22	1252	1074	1018	1018
Total cold run time: 85244 ms
Total hot run time: 54797 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191881 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80b39e9082df408a6db8bb249f8a97420b44ed81, data reload: false

query1	998	426	417	417
query2	6768	2011	1949	1949
query3	6668	244	256	244
query4	34214	23758	23211	23211
query5	4405	645	642	642
query6	296	193	189	189
query7	4631	314	316	314
query8	442	423	428	423
query9	8651	2571	2531	2531
query10	513	349	345	345
query11	17953	15202	15166	15166
query12	174	123	121	121
query13	1678	424	410	410
query14	9560	7361	7304	7304
query15	277	191	198	191
query16	7761	511	534	511
query17	1635	592	583	583
query18	2056	335	338	335
query19	241	174	176	174
query20	145	137	137	137
query21	259	146	142	142
query22	4555	4126	4125	4125
query23	34296	33451	33990	33451
query24	11735	2965	3009	2965
query25	659	415	411	411
query26	1088	178	181	178
query27	2678	306	314	306
query28	7565	2165	2144	2144
query29	845	448	435	435
query30	336	192	190	190
query31	1052	818	848	818
query32	123	80	81	80
query33	815	374	346	346
query34	906	508	503	503
query35	883	789	773	773
query36	1101	978	946	946
query37	173	109	110	109
query38	3899	3819	3838	3819
query39	1536	1492	1470	1470
query40	245	155	155	155
query41	144	142	142	142
query42	136	117	120	117
query43	543	498	504	498
query44	1248	783	784	783
query45	222	199	199	199
query46	1139	806	772	772
query47	1918	1834	1865	1834
query48	410	334	336	334
query49	1209	595	593	593
query50	864	467	476	467
query51	6913	6850	6787	6787
query52	119	111	112	111
query53	301	233	234	233
query54	981	510	507	507
query55	91	90	90	90
query56	358	342	317	317
query57	1227	1124	1153	1124
query58	309	326	308	308
query59	3015	3105	2796	2796
query60	356	346	338	338
query61	176	145	150	145
query62	833	716	733	716
query63	260	228	237	228
query64	5405	2392	1900	1900
query65	3250	3180	3225	3180
query66	1080	691	686	686
query67	15563	15181	15038	15038
query68	8803	597	607	597
query69	740	433	354	354
query70	1248	1081	1108	1081
query71	559	323	323	323
query72	7485	2279	2091	2091
query73	1781	364	363	363
query74	9447	8824	8945	8824
query75	4689	2758	2757	2757
query76	4967	1016	1101	1016
query77	932	449	454	449
query78	9882	9141	9310	9141
query79	8075	562	563	562
query80	2381	623	619	619
query81	643	317	264	264
query82	329	162	157	157
query83	362	217	217	217
query84	299	99	101	99
query85	1083	354	351	351
query86	370	297	339	297
query87	4410	4099	4241	4099
query88	4224	2529	2504	2504
query89	519	332	333	332
query90	2094	240	234	234
query91	157	148	126	126
query92	95	77	76	76
query93	6038	552	548	548
query94	1213	330	339	330
query95	397	302	301	301
query96	627	293	287	287
query97	3220	3127	3112	3112
query98	257	226	226	226
query99	1581	1342	1338	1338
Total cold run time: 320792 ms
Total hot run time: 191881 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 80b39e9082df408a6db8bb249f8a97420b44ed81, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.65	0.09	0.10
query5	0.49	0.49	0.50
query6	1.13	0.75	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.55	0.50	0.50
query10	0.55	0.55	0.55
query11	0.16	0.12	0.12
query12	0.17	0.13	0.12
query13	0.61	0.62	0.60
query14	0.79	0.80	0.78
query15	0.85	0.82	0.83
query16	0.38	0.38	0.38
query17	1.03	1.03	1.01
query18	0.23	0.21	0.21
query19	1.88	1.77	1.70
query20	0.02	0.01	0.01
query21	15.41	0.87	0.68
query22	4.51	7.55	2.13
query23	18.25	1.38	1.28
query24	2.15	0.23	0.22
query25	0.15	0.08	0.09
query26	0.30	0.22	0.21
query27	0.46	0.22	0.23
query28	13.27	1.04	1.02
query29	12.62	3.35	3.29
query30	0.40	0.25	0.24
query31	2.80	0.40	0.40
query32	3.23	0.48	0.49
query33	2.95	2.95	2.98
query34	17.10	4.41	4.41
query35	4.50	4.54	4.43
query36	0.67	0.50	0.50
query37	0.21	0.18	0.18
query38	0.18	0.17	0.17
query39	0.07	0.06	0.06
query40	0.19	0.16	0.14
query41	0.12	0.07	0.07
query42	0.08	0.06	0.07
query43	0.07	0.06	0.06
Total cold run time: 110.62 s
Total hot run time: 31.57 s

@DarvenDuan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38478 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc288260b7fb3caee897da57fdea0a719f4171a0, data reload: false

------ Round 1 ----------------------------------
q1	18372	4489	4381	4381
q2	2631	214	218	214
q3	11292	1315	1099	1099
q4	10208	681	809	681
q5	7806	2885	2863	2863
q6	261	149	149	149
q7	995	641	641	641
q8	9380	2054	2067	2054
q9	7079	6530	6544	6530
q10	7097	2209	2288	2209
q11	483	264	265	264
q12	423	247	246	246
q13	17795	3048	3045	3045
q14	292	256	250	250
q15	570	510	516	510
q16	533	414	405	405
q17	979	620	783	620
q18	7474	6944	6870	6870
q19	6595	1073	1105	1073
q20	711	371	353	353
q21	3860	2985	3042	2985
q22	1149	1046	1036	1036
Total cold run time: 115985 ms
Total hot run time: 38478 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4585	4317	4255	4255
q2	413	297	298	297
q3	2862	2674	2648	2648
q4	1911	1595	1649	1595
q5	5365	5393	5403	5393
q6	248	150	151	150
q7	2118	1731	1768	1731
q8	3284	3373	3356	3356
q9	8450	8410	8421	8410
q10	3408	3187	3214	3187
q11	643	532	532	532
q12	814	650	641	641
q13	16587	3002	2992	2992
q14	320	290	287	287
q15	550	507	500	500
q16	496	446	446	446
q17	1798	1498	1462	1462
q18	7701	7699	7483	7483
q19	1701	1686	1604	1604
q20	2061	1847	1846	1846
q21	5161	5156	5119	5119
q22	1133	1045	1039	1039
Total cold run time: 71609 ms
Total hot run time: 54973 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190313 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc288260b7fb3caee897da57fdea0a719f4171a0, data reload: false

query1	982	403	388	388
query2	6740	1992	1925	1925
query3	6662	226	239	226
query4	34133	23534	23236	23236
query5	4368	646	648	646
query6	314	190	188	188
query7	4596	320	319	319
query8	409	372	365	365
query9	8686	2508	2478	2478
query10	493	329	320	320
query11	16789	15118	15175	15118
query12	177	122	125	122
query13	1701	429	417	417
query14	10114	7405	6726	6726
query15	351	186	183	183
query16	8197	530	497	497
query17	1753	594	592	592
query18	2181	338	333	333
query19	401	170	168	168
query20	142	135	132	132
query21	260	146	140	140
query22	4441	4011	3939	3939
query23	34369	33402	33320	33320
query24	11351	2965	2930	2930
query25	649	446	413	413
query26	1208	179	180	179
query27	2283	302	303	302
query28	7167	2134	2126	2126
query29	859	450	436	436
query30	330	183	180	180
query31	1052	821	840	821
query32	127	78	83	78
query33	819	376	339	339
query34	926	504	499	499
query35	892	785	807	785
query36	1123	960	970	960
query37	168	106	107	106
query38	4016	3985	3791	3791
query39	1514	1473	1479	1473
query40	245	158	156	156
query41	139	138	139	138
query42	133	114	120	114
query43	564	507	501	501
query44	1293	792	787	787
query45	223	198	191	191
query46	1133	786	766	766
query47	1922	1841	1835	1835
query48	426	361	350	350
query49	1207	590	623	590
query50	874	472	465	465
query51	6864	6807	6759	6759
query52	128	111	111	111
query53	302	225	233	225
query54	991	511	506	506
query55	90	90	91	90
query56	351	321	312	312
query57	1214	1132	1135	1132
query58	330	302	309	302
query59	3015	2785	2911	2785
query60	357	322	334	322
query61	146	145	150	145
query62	871	692	705	692
query63	263	228	233	228
query64	5274	2469	1887	1887
query65	3230	3188	3192	3188
query66	1164	679	678	678
query67	15804	14944	14910	14910
query68	5735	595	591	591
query69	716	446	358	358
query70	1255	1189	1197	1189
query71	539	335	318	318
query72	7728	2306	2157	2157
query73	831	358	355	355
query74	9498	8892	8836	8836
query75	5096	2706	2746	2706
query76	4280	1042	971	971
query77	928	464	449	449
query78	10122	9193	9015	9015
query79	8141	556	555	555
query80	2370	607	602	602
query81	640	258	259	258
query82	801	165	157	157
query83	358	214	211	211
query84	287	99	99	99
query85	1539	354	346	346
query86	419	319	336	319
query87	4423	4156	4274	4156
query88	5042	2517	2547	2517
query89	466	330	319	319
query90	2051	238	232	232
query91	158	126	125	125
query92	85	75	73	73
query93	5786	558	551	551
query94	785	319	324	319
query95	396	304	300	300
query96	625	300	283	283
query97	3272	3051	3065	3051
query98	247	231	234	231
query99	1622	1314	1303	1303
Total cold run time: 317537 ms
Total hot run time: 190313 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc288260b7fb3caee897da57fdea0a719f4171a0, data reload: false

query1	0.05	0.05	0.04
query2	0.09	0.04	0.04
query3	0.24	0.05	0.05
query4	1.66	0.07	0.08
query5	0.51	0.49	0.51
query6	1.14	0.72	0.72
query7	0.03	0.01	0.02
query8	0.06	0.06	0.05
query9	0.55	0.50	0.51
query10	0.56	0.56	0.55
query11	0.16	0.12	0.12
query12	0.17	0.13	0.14
query13	0.62	0.63	0.60
query14	0.78	0.79	0.78
query15	0.86	0.83	0.83
query16	0.37	0.39	0.38
query17	0.99	0.98	1.05
query18	0.24	0.23	0.23
query19	1.82	1.84	1.86
query20	0.02	0.02	0.01
query21	15.39	0.85	0.65
query22	4.60	6.12	1.73
query23	18.29	1.47	1.31
query24	2.16	0.23	0.22
query25	0.15	0.08	0.09
query26	0.31	0.22	0.22
query27	0.47	0.23	0.23
query28	13.26	1.04	1.01
query29	12.65	3.35	3.36
query30	0.38	0.19	0.19
query31	2.80	0.40	0.40
query32	3.23	0.48	0.49
query33	2.96	2.99	2.92
query34	16.90	4.41	4.40
query35	4.44	4.44	4.39
query36	0.68	0.49	0.49
query37	0.20	0.17	0.18
query38	0.17	0.17	0.17
query39	0.07	0.06	0.05
query40	0.18	0.16	0.16
query41	0.11	0.08	0.07
query42	0.08	0.07	0.06
query43	0.07	0.08	0.07
Total cold run time: 110.47 s
Total hot run time: 31.24 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 16, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@GoGoWen GoGoWen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morrySnow morrySnow merged commit ed560bb into apache:master Aug 16, 2024
29 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Sep 19, 2024
Currently, Doris only supports accessing backend configurations via HTTP
APIs. To enhance this functionality, I have added some syntax extensions
to allow displaying backend configurations using SQL commands.

The SQL syntax for displaying backend configurations is similar to that
for displaying frontend configurations:

1. Display backend configurations matching a pattern:
This statement will display all active backend configurations that match
the specified pattern.

show backend config like ${pattern}

3. Display backend configurations for a specific BE using the backend ID

show backend config from ${beId}

4. Use both a pattern and a backend ID:

show backend config like ${pattern} from ${backendID}


The output of these statements consists of 6 columns: BE ID, BE host IP,
configuration key, configuration value, configuration type, and whether
it is mutable or not.

+-----------+---------------+---------+-------+---------+-----------+
| BackendId | Host          | Key     | Value | Type    | IsMutable |
+-----------+---------------+---------+-------+---------+-----------+
| 10001     | xx.xx.xxx.xxx | be_port | 9060  | int32_t | false     |
+-----------+---------------+---------+-------+---------+-----------+

This provides an additional way to access backend configurations within
Doris.


Co-authored-by: duanxujian <duanxujian@jd.com>
@DarvenDuan DarvenDuan deleted the show_be_config branch November 6, 2024 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Support show backend config
7 participants