Skip to content

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Jun 11, 2024

Proposed changes

  1. add more meta_service ut;
  2. remove duplicated table_id to reduce fdb value length

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 11, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

}
}

TEST(MetaServiceTest, CommitTxnWithSubTxnTest2) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'TEST' exceeds recommended size/complexity thresholds [readability-function-size]

TEST(MetaServiceTest, CommitTxnWithSubTxnTest2) {
^
Additional context

cloud/test/meta_service_test.cpp:1593: 176 lines including whitespace and comments (threshold 80)

TEST(MetaServiceTest, CommitTxnWithSubTxnTest2) {
^

@doris-robot
Copy link

TPC-H: Total hot run time: 40345 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aba7cc37ea6ba4a96c6c860e1efcea2cfd7ad7a5, data reload: false

------ Round 1 ----------------------------------
q1	17632	4350	4238	4238
q2	2026	194	184	184
q3	10459	1113	1060	1060
q4	10176	758	753	753
q5	7476	2657	2644	2644
q6	213	137	135	135
q7	943	620	595	595
q8	9225	2041	2066	2041
q9	9062	6424	6413	6413
q10	8901	3654	3680	3654
q11	453	235	239	235
q12	447	225	215	215
q13	18898	2946	3003	2946
q14	260	222	227	222
q15	519	477	466	466
q16	487	387	364	364
q17	973	647	699	647
q18	8028	7592	7546	7546
q19	7686	1504	1447	1447
q20	661	319	312	312
q21	5002	3898	3901	3898
q22	391	345	330	330
Total cold run time: 119918 ms
Total hot run time: 40345 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4531	4265	4295	4265
q2	394	257	260	257
q3	3196	2963	2897	2897
q4	2021	1759	1722	1722
q5	5538	5444	5698	5444
q6	220	135	137	135
q7	2231	1876	1922	1876
q8	3278	3434	3405	3405
q9	8956	9049	9069	9049
q10	4109	3940	4077	3940
q11	647	546	515	515
q12	863	665	685	665
q13	16520	3167	3129	3129
q14	295	294	275	275
q15	551	486	516	486
q16	508	458	418	418
q17	1838	1565	1521	1521
q18	8244	8126	7757	7757
q19	1824	1698	1682	1682
q20	3210	1869	1846	1846
q21	8366	4963	4746	4746
q22	599	559	547	547
Total cold run time: 77939 ms
Total hot run time: 56577 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173474 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aba7cc37ea6ba4a96c6c860e1efcea2cfd7ad7a5, data reload: false

query1	934	381	390	381
query2	6453	2415	2408	2408
query3	6635	212	215	212
query4	19488	17328	17195	17195
query5	3556	457	453	453
query6	239	177	158	158
query7	4579	299	290	290
query8	331	295	294	294
query9	8687	2347	2335	2335
query10	577	320	285	285
query11	10455	10080	9889	9889
query12	125	87	88	87
query13	1641	380	375	375
query14	10031	7749	6898	6898
query15	236	196	189	189
query16	7880	276	271	271
query17	1900	564	522	522
query18	2001	280	279	279
query19	202	162	158	158
query20	90	85	82	82
query21	215	137	127	127
query22	4254	4036	4189	4036
query23	33961	33571	33767	33571
query24	11073	2917	2913	2913
query25	628	408	384	384
query26	1216	165	158	158
query27	2941	319	331	319
query28	7690	2092	2152	2092
query29	873	615	641	615
query30	260	158	160	158
query31	967	765	752	752
query32	92	52	55	52
query33	763	293	283	283
query34	997	484	485	484
query35	742	632	605	605
query36	1111	992	961	961
query37	155	74	76	74
query38	2960	2836	2793	2793
query39	902	856	863	856
query40	206	138	128	128
query41	55	53	52	52
query42	119	105	111	105
query43	596	567	537	537
query44	1216	730	742	730
query45	193	163	156	156
query46	1083	725	703	703
query47	1828	1748	1775	1748
query48	380	303	307	303
query49	853	423	410	410
query50	790	386	401	386
query51	6784	6713	6703	6703
query52	107	89	94	89
query53	402	302	288	288
query54	869	447	434	434
query55	74	74	73	73
query56	276	253	250	250
query57	1140	1044	1074	1044
query58	257	249	238	238
query59	3427	3063	3149	3063
query60	315	275	279	275
query61	89	83	88	83
query62	618	443	449	443
query63	327	298	303	298
query64	8767	2244	1694	1694
query65	3155	3082	3117	3082
query66	743	366	327	327
query67	15282	15087	15002	15002
query68	4464	538	563	538
query69	503	385	362	362
query70	1160	1137	1091	1091
query71	433	276	277	276
query72	7142	5464	5821	5464
query73	751	326	330	326
query74	5829	5477	5554	5477
query75	3500	2588	2653	2588
query76	2768	977	992	977
query77	462	298	304	298
query78	10461	9809	9798	9798
query79	3145	516	515	515
query80	2144	527	481	481
query81	558	218	224	218
query82	1122	108	105	105
query83	327	175	171	171
query84	265	92	88	88
query85	1502	277	261	261
query86	477	321	314	314
query87	3227	3050	3079	3050
query88	4196	2437	2446	2437
query89	522	394	398	394
query90	1800	190	197	190
query91	172	96	117	96
query92	65	51	49	49
query93	4537	517	504	504
query94	1125	187	182	182
query95	399	313	301	301
query96	597	270	274	270
query97	3244	2998	2982	2982
query98	209	201	189	189
query99	1206	844	847	844
Total cold run time: 276103 ms
Total hot run time: 173474 ms

dataroaring
dataroaring previously approved these changes Jun 11, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 11, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

ClickBench: Total hot run time: 31.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aba7cc37ea6ba4a96c6c860e1efcea2cfd7ad7a5, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.10	0.09
query5	0.51	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.47	0.49
query10	0.53	0.56	0.53
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.77	0.78
query15	0.83	0.82	0.81
query16	0.36	0.37	0.37
query17	1.03	1.04	1.03
query18	0.21	0.29	0.22
query19	1.81	1.80	1.78
query20	0.01	0.02	0.01
query21	15.43	0.65	0.65
query22	3.78	7.47	2.38
query23	18.24	1.49	1.34
query24	2.17	0.22	0.22
query25	0.16	0.08	0.09
query26	0.26	0.17	0.17
query27	0.08	0.08	0.08
query28	13.21	1.02	0.99
query29	12.61	3.22	3.22
query30	0.27	0.06	0.06
query31	2.86	0.38	0.38
query32	3.28	0.47	0.47
query33	2.94	2.87	2.88
query34	17.01	4.43	4.45
query35	4.52	4.51	4.45
query36	0.65	0.46	0.46
query37	0.17	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.06	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.03
Total cold run time: 108.92 s
Total hot run time: 31.11 s

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run cloud_ut

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run fe_ut

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run feut cloudut

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run cloudut

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 12, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 40102 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30c2997e6e838785510cff81db5947f3ffc119a2, data reload: false

------ Round 1 ----------------------------------
q1	17981	4457	4446	4446
q2	2654	194	191	191
q3	11931	1088	1209	1088
q4	10319	865	852	852
q5	7494	2715	2703	2703
q6	227	141	141	141
q7	991	619	601	601
q8	9283	2070	2056	2056
q9	8965	6474	6461	6461
q10	8966	3747	3712	3712
q11	452	231	236	231
q12	399	237	227	227
q13	18957	2965	2984	2965
q14	265	213	216	213
q15	507	478	510	478
q16	531	385	374	374
q17	971	670	646	646
q18	8022	7574	7458	7458
q19	6845	1489	1440	1440
q20	662	305	306	305
q21	4888	3185	3921	3185
q22	378	329	341	329
Total cold run time: 121688 ms
Total hot run time: 40102 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4341	4234	4250	4234
q2	360	270	262	262
q3	2967	2781	2739	2739
q4	1893	1576	1625	1576
q5	5286	5290	5254	5254
q6	218	128	131	128
q7	2045	1766	1679	1679
q8	3164	3310	3317	3310
q9	8275	8269	8234	8234
q10	3850	3578	3686	3578
q11	628	500	475	475
q12	764	606	587	587
q13	17372	2955	3018	2955
q14	296	256	262	256
q15	521	468	467	467
q16	471	414	404	404
q17	1769	1509	1448	1448
q18	7684	7601	7239	7239
q19	1707	1628	1559	1559
q20	1997	1763	1761	1761
q21	4885	4722	4740	4722
q22	599	545	532	532
Total cold run time: 71092 ms
Total hot run time: 53399 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169192 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30c2997e6e838785510cff81db5947f3ffc119a2, data reload: false

query1	939	380	380	380
query2	6458	2522	2243	2243
query3	6650	212	219	212
query4	19187	17408	17410	17408
query5	4164	463	452	452
query6	248	164	157	157
query7	4591	296	290	290
query8	345	303	290	290
query9	8673	2366	2356	2356
query10	610	315	297	297
query11	10587	10114	10049	10049
query12	145	83	81	81
query13	1642	373	367	367
query14	9747	7874	6149	6149
query15	228	201	192	192
query16	7646	262	257	257
query17	1934	524	533	524
query18	1519	277	267	267
query19	198	154	158	154
query20	97	82	84	82
query21	210	131	126	126
query22	4156	4061	4263	4061
query23	33551	33013	32902	32902
query24	11951	2854	2778	2778
query25	627	360	360	360
query26	1685	154	157	154
query27	2874	316	310	310
query28	7082	2037	2024	2024
query29	1008	647	631	631
query30	288	159	154	154
query31	971	740	750	740
query32	93	58	55	55
query33	766	292	283	283
query34	951	475	454	454
query35	752	612	634	612
query36	1106	939	926	926
query37	143	73	71	71
query38	2914	2756	2725	2725
query39	859	788	804	788
query40	291	124	128	124
query41	56	53	51	51
query42	123	99	102	99
query43	589	555	542	542
query44	1210	731	739	731
query45	195	168	168	168
query46	1116	710	703	703
query47	1844	1784	1732	1732
query48	375	309	294	294
query49	1128	420	419	419
query50	759	401	381	381
query51	6742	6731	6710	6710
query52	107	95	105	95
query53	356	294	294	294
query54	1042	449	444	444
query55	74	74	73	73
query56	292	279	275	275
query57	1162	1069	1056	1056
query58	265	247	248	247
query59	3531	3288	3315	3288
query60	308	291	282	282
query61	114	113	110	110
query62	650	465	442	442
query63	333	303	296	296
query64	10049	2303	1731	1731
query65	3378	3177	3100	3100
query66	1312	339	330	330
query67	15326	15024	14730	14730
query68	4570	535	550	535
query69	526	408	365	365
query70	1154	1093	1141	1093
query71	401	270	276	270
query72	7078	5523	2765	2765
query73	752	323	321	321
query74	5930	5505	5438	5438
query75	3342	2681	2676	2676
query76	2563	945	913	913
query77	511	310	303	303
query78	10254	9955	9703	9703
query79	2291	515	516	515
query80	916	475	513	475
query81	590	221	215	215
query82	770	106	100	100
query83	272	173	169	169
query84	247	86	87	86
query85	1878	291	271	271
query86	484	316	330	316
query87	3300	3083	3113	3083
query88	3979	2449	2442	2442
query89	493	387	384	384
query90	1820	195	190	190
query91	134	102	106	102
query92	65	51	55	51
query93	2458	520	505	505
query94	1262	192	186	186
query95	412	316	319	316
query96	613	274	265	265
query97	3144	3004	3082	3004
query98	232	202	193	193
query99	1150	834	866	834
Total cold run time: 273913 ms
Total hot run time: 169192 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 30c2997e6e838785510cff81db5947f3ffc119a2, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.04	0.04
query4	1.69	0.08	0.07
query5	0.51	0.49	0.49
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.48	0.49
query10	0.53	0.55	0.55
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.61
query14	0.77	0.78	0.80
query15	0.84	0.81	0.82
query16	0.36	0.36	0.35
query17	1.00	0.98	0.98
query18	0.19	0.26	0.24
query19	1.79	1.76	1.72
query20	0.01	0.01	0.01
query21	15.42	0.66	0.66
query22	4.67	7.12	1.58
query23	18.30	1.41	1.26
query24	2.16	0.22	0.21
query25	0.15	0.09	0.09
query26	0.27	0.17	0.17
query27	0.08	0.08	0.09
query28	13.18	1.02	1.01
query29	12.63	3.28	3.28
query30	0.25	0.06	0.06
query31	2.89	0.39	0.39
query32	3.27	0.47	0.46
query33	2.88	2.88	2.85
query34	17.08	4.47	4.40
query35	4.52	4.48	4.45
query36	0.66	0.48	0.47
query37	0.19	0.15	0.14
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.17	0.16	0.13
query41	0.10	0.05	0.04
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.84 s
Total hot run time: 30.18 s

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run cloudut

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run cloudp1

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jun 12, 2024

run cloud_p1

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 12, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit c050055 into apache:master Jun 13, 2024
dataroaring pushed a commit that referenced this pull request Jun 13, 2024
…o reduce fdb value length (#36135)

## Proposed changes

1. add more meta_service ut;
2. remove duplicated table_id to reduce fdb value length
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants