Skip to content

[Fix](show data) fix show data total size wrong#35818

Merged
dataroaring merged 1 commit intoapache:masterfrom
airborne12:fix-show-data
Jun 4, 2024
Merged

[Fix](show data) fix show data total size wrong#35818
dataroaring merged 1 commit intoapache:masterfrom
airborne12:fix-show-data

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Jun 3, 2024

Proposed changes

Issue Number: close #xxx

When setting replica = 3, the show data command returns an incorrect data size. This issue arises because the total size calculation only accounts for the maximum replica data size.

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12
Copy link
Member Author

airborne12 commented Jun 3, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

PR approved by anyone and no changes requested.

xiaokang
xiaokang approved these changes Jun 3, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 3, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

PR approved by at least one committer and no changes requested.

swjtu-zhanglei
swjtu-zhanglei approved these changes Jun 3, 2024
View reviewed changes
Copy link
Contributor

@swjtu-zhanglei swjtu-zhanglei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 41846 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 063e035f5fc69f391a7a78312f9f435ca8d5b3da, data reload: false

------ Round 1 ----------------------------------
q1	17615	4379	4303	4303
q2	2033	203	205	203
q3	10437	1265	1083	1083
q4	10200	808	833	808
q5	7500	2685	2726	2685
q6	224	136	141	136
q7	967	668	630	630
q8	9222	2192	2169	2169
q9	9518	6792	6783	6783
q10	9469	3940	3884	3884
q11	427	248	247	247
q12	458	234	256	234
q13	17241	3198	3267	3198
q14	276	226	231	226
q15	515	481	479	479
q16	501	407	405	405
q17	982	581	656	581
q18	8601	7913	7775	7775
q19	5974	1405	1309	1309
q20	644	329	335	329
q21	5184	4026	4172	4026
q22	418	360	353	353
Total cold run time: 118406 ms
Total hot run time: 41846 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4580	4447	4436	4436
q2	392	278	280	278
q3	3155	2933	2952	2933
q4	2001	1705	1620	1620
q5	5411	5565	5562	5562
q6	226	129	130	129
q7	2218	1792	1859	1792
q8	3226	3401	3400	3400
q9	8710	8674	8728	8674
q10	4116	3885	3871	3871
q11	586	491	501	491
q12	787	636	611	611
q13	16968	3104	3166	3104
q14	294	255	277	255
q15	514	465	475	465
q16	500	451	434	434
q17	1810	1554	1507	1507
q18	7971	7714	7319	7319
q19	1764	1541	1517	1517
q20	3012	1832	1787	1787
q21	4927	4718	4826	4718
q22	635	561	550	550
Total cold run time: 73803 ms
Total hot run time: 55453 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 169949 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 063e035f5fc69f391a7a78312f9f435ca8d5b3da, data reload: false

query1	916	378	376	376
query2	6438	2505	2458	2458
query3	6899	205	205	205
query4	19610	17397	17364	17364
query5	4131	444	476	444
query6	241	171	156	156
query7	4600	302	296	296
query8	345	292	287	287
query9	8590	2386	2366	2366
query10	432	293	286	286
query11	10589	10058	10058	10058
query12	133	92	86	86
query13	1628	397	367	367
query14	10188	6906	7565	6906
query15	225	190	169	169
query16	7812	271	260	260
query17	1696	548	552	548
query18	1945	281	262	262
query19	203	155	153	153
query20	103	89	89	89
query21	207	131	125	125
query22	4232	3980	4068	3980
query23	33889	32743	32784	32743
query24	11021	2855	2811	2811
query25	605	364	358	358
query26	1173	161	156	156
query27	2504	327	322	322
query28	7191	2067	2073	2067
query29	885	628	596	596
query30	288	148	151	148
query31	957	751	735	735
query32	87	57	53	53
query33	759	320	279	279
query34	912	468	466	466
query35	750	637	617	617
query36	1109	914	884	884
query37	158	67	71	67
query38	2898	2763	2788	2763
query39	842	793	787	787
query40	215	125	123	123
query41	57	52	52	52
query42	128	97	97	97
query43	576	560	555	555
query44	1256	744	765	744
query45	202	165	164	164
query46	1088	742	748	742
query47	1842	1746	1785	1746
query48	378	298	306	298
query49	1010	413	410	410
query50	782	385	386	385
query51	7106	6744	6719	6719
query52	108	88	88	88
query53	361	285	287	285
query54	878	443	439	439
query55	72	75	71	71
query56	277	251	259	251
query57	1127	1021	1039	1021
query58	242	256	236	236
query59	3472	3306	3289	3289
query60	290	286	269	269
query61	110	90	88	88
query62	633	452	453	452
query63	313	293	297	293
query64	9018	2216	1714	1714
query65	3173	3207	3139	3139
query66	886	332	338	332
query67	15511	14962	14770	14770
query68	4536	536	532	532
query69	554	394	333	333
query70	1155	1058	1156	1058
query71	427	332	283	283
query72	7346	5321	2766	2766
query73	755	328	320	320
query74	5863	5461	5473	5461
query75	3431	2639	2658	2639
query76	2776	968	877	877
query77	641	299	293	293
query78	10316	9697	9697	9697
query79	1515	513	510	510
query80	816	495	463	463
query81	585	223	225	223
query82	761	100	98	98
query83	241	164	166	164
query84	249	91	84	84
query85	1551	303	328	303
query86	477	314	335	314
query87	3305	3098	3115	3098
query88	3891	2463	2460	2460
query89	495	413	378	378
query90	1812	190	192	190
query91	132	107	106	106
query92	66	51	50	50
query93	2087	513	504	504
query94	1214	199	194	194
query95	405	315	317	315
query96	593	273	273	273
query97	3191	3044	3007	3007
query98	234	223	225	223
query99	1343	868	868	868
Total cold run time: 271065 ms
Total hot run time: 169949 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 063e035f5fc69f391a7a78312f9f435ca8d5b3da, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.04	0.03
query3	0.24	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.49	0.50
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.57	0.51	0.50
query10	0.56	0.57	0.57
query11	0.17	0.13	0.12
query12	0.15	0.13	0.12
query13	0.60	0.59	0.60
query14	0.76	0.76	0.78
query15	0.81	0.82	0.81
query16	0.35	0.35	0.37
query17	0.97	0.99	0.96
query18	0.22	0.24	0.24
query19	1.82	1.76	1.79
query20	0.01	0.01	0.00
query21	15.73	0.65	0.64
query22	4.29	7.68	1.66
query23	18.27	1.40	1.25
query24	1.86	0.21	0.27
query25	0.15	0.08	0.09
query26	0.26	0.16	0.16
query27	0.08	0.08	0.08
query28	13.40	1.11	1.09
query29	13.17	3.28	3.29
query30	0.24	0.05	0.06
query31	2.85	0.39	0.38
query32	3.28	0.47	0.47
query33	2.91	2.86	2.90
query34	17.07	4.37	4.44
query35	4.52	4.50	4.62
query36	0.64	0.46	0.46
query37	0.18	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.16	0.15	0.15
query41	0.09	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.12 s
Total hot run time: 30.35 s

dataroaring
dataroaring approved these changes Jun 4, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit f7ca8fe into apache:master Jun 4, 2024
dataroaring pushed a commit that referenced this pull request Jun 4, 2024
@airborne12 @dataroaring
[Fix](show data) fix show data total size wrong (#35818)
7ba17cb 
When setting replica = 3, the show data command returns an incorrect
data size. This issue arises because the total size calculation only
accounts for the maximum replica data size.
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
@airborne12 @seawinde
[Fix](show data) fix show data total size wrong (apache#35818)
8f4d1aa 
When setting replica = 3, the show data command returns an incorrect
data size. This issue arises because the total size calculation only
accounts for the maximum replica data size.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

+1 more reviewer

@swjtu-zhanglei swjtu-zhanglei swjtu-zhanglei approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@airborne12 @doris-robot @xiaokang @gavinchou @swjtu-zhanglei @dataroaring