Skip to content

[fix](statistics)Skip unknown stats while show cached partition stats.#36243

Merged
Jibing-Li merged 1 commit intoapache:masterfrom
Jibing-Li:unknown
Jun 13, 2024
Merged

[fix](statistics)Skip unknown stats while show cached partition stats.#36243
Jibing-Li merged 1 commit intoapache:masterfrom
Jibing-Li:unknown

Conversation

@Jibing-Li
Copy link
Contributor

@Jibing-Li Jibing-Li commented Jun 13, 2024

Don't show anything if cached partition stats for a column doesn't exists. Before it will show default value which is misleading.

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review June 13, 2024 07:06
@Jibing-Li
Copy link
Contributor Author

Jibing-Li commented Jun 13, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 13, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 13, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 13, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

TPC-H: Total hot run time: 41148 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ccb03751135cf5d64763e6d0d8f1bd378ae0b3f7, data reload: false

------ Round 1 ----------------------------------
q1	17641	4598	4284	4284
q2	2036	190	187	187
q3	10582	1174	1194	1174
q4	10333	714	972	714
q5	7583	2717	2649	2649
q6	223	139	134	134
q7	951	618	584	584
q8	9215	2088	2106	2088
q9	8831	6529	6472	6472
q10	8907	3693	3740	3693
q11	440	236	232	232
q12	417	236	226	226
q13	17761	2978	2980	2978
q14	268	218	213	213
q15	517	465	478	465
q16	519	400	367	367
q17	966	699	713	699
q18	8430	8000	7931	7931
q19	7939	1336	1488	1336
q20	682	327	321	321
q21	5208	4057	4184	4057
q22	394	344	344	344
Total cold run time: 119843 ms
Total hot run time: 41148 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4666	4290	4425	4290
q2	387	281	263	263
q3	3123	2892	2833	2833
q4	1972	1759	1717	1717
q5	5536	5525	5605	5525
q6	227	131	128	128
q7	2197	1863	1865	1863
q8	3299	3490	3456	3456
q9	8778	8762	8699	8699
q10	4097	3891	3902	3891
q11	576	476	480	476
q12	774	624	610	610
q13	15852	3191	3110	3110
q14	318	266	270	266
q15	525	483	482	482
q16	507	439	439	439
q17	1841	1515	1499	1499
q18	8057	7958	7857	7857
q19	1829	1465	1646	1465
q20	2149	1841	1767	1767
q21	7716	4954	4917	4917
q22	628	547	532	532
Total cold run time: 75054 ms
Total hot run time: 56085 ms

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

TPC-DS: Total hot run time: 173307 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ccb03751135cf5d64763e6d0d8f1bd378ae0b3f7, data reload: false

query1	804	377	379	377
query2	6458	2437	2186	2186
query3	6414	207	208	207
query4	20962	17376	17185	17185
query5	3911	489	459	459
query6	246	170	165	165
query7	4407	302	292	292
query8	316	284	268	268
query9	8308	2500	2471	2471
query10	554	302	287	287
query11	10582	10113	10094	10094
query12	147	90	84	84
query13	1583	374	362	362
query14	10102	6924	7629	6924
query15	245	189	182	182
query16	7766	270	266	266
query17	1393	534	525	525
query18	1949	274	281	274
query19	192	154	147	147
query20	94	81	83	81
query21	211	132	128	128
query22	4252	4200	4062	4062
query23	33753	33565	33529	33529
query24	11316	2789	2779	2779
query25	629	362	368	362
query26	1564	152	153	152
query27	2917	322	317	317
query28	7487	2082	2062	2062
query29	1003	619	628	619
query30	247	150	152	150
query31	966	735	747	735
query32	100	53	56	53
query33	790	276	272	272
query34	941	481	463	463
query35	735	647	610	610
query36	1060	926	917	917
query37	147	69	72	69
query38	2924	2716	2711	2711
query39	840	808	787	787
query40	254	125	124	124
query41	55	50	55	50
query42	125	100	95	95
query43	582	555	549	549
query44	1203	739	731	731
query45	199	159	165	159
query46	1078	718	702	702
query47	1860	1798	1804	1798
query48	395	304	315	304
query49	912	414	401	401
query50	775	395	397	395
query51	6768	6716	6666	6666
query52	100	92	93	92
query53	355	294	328	294
query54	968	449	442	442
query55	73	72	77	72
query56	289	261	255	255
query57	1140	1024	1044	1024
query58	249	263	240	240
query59	3373	3420	3264	3264
query60	296	260	267	260
query61	91	87	91	87
query62	617	432	447	432
query63	315	292	291	291
query64	9725	2273	1762	1762
query65	3259	3097	3094	3094
query66	1332	324	331	324
query67	15637	14974	15039	14974
query68	4426	537	532	532
query69	456	315	313	313
query70	1195	1104	1146	1104
query71	396	272	288	272
query72	6946	5475	5730	5475
query73	752	329	323	323
query74	5894	5559	5475	5475
query75	3425	2660	2681	2660
query76	2188	950	872	872
query77	451	305	316	305
query78	10306	9819	9750	9750
query79	2453	518	535	518
query80	1096	467	465	465
query81	587	219	227	219
query82	713	101	102	101
query83	226	168	171	168
query84	250	85	88	85
query85	2015	274	277	274
query86	507	330	309	309
query87	3226	3090	3101	3090
query88	4309	2464	2468	2464
query89	492	392	374	374
query90	1890	187	189	187
query91	137	175	100	100
query92	62	49	49	49
query93	2153	509	495	495
query94	1286	190	188	188
query95	394	313	307	307
query96	593	271	271	271
query97	3242	3035	3052	3035
query98	210	197	191	191
query99	1108	842	869	842
Total cold run time: 273724 ms
Total hot run time: 173307 ms

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

ClickBench: Total hot run time: 31.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ccb03751135cf5d64763e6d0d8f1bd378ae0b3f7, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.68	0.08	0.09
query5	0.50	0.48	0.48
query6	1.13	0.72	0.72
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.55	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.58	0.59
query14	0.77	0.80	0.78
query15	0.83	0.82	0.81
query16	0.37	0.37	0.35
query17	1.00	0.98	0.99
query18	0.23	0.25	0.24
query19	1.84	1.77	1.71
query20	0.01	0.02	0.01
query21	15.43	0.66	0.65
query22	4.48	6.80	1.99
query23	18.29	1.28	1.22
query24	2.11	0.22	0.23
query25	0.15	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.08	0.08
query28	13.20	1.01	0.99
query29	12.62	3.27	3.26
query30	0.26	0.07	0.09
query31	2.83	0.39	0.38
query32	3.28	0.47	0.47
query33	2.92	2.98	2.93
query34	17.26	4.71	4.71
query35	4.76	4.69	4.54
query36	0.64	0.47	0.48
query37	0.17	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.15	0.15
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.06 s
Total hot run time: 31.03 s

morningman
morningman approved these changes Jun 13, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Jibing-Li Jibing-Li merged commit 72a809b into apache:master Jun 13, 2024
@Jibing-Li Jibing-Li deleted the unknown branch June 13, 2024 09:41
dataroaring pushed a commit that referenced this pull request Jun 17, 2024
@Jibing-Li @dataroaring
[fix](statistics)Skip unknown stats while show cached partition stats. (
9f0dd14 
#36243)

Don't show anything if cached partition stats for a column doesn't
exists. Before it will show default value which is misleading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged not-merge/2.1 reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Jibing-Li @doris-robot @morningman @morrySnow @dataroaring