Skip to content

Conversation

@Jibing-Li
Copy link
Contributor

backport: #37545

Fix column cached stats average size calculate bug. Average size is
double type, when calculating, it is calculated by totalSize/rowCount.
We need to cast totalSize to double, otherwise the result is not
accurate, only keep the integer part.
@Jibing-Li Jibing-Li marked this pull request as ready for review July 11, 2024 08:55
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49655 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 46892657c289a9efbe138adaedc620ab7cd9280d, data reload: false

------ Round 1 ----------------------------------
q1	18194	4337	4364	4337
q2	2040	152	145	145
q3	10280	1918	1898	1898
q4	10347	1248	1311	1248
q5	8912	3876	3920	3876
q6	233	125	125	125
q7	2054	1580	1616	1580
q8	9323	2725	2704	2704
q9	10361	10386	10206	10206
q10	8661	3538	3517	3517
q11	427	248	248	248
q12	471	305	301	301
q13	18353	3978	4056	3978
q14	352	344	335	335
q15	508	458	458	458
q16	694	579	573	573
q17	1120	948	974	948
q18	7246	6776	6815	6776
q19	1794	1662	1589	1589
q20	519	319	310	310
q21	4456	4085	4061	4061
q22	523	454	442	442
Total cold run time: 116868 ms
Total hot run time: 49655 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4316	4290	4301	4290
q2	323	234	221	221
q3	4188	4170	4150	4150
q4	2754	2729	2734	2729
q5	7229	7142	7146	7142
q6	234	123	116	116
q7	3256	2906	2867	2867
q8	4338	4466	4498	4466
q9	17005	16785	16766	16766
q10	4237	4266	4299	4266
q11	759	695	676	676
q12	1030	872	880	872
q13	6409	3759	3725	3725
q14	464	429	439	429
q15	501	447	456	447
q16	742	679	688	679
q17	3892	3848	3820	3820
q18	8725	8699	8949	8699
q19	1748	1733	1643	1643
q20	2424	2126	2116	2116
q21	8541	8543	8534	8534
q22	1021	974	990	974
Total cold run time: 84136 ms
Total hot run time: 79627 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 205845 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46892657c289a9efbe138adaedc620ab7cd9280d, data reload: false

query1	939	426	372	372
query2	6532	2943	2752	2752
query3	6922	208	203	203
query4	21463	18137	18079	18079
query5	19915	6526	6447	6447
query6	296	220	233	220
query7	4167	298	307	298
query8	449	416	403	403
query9	3126	2705	2673	2673
query10	418	302	300	300
query11	11517	10967	10993	10967
query12	125	75	77	75
query13	5608	703	693	693
query14	19670	13583	14014	13583
query15	361	244	249	244
query16	6621	296	263	263
query17	1834	2043	900	900
query18	4065	436	424	424
query19	229	162	164	162
query20	92	95	95	95
query21	195	110	103	103
query22	5787	5614	5524	5524
query23	33009	31778	31964	31778
query24	6948	6540	6503	6503
query25	528	427	424	424
query26	527	162	165	162
query27	1805	294	298	294
query28	6199	2376	2321	2321
query29	3080	2733	2837	2733
query30	246	164	169	164
query31	890	732	773	732
query32	75	65	61	61
query33	387	254	250	250
query34	846	478	468	468
query35	1114	885	949	885
query36	1361	1142	1186	1142
query37	90	63	60	60
query38	3058	2964	2926	2926
query39	1383	1315	1317	1315
query40	206	98	92	92
query41	47	44	45	44
query42	79	84	83	83
query43	699	723	640	640
query44	1157	737	736	736
query45	253	233	236	233
query46	1268	989	981	981
query47	1790	1840	1884	1840
query48	999	715	709	709
query49	613	367	380	367
query50	858	577	611	577
query51	4774	4735	4699	4699
query52	90	78	86	78
query53	448	315	319	315
query54	2674	2466	2486	2466
query55	93	88	86	86
query56	236	225	203	203
query57	1171	1257	1127	1127
query58	221	196	214	196
query59	4341	4057	3964	3964
query60	230	207	210	207
query61	98	94	104	94
query62	802	476	486	476
query63	486	344	357	344
query64	2508	1511	1445	1445
query65	3618	3577	3581	3577
query66	790	383	381	381
query67	17223	16122	16052	16052
query68	8531	665	634	634
query69	573	358	352	352
query70	1573	1463	1590	1463
query71	403	296	323	296
query72	6594	3502	3473	3473
query73	738	324	328	324
query74	6382	5951	5835	5835
query75	4598	3756	3668	3668
query76	4762	1159	1215	1159
query77	630	257	257	257
query78	12609	12057	11735	11735
query79	9219	645	654	645
query80	1485	406	395	395
query81	494	245	233	233
query82	1176	97	96	96
query83	166	134	133	133
query84	260	73	72	72
query85	955	341	339	339
query86	344	293	315	293
query87	3246	3055	3055	3055
query88	4975	2336	2330	2330
query89	418	291	296	291
query90	1887	208	211	208
query91	173	160	153	153
query92	58	53	55	53
query93	5767	569	538	538
query94	780	211	215	211
query95	1118	1063	1031	1031
query96	656	324	319	319
query97	6457	6234	6498	6234
query98	188	177	183	177
query99	3082	833	881	833
Total cold run time: 319946 ms
Total hot run time: 205845 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46892657c289a9efbe138adaedc620ab7cd9280d, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.05
query4	1.80	0.06	0.07
query5	0.54	0.53	0.53
query6	1.24	0.62	0.60
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.49	0.47
query10	0.53	0.51	0.54
query11	0.12	0.08	0.09
query12	0.11	0.10	0.09
query13	0.62	0.61	0.61
query14	0.79	0.80	0.78
query15	0.78	0.76	0.76
query16	0.38	0.36	0.37
query17	1.03	0.97	1.02
query18	0.22	0.25	0.25
query19	1.89	1.82	1.87
query20	0.02	0.01	0.01
query21	15.46	0.57	0.56
query22	2.38	2.55	1.61
query23	16.95	1.10	0.82
query24	4.32	3.68	1.14
query25	0.42	0.11	0.04
query26	0.72	0.17	0.17
query27	0.04	0.03	0.04
query28	5.93	0.72	0.76
query29	12.62	2.33	2.40
query30	0.57	0.53	0.53
query31	2.82	0.38	0.36
query32	3.39	0.50	0.50
query33	3.05	3.09	3.06
query34	15.27	4.80	4.78
query35	4.91	4.82	4.85
query36	1.07	1.01	1.01
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.13	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 101.29 s
Total hot run time: 30.75 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 46892657c289a9efbe138adaedc620ab7cd9280d with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@Jibing-Li Jibing-Li merged commit 6554a5e into apache:branch-2.0 Jul 11, 2024
@Jibing-Li Jibing-Li deleted the size2.0 branch July 12, 2024 01:52
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants