Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[opt](nereids)print numNulls of column stats in memo plan #40029

Merged
merged 1 commit into from
Aug 29, 2024

Conversation

englefly
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49502 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16f41fdebb29be81fce070c65525882e420f5fd6, data reload: false

------ Round 1 ----------------------------------
q1	18151	4380	4298	4298
q2	2056	151	146	146
q3	10267	1889	1895	1889
q4	10358	1240	1343	1240
q5	8915	3874	3948	3874
q6	222	120	123	120
q7	2051	1600	1605	1600
q8	9324	2713	2684	2684
q9	10949	10445	10488	10445
q10	8631	3518	3505	3505
q11	412	238	251	238
q12	462	295	306	295
q13	18338	3950	4047	3950
q14	345	323	314	314
q15	497	469	460	460
q16	686	569	576	569
q17	1128	937	942	937
q18	7246	6838	6759	6759
q19	1685	1572	1443	1443
q20	500	312	306	306
q21	4351	4110	4038	4038
q22	509	393	392	392
Total cold run time: 117083 ms
Total hot run time: 49502 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4361	4291	4267	4267
q2	328	231	224	224
q3	4155	4134	4135	4134
q4	2756	2724	2743	2724
q5	7229	7146	7111	7111
q6	236	122	115	115
q7	3219	2875	2778	2778
q8	4340	4459	4448	4448
q9	17345	16981	16969	16969
q10	4207	4246	4250	4246
q11	745	691	697	691
q12	1032	851	853	851
q13	7257	3787	3766	3766
q14	448	425	413	413
q15	501	452	450	450
q16	744	674	684	674
q17	3799	3803	3879	3803
q18	8698	8841	8746	8746
q19	1684	1682	1678	1678
q20	2382	2126	2120	2120
q21	8376	8507	8379	8379
q22	1049	954	939	939
Total cold run time: 84891 ms
Total hot run time: 79526 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202677 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16f41fdebb29be81fce070c65525882e420f5fd6, data reload: false

query1	915	385	443	385
query2	6560	2273	2177	2177
query3	6920	208	203	203
query4	21113	18027	17970	17970
query5	19759	6514	6474	6474
query6	291	218	223	218
query7	4159	296	309	296
query8	252	250	270	250
query9	3135	2719	2649	2649
query10	428	308	306	306
query11	11356	10632	10839	10632
query12	131	79	73	73
query13	5613	675	671	671
query14	17804	13091	13632	13091
query15	364	217	227	217
query16	6454	290	265	265
query17	1694	1446	858	858
query18	2331	418	414	414
query19	213	151	150	150
query20	75	82	84	82
query21	188	97	93	93
query22	5313	4958	5020	4958
query23	32503	32053	31745	31745
query24	7008	6484	6565	6484
query25	524	436	439	436
query26	527	169	168	168
query27	1859	293	301	293
query28	6122	2289	2271	2271
query29	2991	2780	2854	2780
query30	243	168	164	164
query31	911	720	780	720
query32	69	63	64	63
query33	393	264	264	264
query34	844	461	478	461
query35	1132	947	933	933
query36	1373	1198	1116	1116
query37	90	59	60	59
query38	3038	2921	2906	2906
query39	1375	1328	1353	1328
query40	202	97	96	96
query41	41	38	38	38
query42	88	83	87	83
query43	680	639	619	619
query44	1135	718	724	718
query45	246	233	229	229
query46	1209	959	955	955
query47	1843	1700	1751	1700
query48	969	672	677	672
query49	616	377	379	377
query50	873	632	579	579
query51	4829	4743	4697	4697
query52	91	79	71	71
query53	453	324	321	321
query54	2677	2455	2470	2455
query55	93	72	79	72
query56	236	220	211	211
query57	1134	1171	1140	1140
query58	226	202	214	202
query59	3325	3347	3151	3151
query60	221	199	219	199
query61	100	96	95	95
query62	829	453	512	453
query63	491	337	334	334
query64	2441	1559	1416	1416
query65	3645	3561	3544	3544
query66	818	376	383	376
query67	18553	16050	15123	15123
query68	8982	658	653	653
query69	575	362	341	341
query70	1564	1301	1387	1301
query71	416	312	321	312
query72	6597	3456	3466	3456
query73	740	329	321	321
query74	6272	5872	5741	5741
query75	5501	3735	3776	3735
query76	5495	1143	1189	1143
query77	937	259	248	248
query78	12803	13297	13566	13297
query79	11539	620	638	620
query80	880	402	406	402
query81	496	234	228	228
query82	289	94	95	94
query83	188	134	133	133
query84	261	73	71	71
query85	785	324	317	317
query86	347	290	337	290
query87	3271	3010	3035	3010
query88	5062	2279	2296	2279
query89	340	300	277	277
query90	2689	200	215	200
query91	156	131	124	124
query92	59	52	52	52
query93	2140	550	582	550
query94	1181	210	215	210
query95	1130	1079	1061	1061
query96	629	321	324	321
query97	6617	6344	6408	6344
query98	185	170	164	164
query99	2961	833	900	833
Total cold run time: 315221 ms
Total hot run time: 202677 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16f41fdebb29be81fce070c65525882e420f5fd6, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.03
query3	0.25	0.05	0.04
query4	1.80	0.07	0.06
query5	0.53	0.52	0.51
query6	1.23	0.60	0.61
query7	0.01	0.02	0.02
query8	0.03	0.02	0.02
query9	0.53	0.51	0.48
query10	0.55	0.54	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.61	0.61	0.60
query14	0.79	0.77	0.78
query15	0.78	0.76	0.76
query16	0.36	0.37	0.36
query17	1.02	1.03	0.97
query18	0.24	0.25	0.24
query19	1.96	1.85	1.87
query20	0.02	0.01	0.01
query21	15.46	0.55	0.53
query22	2.15	2.10	1.55
query23	17.15	0.95	1.09
query24	7.75	0.99	1.38
query25	0.36	0.11	0.05
query26	0.81	0.16	0.15
query27	0.04	0.03	0.03
query28	5.07	0.77	0.73
query29	12.75	2.25	2.28
query30	0.54	0.54	0.53
query31	2.82	0.38	0.37
query32	3.36	0.50	0.49
query33	3.11	3.05	3.09
query34	15.25	4.81	4.83
query35	4.87	4.86	4.87
query36	1.07	1.01	1.01
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.97 s
Total hot run time: 30.66 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 16f41fdebb29be81fce070c65525882e420f5fd6 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@englefly englefly merged commit 5a80fb0 into apache:branch-2.0 Aug 29, 2024
23 of 25 checks passed
@englefly englefly deleted the numnulls branch August 29, 2024 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants