Skip to content

[Fix](func) CoreDump and Result Error in percentile function #36648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
xiaokang merged 1 commit into from
Jun 23, 2024
Merged

[Fix](func) CoreDump and Result Error in percentile function #36648

xiaokang merged 1 commit into from
Jun 23, 2024

Conversation

@HappenLee
Copy link
Contributor

@HappenLee HappenLee commented Jun 21, 2024

Proposed changes

cherry pick #36643

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actions bot reviewed Jun 21, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/aggregate_functions/aggregate_function_percentile_approx.h
}

double get() const { return vec_counts[0].terminate(vec_quantile[0]); }
double get() const { return vec_counts.empty() ? 0 : vec_counts[0].terminate(vec_quantile[0]); }
Copy link
Contributor

@github-actions github-actions bot Jun 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'get' should be marked [[nodiscard]] [modernize-use-nodiscard]

Suggested change
double get() const { return vec_counts.empty() ? 0 : vec_counts[0].terminate(vec_quantile[0]); }
[[nodiscard]] double get() const { return vec_counts.empty() ? 0 : vec_counts[0].terminate(vec_quantile[0]); }

@xiaokang
Copy link
Contributor

xiaokang commented Jun 23, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 23, 2024

TPC-H: Total hot run time: 50281 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0a9d938300b38d87c39f92ceb289d4fab222da68, data reload: false

------ Round 1 ----------------------------------
q1	17641	4390	4342	4342
q2	2068	155	147	147
q3	10546	1867	1909	1867
q4	10371	1252	1316	1252
q5	8496	3913	3910	3910
q6	241	149	123	123
q7	2063	1600	1585	1585
q8	9463	2742	2722	2722
q9	10521	10564	10300	10300
q10	8717	3534	3523	3523
q11	426	252	257	252
q12	476	301	303	301
q13	18485	3941	4071	3941
q14	361	330	339	330
q15	514	470	466	466
q16	729	602	626	602
q17	1135	966	947	947
q18	7663	7272	7342	7272
q19	2526	1585	1553	1553
q20	954	307	318	307
q21	4594	4114	4094	4094
q22	530	445	465	445
Total cold run time: 118520 ms
Total hot run time: 50281 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4387	4336	4329	4329
q2	317	224	220	220
q3	4199	4154	4150	4150
q4	2762	2747	2731	2731
q5	7144	7081	7096	7081
q6	243	121	124	121
q7	3229	2837	2803	2803
q8	4383	4485	4513	4485
q9	16942	16728	16786	16728
q10	4218	4257	4277	4257
q11	768	708	679	679
q12	1028	863	874	863
q13	7248	3766	3723	3723
q14	447	426	425	425
q15	491	460	454	454
q16	746	681	681	681
q17	3803	3833	3762	3762
q18	8760	8849	8798	8798
q19	1706	1727	1702	1702
q20	2404	2124	2105	2105
q21	8545	8478	8544	8478
q22	1017	1002	959	959
Total cold run time: 84787 ms
Total hot run time: 79534 ms

@doris-robot
Copy link

doris-robot commented Jun 23, 2024

TeamCity be ut coverage result:
Function Coverage: 37.86% (8107/21415)
Line Coverage: 29.52% (66344/224725)
Region Coverage: 28.99% (34186/117934)
Branch Coverage: 24.85% (17545/70606)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0a9d938300b38d87c39f92ceb289d4fab222da68_0a9d938300b38d87c39f92ceb289d4fab222da68/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 23, 2024

TPC-DS: Total hot run time: 206061 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0a9d938300b38d87c39f92ceb289d4fab222da68, data reload: false

query1	958	420	379	379
query2	6551	2904	2814	2814
query3	6915	206	202	202
query4	20165	18123	18068	18068
query5	19724	6470	6490	6470
query6	312	227	235	227
query7	4171	312	319	312
query8	453	432	401	401
query9	3144	2681	2600	2600
query10	464	308	309	308
query11	11281	10796	10706	10706
query12	122	79	78	78
query13	5590	708	693	693
query14	17717	13803	13446	13446
query15	367	243	248	243
query16	6448	287	263	263
query17	1757	1446	898	898
query18	2280	419	417	417
query19	224	152	156	152
query20	83	79	81	79
query21	198	96	101	96
query22	5138	4894	4954	4894
query23	32682	31897	32000	31897
query24	6972	6510	6537	6510
query25	531	451	450	450
query26	533	165	165	165
query27	1894	304	298	298
query28	6167	2309	2288	2288
query29	2908	2722	2690	2690
query30	243	168	165	165
query31	910	769	748	748
query32	73	66	63	63
query33	403	267	260	260
query34	852	474	476	474
query35	1135	893	961	893
query36	1234	1255	1069	1069
query37	94	61	60	60
query38	3038	2948	2960	2948
query39	1386	1333	1317	1317
query40	200	100	96	96
query41	54	44	43	43
query42	82	81	81	81
query43	738	780	615	615
query44	1119	722	733	722
query45	255	241	232	232
query46	1220	957	954	954
query47	2012	1768	1814	1768
query48	1015	733	699	699
query49	612	384	376	376
query50	863	629	592	592
query51	4715	4733	4696	4696
query52	97	76	82	76
query53	452	329	335	329
query54	2643	2454	2475	2454
query55	96	77	87	77
query56	255	230	214	214
query57	1255	1133	1064	1064
query58	218	210	192	192
query59	4258	3842	4062	3842
query60	214	196	212	196
query61	98	98	98	98
query62	839	469	550	469
query63	491	352	347	347
query64	2608	1527	1390	1390
query65	3651	3543	3563	3543
query66	792	381	392	381
query67	17750	17966	18130	17966
query68	4390	659	638	638
query69	512	346	339	339
query70	1337	1575	1346	1346
query71	377	315	310	310
query72	6393	3473	3516	3473
query73	709	321	315	315
query74	6235	5848	5817	5817
query75	4548	3617	3718	3617
query76	2715	1157	1193	1157
query77	433	246	257	246
query78	12488	11586	11736	11586
query79	7440	643	643	643
query80	3144	420	393	393
query81	558	241	234	234
query82	1410	96	96	96
query83	245	143	134	134
query84	227	74	73	73
query85	1626	320	313	313
query86	439	295	287	287
query87	3259	3071	3034	3034
query88	4760	2276	2285	2276
query89	384	310	283	283
query90	1635	213	211	211
query91	179	141	141	141
query92	76	52	53	52
query93	2210	536	571	536
query94	959	212	203	203
query95	1108	1055	1051	1051
query96	645	314	314	314
query97	6514	6351	6374	6351
query98	188	180	182	180
query99	2617	908	859	859
Total cold run time: 303479 ms
Total hot run time: 206061 ms

@doris-robot
Copy link

doris-robot commented Jun 23, 2024

ClickBench: Total hot run time: 31.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0a9d938300b38d87c39f92ceb289d4fab222da68, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.80	0.06	0.06
query5	0.53	0.52	0.52
query6	1.27	0.62	0.61
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.48	0.49
query10	0.54	0.53	0.52
query11	0.13	0.09	0.09
query12	0.13	0.10	0.09
query13	0.61	0.61	0.60
query14	0.78	0.78	0.77
query15	0.76	0.76	0.75
query16	0.35	0.38	0.38
query17	1.01	1.00	0.99
query18	0.23	0.24	0.24
query19	1.94	1.86	1.87
query20	0.01	0.01	0.02
query21	15.47	0.55	0.54
query22	1.98	2.00	1.64
query23	17.04	0.98	1.00
query24	3.90	1.76	1.56
query25	0.34	0.07	0.08
query26	0.60	0.14	0.16
query27	0.05	0.05	0.03
query28	8.29	0.71	0.79
query29	12.60	2.36	2.30
query30	0.59	0.52	0.50
query31	2.81	0.38	0.37
query32	3.37	0.49	0.49
query33	3.06	3.07	3.03
query34	15.27	4.80	4.78
query35	4.88	4.83	4.86
query36	1.04	1.01	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 102.66 s
Total hot run time: 31.32 s

@doris-robot
Copy link

doris-robot commented Jun 23, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0a9d938300b38d87c39f92ceb289d4fab222da68 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@xiaokang xiaokang merged commit 4c7af57 into apache:branch-2.0 Jun 23, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@HappenLee
[Fix](func) CoreDump and Result Error in percentile function apache#3…
1b5876f 
…6643 (apache#36648)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

area/nereids kind/test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@HappenLee @doris-robot @xiaokang