Skip to content

[BUG] fix scan range boundary handling is incorrect #34832 #35829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Jun 5, 2024
Merged

[BUG] fix scan range boundary handling is incorrect #34832 #35829

yiguolei merged 1 commit into from
Jun 5, 2024

Conversation

@zhiqiang-hhhh
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh commented Jun 3, 2024

cherry pick from #34832

@qzsee @zhiqiang-hhhh
[BUG] fix scan range boundary handling is incorrect (apache#34832)
1aa0f54 
fix scan range boundary handling is incorrect
Co-authored-by: shizhiqiang03 <shizhiqiang03@meituan.com>
@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Jun 3, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 50509 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1aa0f54172a1c24f4692b901940edcbd358df4ec, data reload: false

------ Round 1 ----------------------------------
q1	18038	4416	4400	4400
q2	2089	171	150	150
q3	10260	1930	1964	1930
q4	10119	1307	1347	1307
q5	8538	4073	3994	3994
q6	256	129	130	129
q7	2063	1616	1656	1616
q8	9514	2757	2765	2757
q9	13199	10568	10715	10568
q10	8634	3584	3570	3570
q11	429	240	247	240
q12	471	303	301	301
q13	18433	3960	4056	3960
q14	372	337	332	332
q15	522	459	467	459
q16	663	563	567	563
q17	1162	1018	960	960
q18	7298	6882	7074	6882
q19	1730	1622	1599	1599
q20	551	317	309	309
q21	4559	4224	4090	4090
q22	504	404	393	393
Total cold run time: 119404 ms
Total hot run time: 50509 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4358	4280	4308	4280
q2	330	228	228	228
q3	4192	4190	4183	4183
q4	2765	2760	2755	2755
q5	7253	7203	7236	7203
q6	247	122	126	122
q7	3257	2857	2814	2814
q8	4407	4482	4534	4482
q9	17470	17014	17212	17014
q10	4267	4264	4313	4264
q11	770	684	651	651
q12	1037	861	867	861
q13	6838	3766	3778	3766
q14	463	425	418	418
q15	506	461	451	451
q16	731	679	685	679
q17	3727	3798	3832	3798
q18	8887	8810	8915	8810
q19	1748	1652	1710	1652
q20	2380	2148	2089	2089
q21	8540	8573	8399	8399
q22	1025	927	953	927
Total cold run time: 85198 ms
Total hot run time: 79846 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TeamCity be ut coverage result:
Function Coverage: 37.81% (8101/21423)
Line Coverage: 29.48% (66161/224454)
Region Coverage: 28.97% (34110/117749)
Branch Coverage: 24.82% (17509/70536)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1aa0f54172a1c24f4692b901940edcbd358df4ec_1aa0f54172a1c24f4692b901940edcbd358df4ec/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 204839 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1aa0f54172a1c24f4692b901940edcbd358df4ec, data reload: false

query1	935	491	382	382
query2	6547	2843	2628	2628
query3	6916	220	214	214
query4	20076	18076	17912	17912
query5	19731	6520	6563	6520
query6	303	228	237	228
query7	4167	308	313	308
query8	281	237	249	237
query9	3150	2725	2652	2652
query10	432	307	299	299
query11	11243	10802	10739	10739
query12	125	77	76	76
query13	5589	695	670	670
query14	18164	13550	13557	13550
query15	385	225	230	225
query16	6450	288	283	283
query17	1671	1472	871	871
query18	2308	420	415	415
query19	212	152	154	152
query20	80	77	83	77
query21	189	93	103	93
query22	5286	5120	4979	4979
query23	32651	31721	31843	31721
query24	6977	6566	6445	6445
query25	532	440	413	413
query26	528	169	166	166
query27	1878	299	303	299
query28	6147	2361	2304	2304
query29	2834	2794	2756	2756
query30	247	167	172	167
query31	900	724	787	724
query32	71	64	62	62
query33	408	260	247	247
query34	872	460	474	460
query35	1151	940	937	937
query36	1209	1091	1146	1091
query37	96	60	59	59
query38	3063	2924	2913	2913
query39	1397	1340	1335	1335
query40	205	93	96	93
query41	40	40	36	36
query42	91	91	87	87
query43	743	635	694	635
query44	1108	724	723	723
query45	246	236	230	230
query46	1234	959	974	959
query47	2041	1690	1882	1690
query48	992	716	718	716
query49	615	370	378	370
query50	863	566	631	566
query51	4777	4634	4649	4634
query52	89	79	76	76
query53	457	325	325	325
query54	2669	2493	2473	2473
query55	86	87	86	86
query56	237	216	224	216
query57	1200	1083	1122	1083
query58	207	207	203	203
query59	3935	3738	3812	3738
query60	234	192	211	192
query61	105	96	97	96
query62	886	497	485	485
query63	485	344	354	344
query64	2531	1525	1437	1437
query65	3606	3602	3573	3573
query66	803	371	391	371
query67	18041	17008	16797	16797
query68	8363	668	664	664
query69	580	352	349	349
query70	1837	1605	1529	1529
query71	400	297	319	297
query72	6653	3569	3561	3561
query73	751	326	329	326
query74	6295	5896	5830	5830
query75	4646	3690	3774	3690
query76	4836	1172	1137	1137
query77	680	255	260	255
query78	12991	11833	11557	11557
query79	8842	649	668	649
query80	2024	413	408	408
query81	502	238	235	235
query82	1396	99	96	96
query83	176	133	134	133
query84	262	72	72	72
query85	1220	324	330	324
query86	346	294	305	294
query87	3277	3008	3044	3008
query88	5122	2342	2381	2342
query89	371	289	284	284
query90	1848	215	222	215
query91	176	142	154	142
query92	63	56	51	51
query93	4861	567	552	552
query94	758	222	214	214
query95	1132	1043	1062	1043
query96	661	336	330	330
query97	6519	6344	6451	6344
query98	193	174	173	173
query99	2952	892	900	892
Total cold run time: 314460 ms
Total hot run time: 204839 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.52 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1aa0f54172a1c24f4692b901940edcbd358df4ec, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.03
query3	0.26	0.05	0.04
query4	1.80	0.07	0.06
query5	0.53	0.53	0.51
query6	1.25	0.62	0.61
query7	0.01	0.01	0.00
query8	0.03	0.02	0.03
query9	0.53	0.48	0.48
query10	0.53	0.53	0.54
query11	0.12	0.08	0.09
query12	0.11	0.09	0.10
query13	0.62	0.62	0.61
query14	0.76	0.79	0.78
query15	0.77	0.76	0.76
query16	0.37	0.40	0.35
query17	1.02	0.99	0.98
query18	0.24	0.25	0.26
query19	1.93	1.86	1.87
query20	0.02	0.01	0.01
query21	15.47	0.56	0.55
query22	1.90	1.99	1.87
query23	17.40	0.95	0.84
query24	6.94	0.59	1.15
query25	0.38	0.11	0.04
query26	0.75	0.14	0.16
query27	0.04	0.03	0.04
query28	6.50	0.73	0.73
query29	12.77	2.31	2.34
query30	0.55	0.59	0.52
query31	2.80	0.39	0.37
query32	3.38	0.49	0.49
query33	3.11	3.03	3.08
query34	15.23	4.86	4.81
query35	4.86	4.88	4.88
query36	1.08	1.00	1.00
query37	0.06	0.04	0.05
query38	0.04	0.02	0.03
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.53 s
Total hot run time: 30.52 s

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1aa0f54172a1c24f4692b901940edcbd358df4ec with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@yiguolei yiguolei merged commit 7546783 into apache:branch-2.0 Jun 5, 2024
@zhiqiang-hhhh zhiqiang-hhhh deleted the pick_34832_to_upstream_branch-2.0 branch June 5, 2024 02:06
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zhiqiang-hhhh @qzsee
[BUG] fix scan range boundary handling is incorrect apache#34832 (apa…
2e68cca 
…che#35829)

cherry pick from apache#34832

Co-authored-by: shee <13843187+qzsee@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

kind/test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zhiqiang-hhhh @doris-robot @yiguolei @qzsee