Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](inverted index) fix bkd index file is empty error status #35280

Merged
merged 1 commit into from
May 24, 2024
Merged

[Fix](inverted index) fix bkd index file is empty error status #35280

merged 1 commit into from
May 24, 2024

Conversation

airborne12
Copy link
Member

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49345 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 481bff82d810c0127df3d8a0ef6b4bccb7c3320f, data reload: false

------ Round 1 ----------------------------------
q1	17812	4365	4306	4306
q2	2038	151	142	142
q3	10286	1881	1958	1881
q4	10345	1214	1322	1214
q5	8497	3846	3927	3846
q6	231	129	127	127
q7	2010	1640	1588	1588
q8	9473	2746	2706	2706
q9	13680	10245	10237	10237
q10	8676	3514	3532	3514
q11	418	248	251	248
q12	463	306	304	304
q13	18342	3941	4056	3941
q14	354	331	326	326
q15	512	450	458	450
q16	690	578	563	563
q17	1139	949	918	918
q18	7207	6747	6850	6747
q19	1704	1583	1546	1546
q20	529	292	306	292
q21	4469	4165	4063	4063
q22	487	391	386	386
Total cold run time: 119362 ms
Total hot run time: 49345 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4305	4295	4260	4260
q2	318	231	228	228
q3	4181	4181	4121	4121
q4	2752	2750	2717	2717
q5	7228	7120	7138	7120
q6	238	125	123	123
q7	3158	2834	2846	2834
q8	4340	4431	4447	4431
q9	16825	16805	16659	16659
q10	4260	4280	4242	4242
q11	759	675	684	675
q12	1032	842	871	842
q13	7237	3760	3737	3737
q14	456	434	435	434
q15	501	473	457	457
q16	736	674	705	674
q17	3913	3808	3929	3808
q18	8793	8718	8883	8718
q19	1719	1640	1644	1640
q20	2392	2100	2146	2100
q21	8468	8451	8432	8432
q22	1055	955	913	913
Total cold run time: 84666 ms
Total hot run time: 79165 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (8078/21369)
Line Coverage: 29.46% (65965/223926)
Region Coverage: 28.93% (33965/117403)
Branch Coverage: 24.78% (17426/70326)
Coverage Report: http://coverage.selectdb-in.cc/coverage/481bff82d810c0127df3d8a0ef6b4bccb7c3320f_481bff82d810c0127df3d8a0ef6b4bccb7c3320f/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 203225 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 481bff82d810c0127df3d8a0ef6b4bccb7c3320f, data reload: false

query1	933	384	383	383
query2	6530	2607	2915	2607
query3	6924	212	214	212
query4	20174	17974	18037	17974
query5	19726	6492	6543	6492
query6	286	211	227	211
query7	4151	301	304	301
query8	262	283	248	248
query9	3115	2682	2610	2610
query10	402	287	297	287
query11	11377	10689	10764	10689
query12	127	72	75	72
query13	5575	681	675	675
query14	17590	13367	13250	13250
query15	355	219	229	219
query16	6478	274	259	259
query17	1716	1455	860	860
query18	2311	407	400	400
query19	208	147	147	147
query20	82	78	82	78
query21	190	101	93	93
query22	5174	5074	5053	5053
query23	32347	31861	31880	31861
query24	6921	6514	6615	6514
query25	516	442	417	417
query26	525	158	155	155
query27	1898	292	293	292
query28	6211	2375	2335	2335
query29	2878	2721	2815	2721
query30	246	163	168	163
query31	909	712	735	712
query32	69	63	57	57
query33	399	254	261	254
query34	844	470	484	470
query35	1140	933	888	888
query36	1475	1174	1212	1174
query37	89	60	59	59
query38	3091	2912	2957	2912
query39	1376	1333	1308	1308
query40	203	90	93	90
query41	37	40	36	36
query42	87	87	78	78
query43	728	755	670	670
query44	1150	715	710	710
query45	248	229	226	226
query46	1215	955	971	955
query47	1974	1745	1798	1745
query48	1018	738	723	723
query49	620	366	367	366
query50	869	590	606	590
query51	4754	4684	4705	4684
query52	96	81	76	76
query53	441	321	312	312
query54	2644	2468	2482	2468
query55	85	74	87	74
query56	225	211	220	211
query57	1161	1072	1116	1072
query58	215	205	193	193
query59	4199	3919	3903	3903
query60	214	218	196	196
query61	86	83	86	83
query62	837	493	503	493
query63	489	339	333	333
query64	2384	1516	1359	1359
query65	3642	3558	3532	3532
query66	804	390	377	377
query67	17856	16122	15280	15280
query68	7061	667	668	667
query69	568	353	347	347
query70	1675	1493	1654	1493
query71	399	299	323	299
query72	6479	3448	3428	3428
query73	735	324	324	324
query74	6311	5858	5887	5858
query75	4600	3612	3716	3612
query76	4413	1170	1184	1170
query77	540	256	253	253
query78	12646	11837	12673	11837
query79	8907	659	650	650
query80	2610	387	385	385
query81	525	237	229	229
query82	1309	97	98	97
query83	186	135	132	132
query84	260	68	74	68
query85	1393	296	300	296
query86	360	292	290	290
query87	3190	2982	3044	2982
query88	5141	2381	2386	2381
query89	398	287	285	285
query90	1752	210	210	210
query91	167	135	145	135
query92	58	56	53	53
query93	4164	622	557	557
query94	892	209	201	201
query95	1084	1051	1024	1024
query96	645	340	335	335
query97	6473	6363	6349	6349
query98	193	174	172	172
query99	2735	859	889	859
Total cold run time: 310770 ms
Total hot run time: 203225 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 481bff82d810c0127df3d8a0ef6b4bccb7c3320f, data reload: false

query1	0.02	0.03	0.03
query2	0.07	0.02	0.02
query3	0.24	0.04	0.05
query4	1.80	0.06	0.08
query5	0.54	0.52	0.52
query6	1.23	0.61	0.63
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.47
query10	0.55	0.54	0.54
query11	0.12	0.09	0.09
query12	0.11	0.10	0.09
query13	0.62	0.61	0.61
query14	0.78	0.80	0.77
query15	0.78	0.76	0.76
query16	0.37	0.37	0.34
query17	1.02	1.03	1.00
query18	0.22	0.24	0.23
query19	1.90	1.87	1.81
query20	0.02	0.01	0.01
query21	15.48	0.55	0.55
query22	2.29	2.20	2.15
query23	17.33	0.93	0.97
query24	5.49	1.20	1.01
query25	0.37	0.08	0.05
query26	0.65	0.15	0.16
query27	0.05	0.05	0.03
query28	7.73	0.70	0.78
query29	12.64	2.35	2.34
query30	0.57	0.52	0.55
query31	2.94	0.39	0.38
query32	3.88	0.50	0.50
query33	3.06	3.05	3.06
query34	15.26	4.78	4.80
query35	4.84	4.85	4.84
query36	1.08	1.00	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.15	0.14	0.13
query41	0.07	0.02	0.02
query42	0.02	0.02	0.01
query43	0.02	0.01	0.01
Total cold run time: 104.99 s
Total hot run time: 31.25 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 481bff82d810c0127df3d8a0ef6b4bccb7c3320f with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@xiaokang xiaokang merged commit 54bc272 into apache:branch-2.0 May 24, 2024
23 of 27 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@airborne12
[Fix](inverted index) fix bkd index file is empty error status (apach…
978963e 
…e#35280)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qidaye qidaye qidaye approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.11-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@airborne12 @doris-robot @qidaye @xiaokang