Skip to content

[fix](inverted index)Remove the strong check for parser when creating a table with inverted index (#31391)#31429

Merged
xiaokang merged 1 commit intoapache:branch-2.0from
qidaye:pick_fix_index_property_check
Feb 27, 2024
Merged

[fix](inverted index)Remove the strong check for parser when creating a table with inverted index (#31391)#31429
xiaokang merged 1 commit intoapache:branch-2.0from
qidaye:pick_fix_index_property_check

Conversation

@qidaye
Copy link
Copy Markdown
Contributor

@qidaye qidaye commented Feb 26, 2024

Proposed changes

pick from master #31391

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@qidaye
Copy link
Copy Markdown
Contributor Author

qidaye commented Feb 26, 2024

run buildall

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Feb 26, 2024
@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 49972 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4ba3b1757c1479b3d3f82cb53e5029085631960c, data reload: false

------ Round 1 ----------------------------------
q1	17760	4407	4441	4407
q2	2056	147	139	139
q3	10602	1915	1931	1915
q4	10320	1273	1335	1273
q5	8423	3893	3983	3893
q6	233	120	119	119
q7	2008	1621	1595	1595
q8	9352	2696	2707	2696
q9	10800	10622	10390	10390
q10	8606	3511	3495	3495
q11	424	242	240	240
q12	462	295	301	295
q13	18335	3969	4061	3969
q14	351	329	320	320
q15	505	452	456	452
q16	697	596	592	592
q17	1127	959	960	959
q18	7230	6887	6947	6887
q19	1662	1566	1589	1566
q20	539	312	297	297
q21	4488	4140	4073	4073
q22	496	400	400	400
Total cold run time: 116476 ms
Total hot run time: 49972 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4344	4293	4275	4275
q2	315	225	228	225
q3	4124	4175	4137	4137
q4	2737	2747	2742	2742
q5	7265	7183	7200	7183
q6	234	118	113	113
q7	3216	2844	2837	2837
q8	4357	4491	4461	4461
q9	17066	16906	17100	16906
q10	4236	4292	4237	4237
q11	738	696	682	682
q12	1045	846	847	846
q13	6997	3727	3747	3727
q14	474	423	419	419
q15	500	467	453	453
q16	757	700	680	680
q17	3847	3868	3816	3816
q18	8818	8812	8726	8726
q19	1706	1711	1659	1659
q20	2394	2152	2114	2114
q21	8530	8454	8458	8454
q22	1016	984	952	952
Total cold run time: 84716 ms
Total hot run time: 79644 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 202107 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4ba3b1757c1479b3d3f82cb53e5029085631960c, data reload: false

query1	926	392	378	378
query2	6518	2095	2027	2027
query3	6939	208	211	208
query4	21014	17966	18043	17966
query5	19714	6458	6528	6458
query6	294	217	224	217
query7	4160	301	291	291
query8	272	273	243	243
query9	3070	2685	2595	2595
query10	421	280	312	280
query11	11168	10682	10733	10682
query12	127	79	71	71
query13	5599	659	625	625
query14	17725	13195	13071	13071
query15	384	239	240	239
query16	6473	259	271	259
query17	1747	1455	859	859
query18	2335	397	398	397
query19	198	144	159	144
query20	75	75	76	75
query21	191	93	99	93
query22	5288	5053	5089	5053
query23	32278	32050	32147	32050
query24	6988	6526	6453	6453
query25	512	449	404	404
query26	524	161	157	157
query27	1875	289	285	285
query28	6052	2266	2217	2217
query29	2858	2857	2745	2745
query30	238	163	160	160
query31	908	721	716	716
query32	68	55	56	55
query33	391	251	240	240
query34	863	465	459	459
query35	1122	896	896	896
query36	1892	1357	1584	1357
query37	88	60	62	60
query38	3111	2955	2954	2954
query39	1375	1308	1313	1308
query40	209	90	93	90
query41	35	32	36	32
query42	83	89	81	81
query43	583	632	605	605
query44	1086	720	705	705
query45	237	225	226	225
query46	1261	980	958	958
query47	1873	1724	1696	1696
query48	975	684	658	658
query49	607	383	363	363
query50	857	612	622	612
query51	4758	4742	4652	4652
query52	82	77	71	71
query53	438	313	312	312
query54	2655	2461	2441	2441
query55	87	74	82	74
query56	230	211	201	201
query57	1267	1037	1151	1037
query58	199	190	188	188
query59	3389	3220	3067	3067
query60	211	186	182	182
query61	83	86	80	80
query62	836	504	491	491
query63	479	325	324	324
query64	2517	1470	1425	1425
query65	3636	3583	3578	3578
query66	760	361	358	358
query67	17094	15519	16093	15519
query68	8199	662	672	662
query69	571	331	336	331
query70	1765	1772	1784	1772
query71	407	302	312	302
query72	6543	3421	3469	3421
query73	722	325	318	318
query74	6258	5919	5846	5846
query75	4515	3650	3747	3650
query76	4663	1184	1214	1184
query77	566	262	248	248
query78	12545	11830	12110	11830
query79	7989	657	639	639
query80	2251	389	390	389
query81	534	232	239	232
query82	1043	106	93	93
query83	186	130	127	127
query84	254	69	68	68
query85	1387	276	277	276
query86	396	348	358	348
query87	3220	3085	3044	3044
query88	4992	2326	2345	2326
query89	353	280	287	280
query90	1754	198	208	198
query91	162	115	128	115
query92	61	53	49	49
query93	4097	598	615	598
query94	885	203	211	203
query95	1098	1071	1078	1071
query96	642	340	324	324
query97	6545	6391	6383	6383
query98	177	172	166	166
query99	2966	939	842	842
Total cold run time: 310054 ms
Total hot run time: 202107 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 31.07 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4ba3b1757c1479b3d3f82cb53e5029085631960c, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.25	0.06	0.05
query4	1.82	0.06	0.06
query5	0.54	0.53	0.51
query6	1.25	0.61	0.62
query7	0.01	0.01	0.01
query8	0.02	0.02	0.02
query9	0.51	0.49	0.48
query10	0.54	0.54	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.77	0.79	0.80
query15	0.79	0.76	0.76
query16	0.37	0.37	0.36
query17	1.01	1.00	1.01
query18	0.21	0.25	0.24
query19	1.84	1.88	1.86
query20	0.02	0.01	0.01
query21	15.49	0.57	0.55
query22	1.73	2.60	1.85
query23	17.36	0.81	0.86
query24	4.64	1.02	1.28
query25	1.56	0.12	0.11
query26	0.22	0.14	0.14
query27	0.10	0.11	0.11
query28	8.01	0.73	0.70
query29	12.63	2.33	2.23
query30	0.57	0.54	0.57
query31	2.81	0.38	0.40
query32	3.41	0.49	0.49
query33	3.07	3.08	3.10
query34	15.29	4.81	4.80
query35	4.84	4.87	4.87
query36	1.04	1.04	1.01
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.15	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.02 s
Total hot run time: 31.07 s

@doris-robot
Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4ba3b1757c1479b3d3f82cb53e5029085631960c with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@xiaokang xiaokang merged commit b8193dc into apache:branch-2.0 Feb 27, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants