Skip to content

Conversation

@BePPPower
Copy link
Contributor

@BePPPower BePPPower commented Mar 13, 2024

Proposed changes

Issue Number: #31719

  1. nereids supports trino-connector catalog
  2. Fix some trino-connector's bug when reading data

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@BePPPower
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 36548 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc1938f6b6cd871a6ce4c43b7b0cb140162fca6c, data reload: false

------ Round 1 ----------------------------------
q1	17674	4181	4165	4165
q2	2022	150	146	146
q3	10569	931	900	900
q4	4687	698	688	688
q5	7611	2806	2763	2763
q6	183	123	120	120
q7	1195	815	799	799
q8	9481	2047	1997	1997
q9	7296	6477	6449	6449
q10	8165	2493	2548	2493
q11	427	222	216	216
q12	792	308	290	290
q13	17958	2976	2999	2976
q14	286	247	246	246
q15	493	451	460	451
q16	487	400	395	395
q17	940	554	521	521
q18	6683	5808	5812	5808
q19	1563	1398	1454	1398
q20	554	277	267	267
q21	7446	3170	3277	3170
q22	811	290	299	290
Total cold run time: 107323 ms
Total hot run time: 36548 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4072	4116	4071	4071
q2	318	226	230	226
q3	2994	2828	2760	2760
q4	1831	1536	1563	1536
q5	5179	5251	5228	5228
q6	193	118	115	115
q7	2232	1842	1841	1841
q8	3149	3271	3256	3256
q9	8571	8491	8545	8491
q10	6156	3663	3720	3663
q11	531	446	434	434
q12	736	522	533	522
q13	13594	2846	2826	2826
q14	276	244	253	244
q15	484	450	437	437
q16	433	404	396	396
q17	1717	1496	1446	1446
q18	7574	7143	7127	7127
q19	5889	1535	1472	1472
q20	1896	1715	1694	1694
q21	4652	4663	4638	4638
q22	514	452	480	452
Total cold run time: 72991 ms
Total hot run time: 52875 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186490 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc1938f6b6cd871a6ce4c43b7b0cb140162fca6c, data reload: false

query1	934	349	340	340
query2	7780	1960	1999	1960
query3	6748	230	213	213
query4	31102	20695	20658	20658
query5	4345	391	401	391
query6	275	173	175	173
query7	4615	293	282	282
query8	225	177	172	172
query9	9886	2266	2246	2246
query10	463	238	241	238
query11	14657	14174	14110	14110
query12	136	88	91	88
query13	1630	423	417	417
query14	11591	10824	10346	10346
query15	257	200	189	189
query16	8105	262	264	262
query17	1932	563	549	549
query18	2090	294	268	268
query19	277	155	151	151
query20	90	85	84	84
query21	196	127	126	126
query22	4691	4515	4428	4428
query23	31687	31229	30953	30953
query24	10850	2777	2822	2777
query25	619	383	385	383
query26	1263	176	155	155
query27	2740	340	349	340
query28	7842	1859	1845	1845
query29	904	664	639	639
query30	314	157	148	148
query31	965	724	717	717
query32	95	60	56	56
query33	769	254	252	252
query34	1078	461	478	461
query35	813	603	599	599
query36	995	863	852	852
query37	131	79	80	79
query38	3278	3184	3179	3179
query39	1439	1372	1374	1372
query40	209	113	111	111
query41	50	47	50	47
query42	102	98	98	98
query43	487	447	459	447
query44	1271	714	712	712
query45	257	261	268	261
query46	1060	694	683	683
query47	1660	1604	1606	1604
query48	459	370	371	370
query49	1142	347	342	342
query50	758	365	369	365
query51	6790	6829	6785	6785
query52	110	91	88	88
query53	340	270	270	270
query54	318	238	232	232
query55	83	80	77	77
query56	240	257	235	235
query57	1082	1028	1018	1018
query58	230	201	206	201
query59	2747	2635	2440	2440
query60	269	257	262	257
query61	116	112	104	104
query62	592	392	393	392
query63	295	274	270	270
query64	5255	3822	3720	3720
query65	3064	3003	2983	2983
query66	829	318	350	318
query67	14894	14231	14444	14231
query68	6223	519	525	519
query69	617	381	392	381
query70	1243	1135	1185	1135
query71	498	266	273	266
query72	6513	2664	2468	2468
query73	719	326	320	320
query74	7825	6565	6555	6555
query75	9140	7999	8186	7999
query76	4320	889	879	879
query77	626	255	248	248
query78	10321	9701	9703	9701
query79	8009	529	523	523
query80	1931	399	377	377
query81	561	212	205	205
query82	1654	194	209	194
query83	318	144	144	144
query84	282	78	75	75
query85	1639	326	313	313
query86	490	300	298	298
query87	3440	3219	3267	3219
query88	4993	2281	2287	2281
query89	532	374	360	360
query90	1996	174	173	173
query91	164	131	124	124
query92	59	50	49	49
query93	6367	509	498	498
query94	1203	178	178	178
query95	434	341	337	337
query96	602	276	278	276
query97	4419	4258	4301	4258
query98	235	216	213	213
query99	1092	768	765	765
Total cold run time: 310981 ms
Total hot run time: 186490 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc1938f6b6cd871a6ce4c43b7b0cb140162fca6c, data reload: false

query1	0.04	0.04	0.04
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.68	0.09	0.09
query5	0.53	0.52	0.52
query6	1.19	0.62	0.62
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.48	0.48
query10	0.52	0.54	0.62
query11	0.11	0.09	0.10
query12	0.11	0.09	0.09
query13	0.58	0.57	0.58
query14	0.75	0.75	0.75
query15	0.80	0.78	0.78
query16	0.38	0.38	0.41
query17	1.02	1.02	1.02
query18	0.19	0.27	0.23
query19	1.79	1.72	1.69
query20	0.01	0.01	0.01
query21	15.43	0.57	0.56
query22	2.69	2.48	1.59
query23	17.30	0.66	0.73
query24	3.24	0.25	1.00
query25	0.17	0.25	0.10
query26	0.42	0.12	0.12
query27	0.04	0.03	0.03
query28	11.70	0.85	0.84
query29	12.63	3.13	3.22
query30	0.72	0.61	0.59
query31	2.79	0.32	0.32
query32	3.41	0.44	0.43
query33	2.79	2.71	2.82
query34	16.26	4.30	4.33
query35	4.30	4.31	4.29
query36	1.08	1.02	1.00
query37	0.07	0.05	0.06
query38	0.04	0.04	0.03
query39	0.03	0.01	0.01
query40	0.18	0.14	0.14
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 106.03 s
Total hot run time: 29.24 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dc1938f6b6cd871a6ce4c43b7b0cb140162fca6c with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.1 seconds inserted 10000000 Rows, about 709K ops/s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 14, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit aa53eb3 into apache:master Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants