Skip to content

[refactor](pipeline) Delete non-pipeline data sink#35858

Merged
yiguolei merged 4 commits intoapache:masterfrom
Gabriel39:refactor_0604
Jun 4, 2024
Merged

[refactor](pipeline) Delete non-pipeline data sink#35858
yiguolei merged 4 commits intoapache:masterfrom
Gabriel39:refactor_0604

Conversation

@Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Jun 4, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Contributor Author

Gabriel39 commented Jun 4, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

TeamCity be ut coverage result:
Function Coverage: 36.38% (8977/24673)
Line Coverage: 27.89% (73295/262754)
Region Coverage: 27.33% (37955/138863)
Branch Coverage: 23.93% (19270/80518)
Coverage Report: http://coverage.selectdb-in.cc/coverage/05d96598584de4bc13e2c7ea6fe5a9e1af7443a3_05d96598584de4bc13e2c7ea6fe5a9e1af7443a3/report/index.html

yiguolei
yiguolei previously approved these changes Jun 4, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

@Gabriel39
Copy link
Contributor Author

Gabriel39 commented Jun 4, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

TeamCity be ut coverage result:
Function Coverage: 36.39% (8978/24674)
Line Coverage: 27.89% (73293/262754)
Region Coverage: 27.33% (37956/138864)
Branch Coverage: 23.94% (19272/80518)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e793e23f1e6101af80d0806aebf8665aaf726905_e793e23f1e6101af80d0806aebf8665aaf726905/report/index.html

HappenLee
HappenLee approved these changes Jun 4, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

TPC-H: Total hot run time: 41162 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e793e23f1e6101af80d0806aebf8665aaf726905, data reload: false

------ Round 1 ----------------------------------
q1	17618	4443	4289	4289
q2	2018	196	199	196
q3	10456	1232	1270	1232
q4	10183	905	879	879
q5	7486	2751	2823	2751
q6	235	134	136	134
q7	965	619	612	612
q8	9220	2164	2146	2146
q9	9253	6736	6725	6725
q10	9821	3941	3833	3833
q11	432	242	246	242
q12	456	234	243	234
q13	17262	3229	3225	3225
q14	272	216	222	216
q15	513	467	468	467
q16	500	400	394	394
q17	1007	647	680	647
q18	8503	7927	7637	7637
q19	7798	1455	1341	1341
q20	669	326	328	326
q21	5204	4013	3294	3294
q22	407	342	346	342
Total cold run time: 120278 ms
Total hot run time: 41162 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4597	4481	4466	4466
q2	373	265	284	265
q3	3117	2928	2997	2928
q4	1893	1640	1605	1605
q5	5501	5512	5473	5473
q6	232	126	126	126
q7	2167	1832	1802	1802
q8	3249	3345	3378	3345
q9	8664	8615	8656	8615
q10	4089	3886	3773	3773
q11	591	482	500	482
q12	768	580	606	580
q13	16193	3151	3151	3151
q14	303	276	270	270
q15	528	481	488	481
q16	479	431	451	431
q17	1862	1488	1498	1488
q18	8067	7757	7392	7392
q19	1801	1569	1663	1569
q20	3004	1780	1782	1780
q21	9283	4755	4723	4723
q22	617	562	530	530
Total cold run time: 77378 ms
Total hot run time: 55275 ms

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

TPC-DS: Total hot run time: 171136 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e793e23f1e6101af80d0806aebf8665aaf726905, data reload: false

query1	945	379	366	366
query2	6465	2561	2224	2224
query3	6660	211	211	211
query4	19751	17154	17502	17154
query5	4184	471	463	463
query6	258	153	170	153
query7	4582	303	311	303
query8	318	276	285	276
query9	8688	2448	2422	2422
query10	446	292	280	280
query11	10538	10075	9926	9926
query12	135	89	85	85
query13	1632	358	363	358
query14	10127	7420	7250	7250
query15	233	192	175	175
query16	7565	274	268	268
query17	1384	527	537	527
query18	1948	267	268	267
query19	205	174	166	166
query20	92	89	84	84
query21	205	129	130	129
query22	4494	3907	3981	3907
query23	33415	33102	32916	32916
query24	11195	2812	2753	2753
query25	605	348	363	348
query26	1202	153	152	152
query27	2361	331	319	319
query28	7029	2106	2095	2095
query29	899	610	588	588
query30	267	150	151	150
query31	955	777	722	722
query32	90	50	52	50
query33	766	285	267	267
query34	984	475	478	475
query35	750	608	618	608
query36	1110	944	925	925
query37	133	67	70	67
query38	2843	2757	2748	2748
query39	842	789	789	789
query40	211	126	120	120
query41	61	51	51	51
query42	120	94	102	94
query43	602	552	544	544
query44	1244	720	736	720
query45	192	164	161	161
query46	1075	735	733	733
query47	1818	1780	1778	1778
query48	388	292	299	292
query49	1072	398	423	398
query50	791	394	400	394
query51	6923	6705	6818	6705
query52	99	93	87	87
query53	349	281	316	281
query54	896	448	453	448
query55	74	72	72	72
query56	284	252	253	252
query57	1152	1060	1059	1059
query58	243	262	236	236
query59	3327	3112	3142	3112
query60	298	285	266	266
query61	89	89	85	85
query62	626	442	437	437
query63	313	290	292	290
query64	8903	2187	1700	1700
query65	3148	3103	3099	3099
query66	858	323	323	323
query67	15430	14836	14653	14653
query68	4486	537	535	535
query69	452	303	296	296
query70	1106	1069	1071	1069
query71	384	285	273	273
query72	7353	4723	5611	4723
query73	746	334	332	332
query74	5920	5447	5496	5447
query75	3340	2641	2700	2641
query76	2355	933	905	905
query77	461	296	292	292
query78	10275	9704	9619	9619
query79	1585	525	520	520
query80	2495	455	451	451
query81	588	214	216	214
query82	1052	102	101	101
query83	332	172	167	167
query84	267	93	82	82
query85	1066	269	266	266
query86	380	315	323	315
query87	3332	3045	3073	3045
query88	3097	2448	2438	2438
query89	474	396	382	382
query90	1719	183	192	183
query91	123	95	94	94
query92	57	53	49	49
query93	1288	514	509	509
query94	1185	184	252	184
query95	399	315	316	315
query96	584	264	273	264
query97	3184	3018	3009	3009
query98	242	223	215	215
query99	1346	845	855	845
Total cold run time: 268560 ms
Total hot run time: 171136 ms

@doris-robot
Copy link

doris-robot commented Jun 4, 2024

ClickBench: Total hot run time: 30.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e793e23f1e6101af80d0806aebf8665aaf726905, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.06	0.06
query4	1.68	0.09	0.09
query5	0.51	0.50	0.49
query6	1.13	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.54	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.60
query14	0.77	0.79	0.77
query15	0.82	0.82	0.82
query16	0.37	0.36	0.36
query17	1.01	0.99	1.00
query18	0.23	0.24	0.26
query19	1.89	1.71	1.71
query20	0.01	0.01	0.01
query21	15.74	0.66	0.65
query22	4.15	7.19	2.10
query23	18.28	1.42	1.27
query24	2.05	0.21	0.21
query25	0.15	0.09	0.08
query26	0.27	0.17	0.18
query27	0.07	0.07	0.08
query28	13.31	1.01	1.00
query29	13.32	3.43	3.35
query30	0.24	0.05	0.05
query31	2.89	0.40	0.38
query32	3.28	0.47	0.46
query33	2.87	2.88	2.89
query34	16.91	4.39	4.42
query35	4.48	4.48	4.61
query36	0.70	0.48	0.46
query37	0.18	0.15	0.15
query38	0.15	0.14	0.15
query39	0.04	0.03	0.03
query40	0.15	0.16	0.15
query41	0.10	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.24 s
Total hot run time: 30.8 s

@yiguolei yiguolei merged commit 7b24e3c into apache:master Jun 4, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
@Gabriel39 @seawinde
[refactor](pipeline) Delete non-pipeline data sink (apache#35858)
eb18072 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Jun 7, 2024
@Gabriel39 @dataroaring
[refactor](pipeline) Delete non-pipeline data sink (#35858)
c4f5b76 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Gabriel39 @doris-robot @yiguolei @HappenLee @dataroaring