Skip to content

[pipelineX](debug) Refactor code and complete debug string #31733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Mar 5, 2024
Merged

[pipelineX](debug) Refactor code and complete debug string #31733

yiguolei merged 1 commit into from
Mar 5, 2024

Conversation

@Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Mar 4, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Mar 4, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2024

PR approved by anyone and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

@Gabriel39
Copy link
Contributor Author

Gabriel39 commented Mar 4, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Mar 4, 2024

TeamCity be ut coverage result:
Function Coverage: 35.42% (8553/24149)
Line Coverage: 27.19% (69402/255295)
Region Coverage: 26.40% (36002/136368)
Branch Coverage: 23.24% (18392/79130)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2dc2b43266ced51c68acced9bcc2b3236f14c17d_2dc2b43266ced51c68acced9bcc2b3236f14c17d/report/index.html

@doris-robot
Copy link

doris-robot commented Mar 4, 2024

TPC-H: Total hot run time: 32471 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2dc2b43266ced51c68acced9bcc2b3236f14c17d, data reload: false

------ Round 1 ----------------------------------
q1	17688	4516	4459	4459
q2	2103	214	204	204
q3	10416	1124	1117	1117
q4	4696	1223	1239	1223
q5	7619	2049	2057	2049
q6	165	111	105	105
q7	1124	1077	1049	1049
q8	9505	1061	1098	1061
q9	6299	4814	4850	4814
q10	7938	2085	2024	2024
q11	518	315	339	315
q12	845	441	434	434
q13	18125	2997	2941	2941
q14	319	312	293	293
q15	412	362	356	356
q16	549	530	536	530
q17	773	327	323	323
q18	6090	5062	4962	4962
q19	1177	423	425	423
q20	638	437	447	437
q21	6771	2966	2891	2891
q22	536	479	461	461
Total cold run time: 104306 ms
Total hot run time: 32471 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4622	4628	4594	4594
q2	300	231	224	224
q3	2204	1864	1922	1864
q4	1642	1379	1371	1371
q5	3292	3092	3016	3016
q6	159	100	107	100
q7	1943	1821	1913	1821
q8	1803	1676	1643	1643
q9	5600	5386	5342	5342
q10	2715	2369	2378	2369
q11	505	407	405	405
q12	752	648	665	648
q13	3232	2868	2757	2757
q14	292	287	272	272
q15	378	356	340	340
q16	489	509	502	502
q17	1105	790	763	763
q18	6086	5474	5485	5474
q19	1138	681	642	642
q20	1604	1495	1523	1495
q21	4818	4345	4141	4141
q22	661	612	593	593
Total cold run time: 45340 ms
Total hot run time: 40376 ms

@doris-robot
Copy link

doris-robot commented Mar 4, 2024

TPC-DS: Total hot run time: 186494 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2dc2b43266ced51c68acced9bcc2b3236f14c17d, data reload: false

query1	981	397	378	378
query2	7813	2394	2318	2318
query3	6738	298	298	298
query4	27034	23282	23076	23076
query5	4967	645	671	645
query6	349	283	292	283
query7	4694	475	470	470
query8	307	273	255	255
query9	8676	3407	3427	3407
query10	563	344	361	344
query11	18349	17353	17531	17353
query12	206	142	140	140
query13	1739	644	660	644
query14	9492	7460	8433	7460
query15	266	251	247	247
query16	7363	391	380	380
query17	939	683	639	639
query18	2001	430	405	405
query19	268	234	231	231
query20	139	130	135	130
query21	235	164	165	164
query22	5307	5166	5108	5108
query23	25194	24414	24656	24414
query24	7436	2114	2114	2114
query25	756	572	560	560
query26	793	265	255	255
query27	2421	490	484	484
query28	3208	2796	2760	2760
query29	1041	760	728	728
query30	406	202	195	195
query31	1497	869	819	819
query32	131	95	97	95
query33	678	385	387	385
query34	970	644	666	644
query35	993	894	884	884
query36	1278	1202	1191	1191
query37	133	106	112	106
query38	3383	3176	3189	3176
query39	1207	1142	1132	1132
query40	241	166	164	164
query41	86	82	85	82
query42	158	160	165	160
query43	591	594	568	568
query44	1263	1038	1034	1034
query45	229	222	220	220
query46	878	783	783	783
query47	1726	1720	1685	1685
query48	594	509	510	509
query49	1141	493	463	463
query50	684	549	555	549
query51	9260	9109	8902	8902
query52	145	140	141	140
query53	333	326	317	317
query54	430	349	344	344
query55	183	128	129	128
query56	362	334	355	334
query57	1103	1058	1075	1058
query58	360	344	333	333
query59	3531	3355	3356	3355
query60	395	407	376	376
query61	165	163	163	163
query62	571	417	402	402
query63	319	320	337	320
query64	4254	1862	1772	1772
query65	2425	2420	2333	2333
query66	1089	382	390	382
query67	15479	15411	15266	15266
query68	4292	712	729	712
query69	671	558	567	558
query70	1629	1581	1519	1519
query71	488	416	415	415
query72	2808	2329	2374	2329
query73	653	479	471	471
query74	7226	7610	7426	7426
query75	3106	2692	2656	2656
query76	2453	625	648	625
query77	551	415	404	404
query78	6944	6659	6466	6466
query79	1094	742	741	741
query80	707	600	596	596
query81	484	262	256	256
query82	170	131	134	131
query83	251	227	233	227
query84	305	129	126	126
query85	869	555	488	488
query86	426	399	411	399
query87	3557	3336	3280	3280
query88	3720	3446	3465	3446
query89	436	419	421	419
query90	1549	226	220	220
query91	208	199	203	199
query92	85	79	79	79
query93	1046	709	712	709
query94	590	269	275	269
query95	591	518	527	518
query96	638	385	395	385
query97	3040	3002	2918	2918
query98	331	302	296	296
query99	884	772	752	752
Total cold run time: 259748 ms
Total hot run time: 186494 ms

@doris-robot
Copy link

doris-robot commented Mar 4, 2024

ClickBench: Total hot run time: 26.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2dc2b43266ced51c68acced9bcc2b3236f14c17d, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.03	0.03
query3	0.27	0.07	0.06
query4	1.59	0.08	0.09
query5	0.50	0.50	0.53
query6	1.57	0.99	0.98
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.72	0.68	0.66
query10	0.71	0.70	0.69
query11	0.20	0.15	0.14
query12	0.19	0.15	0.16
query13	0.51	0.47	0.46
query14	0.73	0.70	0.72
query15	0.92	0.96	0.90
query16	0.40	0.43	0.39
query17	1.17	1.18	1.22
query18	0.33	0.32	0.32
query19	2.23	2.06	2.07
query20	0.03	0.02	0.02
query21	15.39	0.94	0.97
query22	0.57	0.57	0.56
query23	13.64	0.63	0.63
query24	0.62	0.18	0.18
query25	0.05	0.04	0.05
query26	0.21	0.18	0.18
query27	0.05	0.05	0.04
query28	1.23	0.75	0.73
query29	12.67	4.56	4.70
query30	0.85	0.79	0.75
query31	0.87	0.41	0.39
query32	3.24	0.39	0.38
query33	2.57	2.19	2.28
query34	2.23	2.29	2.26
query35	2.26	2.24	2.25
query36	0.99	1.01	0.99
query37	0.08	0.06	0.06
query38	0.06	0.04	0.06
query39	0.04	0.03	0.03
query40	0.19	0.19	0.17
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 70.3 s
Total hot run time: 26.88 s

yiguolei
yiguolei reviewed Mar 4, 2024
View reviewed changes
be/src/vec/runtime/vdata_stream_recvr.h

Status get_batch(Block* block, bool* eos) override {
std::lock_guard<std::mutex> l(_lock); // protect _block_queue
DCHECK(_is_cancelled || !_block_queue.empty() || _num_remaining_senders == 0)
Copy link
Contributor

@yiguolei yiguolei Mar 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这么改比dcheck的好处是啥啊

@yiguolei yiguolei merged commit 3ba0485 into apache:master Mar 5, 2024
yiguolei pushed a commit that referenced this pull request Mar 6, 2024
@Gabriel39 @Doris-Extras
[pipelineX](debug) Refactor code and complete debug string (#31733)
8085634
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Gabriel39 @doris-robot @BiteTheDDDDt @yiguolei