Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](shuffle) Fix local exchange dependency blocking #38151

Merged
merged 1 commit into from
Jul 22, 2024

Conversation

Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Jul 19, 2024

Proposed changes

If receiver is closed now, local exchange channel dependency is still blocked which is not correct.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41747 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b7992dcabae8512dd879a84148e0bbe38e11fdc, data reload: false

------ Round 1 ----------------------------------
q1	17694	4848	4734	4734
q2	2005	191	209	191
q3	10438	1251	1252	1251
q4	10198	839	824	824
q5	7573	2872	2860	2860
q6	242	136	133	133
q7	1033	599	610	599
q8	9224	2262	2297	2262
q9	8876	6982	6900	6900
q10	9035	3970	3921	3921
q11	459	238	244	238
q12	478	227	225	225
q13	18687	2946	2971	2946
q14	284	229	229	229
q15	537	485	469	469
q16	520	375	389	375
q17	1017	684	736	684
q18	8089	7521	7459	7459
q19	8502	1559	1434	1434
q20	726	325	312	312
q21	5105	3422	4047	3422
q22	361	279	290	279
Total cold run time: 121083 ms
Total hot run time: 41747 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4840	4789	4894	4789
q2	423	259	267	259
q3	3347	3043	3045	3043
q4	2081	1696	1779	1696
q5	5641	5742	5692	5692
q6	255	131	133	131
q7	2316	1869	1818	1818
q8	3534	3697	3706	3697
q9	8971	8836	8688	8688
q10	4192	3940	3967	3940
q11	607	505	476	476
q12	840	669	632	632
q13	16031	3165	3138	3138
q14	312	272	275	272
q15	528	482	468	468
q16	497	431	423	423
q17	1967	1646	1588	1588
q18	8091	7949	7884	7884
q19	1889	1682	1668	1668
q20	2190	1863	1822	1822
q21	5221	5017	5040	5017
q22	619	508	497	497
Total cold run time: 74392 ms
Total hot run time: 57638 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174970 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b7992dcabae8512dd879a84148e0bbe38e11fdc, data reload: false

query1	912	363	360	360
query2	6365	1827	1800	1800
query3	6633	208	220	208
query4	22935	17466	17356	17356
query5	3630	473	484	473
query6	260	185	175	175
query7	4587	309	303	303
query8	263	201	190	190
query9	8613	2547	2540	2540
query10	451	293	280	280
query11	10445	10049	10146	10049
query12	116	91	82	82
query13	1647	401	391	391
query14	10213	7722	7737	7722
query15	239	162	171	162
query16	7801	506	449	449
query17	1557	566	539	539
query18	2003	270	266	266
query19	193	153	150	150
query20	87	80	77	77
query21	205	125	125	125
query22	4561	4188	4061	4061
query23	33689	33450	33620	33450
query24	10884	2881	2892	2881
query25	596	388	388	388
query26	1093	157	153	153
query27	2518	289	286	286
query28	6800	2114	2091	2091
query29	938	642	649	642
query30	250	148	150	148
query31	955	787	770	770
query32	93	54	52	52
query33	741	347	334	334
query34	944	481	495	481
query35	844	749	755	749
query36	1125	954	980	954
query37	147	90	87	87
query38	2922	2850	2841	2841
query39	879	838	861	838
query40	203	122	122	122
query41	47	46	40	40
query42	107	101	102	101
query43	511	482	456	456
query44	1199	760	789	760
query45	189	160	155	155
query46	1088	761	736	736
query47	1861	1779	1761	1761
query48	394	313	323	313
query49	845	405	422	405
query50	772	393	391	391
query51	6944	6832	6743	6743
query52	104	93	94	93
query53	363	289	287	287
query54	918	452	444	444
query55	79	82	77	77
query56	292	280	265	265
query57	1132	1051	1048	1048
query58	242	241	249	241
query59	2843	2623	2622	2622
query60	307	273	282	273
query61	94	92	94	92
query62	795	641	623	623
query63	316	289	287	287
query64	9402	2187	7410	2187
query65	3134	3077	3102	3077
query66	732	329	323	323
query67	15633	15082	15277	15082
query68	5410	553	548	548
query69	672	451	372	372
query70	1142	1163	1089	1089
query71	453	282	273	273
query72	7701	5725	5590	5590
query73	772	331	334	331
query74	6031	5633	5616	5616
query75	3568	2696	2668	2668
query76	3653	950	973	950
query77	639	302	291	291
query78	9581	9021	8937	8937
query79	3061	535	533	533
query80	2231	521	464	464
query81	598	215	218	215
query82	1356	129	128	128
query83	299	169	165	165
query84	275	91	84	84
query85	1478	306	317	306
query86	471	316	331	316
query87	3330	3080	3090	3080
query88	3981	2509	2517	2509
query89	466	369	378	369
query90	1757	190	192	190
query91	128	98	98	98
query92	61	46	48	46
query93	3154	521	517	517
query94	1173	291	288	288
query95	416	322	324	322
query96	610	298	296	296
query97	3221	3012	3009	3009
query98	218	205	204	204
query99	1564	1269	1259	1259
Total cold run time: 279150 ms
Total hot run time: 174970 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b7992dcabae8512dd879a84148e0bbe38e11fdc, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.08	0.08
query5	0.49	0.49	0.47
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.54	0.48	0.49
query10	0.55	0.52	0.53
query11	0.15	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.76	0.78	0.78
query15	0.85	0.81	0.82
query16	0.36	0.37	0.36
query17	0.96	1.01	0.98
query18	0.22	0.21	0.23
query19	1.75	1.71	1.82
query20	0.01	0.01	0.01
query21	15.40	0.74	0.67
query22	4.54	7.78	1.78
query23	18.28	1.31	1.34
query24	2.10	0.23	0.22
query25	0.15	0.08	0.09
query26	0.30	0.21	0.20
query27	0.45	0.24	0.23
query28	13.26	1.02	0.99
query29	12.63	3.23	3.27
query30	0.26	0.06	0.06
query31	2.89	0.38	0.39
query32	3.28	0.48	0.48
query33	2.84	2.94	2.86
query34	17.02	4.35	4.31
query35	4.40	4.42	4.38
query36	0.65	0.48	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.09	0.04	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.76 s
Total hot run time: 30.32 s

yiguolei pushed a commit that referenced this pull request Jul 19, 2024
## Proposed changes

pick #38151

<!--Describe your changes.-->
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@Gabriel39 Gabriel39 merged commit 20e5dc7 into apache:master Jul 22, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
If receiver is closed now, local exchange channel dependency is still
blocked which is not correct.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants