Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug](join) fix broadcast join running when hash table build not finished #37792

Merged
merged 1 commit into from
Jul 15, 2024

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jul 15, 2024

in pr #37643
wants fix the bug of join which not build hash table, but running early and not wait the finished signal.
But this may be a normal phenomenon, as it should allows all sinks to run when the source operator have closed,
so here return eof status directly when signal == false.

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39993 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac2460a74c3a1580b47f4eddc32711cfa4851ffd, data reload: false

------ Round 1 ----------------------------------
q1	18032	4439	4385	4385
q2	2262	190	188	188
q3	10479	1213	1081	1081
q4	10186	788	858	788
q5	7551	2681	2638	2638
q6	222	136	137	136
q7	963	604	590	590
q8	9213	2073	2073	2073
q9	8762	6581	6559	6559
q10	8760	3836	3792	3792
q11	466	242	255	242
q12	405	228	224	224
q13	18815	2979	3007	2979
q14	272	233	236	233
q15	531	483	486	483
q16	491	388	375	375
q17	969	684	627	627
q18	8070	7487	7450	7450
q19	7204	1476	1454	1454
q20	713	329	337	329
q21	4909	3090	3237	3090
q22	349	277	277	277
Total cold run time: 119624 ms
Total hot run time: 39993 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4418	4221	4231	4221
q2	378	258	291	258
q3	3045	2750	2761	2750
q4	1870	1618	1652	1618
q5	5307	5328	5326	5326
q6	223	128	132	128
q7	2113	1709	1705	1705
q8	3178	3346	3295	3295
q9	8429	8391	8473	8391
q10	3866	3726	3719	3719
q11	606	486	499	486
q12	779	626	576	576
q13	17561	3006	3001	3001
q14	294	268	268	268
q15	530	478	479	478
q16	460	406	419	406
q17	1769	1487	1451	1451
q18	7684	7450	7317	7317
q19	1894	1475	1544	1475
q20	2003	1787	1787	1787
q21	4794	4646	4721	4646
q22	594	509	500	500
Total cold run time: 71795 ms
Total hot run time: 53802 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173538 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac2460a74c3a1580b47f4eddc32711cfa4851ffd, data reload: false

query1	911	382	367	367
query2	6479	1839	1732	1732
query3	6677	204	213	204
query4	24679	17475	17363	17363
query5	4187	503	494	494
query6	268	174	183	174
query7	4590	291	300	291
query8	230	201	200	200
query9	8519	2369	2361	2361
query10	444	295	286	286
query11	10885	10031	10070	10031
query12	131	86	84	84
query13	1648	373	378	373
query14	10213	8047	8017	8017
query15	212	167	174	167
query16	7897	325	325	325
query17	1809	550	564	550
query18	2026	287	288	287
query19	196	152	156	152
query20	90	80	89	80
query21	211	128	125	125
query22	4368	4089	4066	4066
query23	33630	33189	33291	33189
query24	12034	2977	2867	2867
query25	671	391	390	390
query26	1754	150	149	149
query27	2699	271	272	271
query28	7176	1982	1968	1968
query29	1060	661	623	623
query30	290	151	150	150
query31	957	736	770	736
query32	104	61	59	59
query33	782	320	307	307
query34	901	491	490	490
query35	696	586	607	586
query36	1104	949	910	910
query37	152	83	91	83
query38	2852	2754	2781	2754
query39	856	827	838	827
query40	288	126	126	126
query41	48	48	49	48
query42	121	99	106	99
query43	507	460	467	460
query44	1187	754	751	751
query45	198	166	159	159
query46	1098	734	715	715
query47	1868	1798	1799	1798
query48	375	298	301	298
query49	1185	431	422	422
query50	792	398	408	398
query51	6836	6782	6734	6734
query52	104	95	100	95
query53	372	301	294	294
query54	1093	469	463	463
query55	75	73	76	73
query56	308	281	308	281
query57	1168	1083	1053	1053
query58	258	244	255	244
query59	2856	2581	2664	2581
query60	319	273	287	273
query61	98	96	105	96
query62	826	650	661	650
query63	315	303	289	289
query64	10531	2218	1711	1711
query65	3382	3133	3158	3133
query66	1316	334	329	329
query67	15656	15041	15163	15041
query68	8827	554	539	539
query69	778	448	356	356
query70	1193	1122	1155	1122
query71	521	279	313	279
query72	8247	6074	5531	5531
query73	1704	329	321	321
query74	6050	5704	5662	5662
query75	5013	2695	2653	2653
query76	5108	942	979	942
query77	760	318	306	306
query78	9736	8964	8916	8916
query79	10045	522	512	512
query80	1119	474	482	474
query81	567	217	214	214
query82	759	135	131	131
query83	332	164	163	163
query84	283	86	92	86
query85	1385	310	326	310
query86	401	311	331	311
query87	3299	3091	3158	3091
query88	4786	2487	2459	2459
query89	537	389	371	371
query90	2076	197	193	193
query91	130	100	103	100
query92	65	50	49	49
query93	7031	505	502	502
query94	1356	218	218	218
query95	409	319	317	317
query96	615	272	272	272
query97	3199	3075	2986	2986
query98	218	204	207	204
query99	1613	1263	1246	1246
Total cold run time: 305730 ms
Total hot run time: 173538 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac2460a74c3a1580b47f4eddc32711cfa4851ffd, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.08
query5	0.50	0.48	0.48
query6	1.14	0.72	0.74
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.54	0.49	0.50
query10	0.54	0.52	0.51
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.76	0.80	0.78
query15	0.85	0.81	0.82
query16	0.37	0.36	0.37
query17	0.99	1.01	0.96
query18	0.22	0.21	0.22
query19	1.83	1.67	1.67
query20	0.01	0.01	0.01
query21	15.40	0.73	0.66
query22	4.03	6.57	2.58
query23	18.25	1.40	1.32
query24	2.05	0.24	0.22
query25	0.16	0.08	0.09
query26	0.29	0.20	0.20
query27	0.45	0.23	0.23
query28	13.34	1.02	1.02
query29	12.65	3.42	3.37
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.27	0.49	0.46
query33	2.92	2.97	2.88
query34	17.04	4.47	4.36
query35	4.41	4.43	4.51
query36	0.65	0.49	0.48
query37	0.19	0.16	0.16
query38	0.16	0.14	0.14
query39	0.04	0.04	0.03
query40	0.14	0.12	0.11
query41	0.08	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.47 s
Total hot run time: 31.37 s

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 15, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@Gabriel39 Gabriel39 merged commit 7919f07 into apache:master Jul 15, 2024
26 of 30 checks passed
yiguolei pushed a commit that referenced this pull request Jul 16, 2024
…uild not finished (#37844)

cherry-pick from master #37792

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
…shed (apache#37792)

in pr https://github.com/apache/doris/pull/37643/files
wants fix the bug of join which not build hash table, but running early
and not wait the finished signal.
But this may be a normal phenomenon, as it should allows all sinks to
run when the source operator have closed,
so here return eof status directly when signal == false.
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
…shed (#37792)

in pr https://github.com/apache/doris/pull/37643/files
wants fix the bug of join which not build hash table, but running early
and not wait the finished signal.
But this may be a normal phenomenon, as it should allows all sinks to
run when the source operator have closed,
so here return eof status directly when signal == false.
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants