Skip to content

[fix](hashjoin) check cancel status in building hash table loop to improve cancel efficiency#40833

Closed
jacktengg wants to merge 1 commit intoapache:masterfrom
jacktengg:fix-0913-hash-build-cancel
Closed

[fix](hashjoin) check cancel status in building hash table loop to improve cancel efficiency#40833
jacktengg wants to merge 1 commit intoapache:masterfrom
jacktengg:fix-0913-hash-build-cancel

Conversation

@jacktengg
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

The query cancel status was not checked during the hash table build loop, and it will result in the query not canceled in time.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch from a80272d to a7cc005 Compare September 13, 2024 16:07
@jacktengg
Copy link
Contributor Author

run buildall

@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch 2 times, most recently from 5280f07 to d417a3c Compare September 13, 2024 16:25
@jacktengg
Copy link
Contributor Author

run buildall

@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch 3 times, most recently from 8c61dd3 to 0b05b68 Compare September 13, 2024 16:49
@jacktengg
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 43504 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0b05b68e14a01067765fe6035e3888d6887e9402, data reload: false

------ Round 1 ----------------------------------
q1	18046	7582	7390	7390
q2	2492	200	199	199
q3	10766	1359	1435	1359
q4	10163	994	1016	994
q5	7726	3244	3188	3188
q6	245	161	159	159
q7	1056	633	620	620
q8	9465	2078	2042	2042
q9	6862	6350	6373	6350
q10	7039	2542	2531	2531
q11	433	251	259	251
q12	410	230	227	227
q13	17749	3062	3081	3062
q14	289	251	262	251
q15	581	537	530	530
q16	525	449	438	438
q17	1027	963	963	963
q18	7538	6824	6868	6824
q19	1401	1261	1259	1259
q20	620	354	328	328
q21	3958	3556	3600	3556
q22	1086	983	1022	983
Total cold run time: 109477 ms
Total hot run time: 43504 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7176	7230	7239	7230
q2	350	235	244	235
q3	2941	2938	2930	2930
q4	1982	1935	1973	1935
q5	5523	5479	5552	5479
q6	235	154	152	152
q7	2039	1678	1681	1678
q8	3262	3332	3363	3332
q9	8486	8469	8498	8469
q10	3423	3489	3475	3475
q11	570	479	473	473
q12	774	557	598	557
q13	6638	3070	3112	3070
q14	313	271	271	271
q15	585	526	531	526
q16	505	465	468	465
q17	1786	1724	1713	1713
q18	7970	7673	7580	7580
q19	1735	1720	1711	1711
q20	2052	1819	1814	1814
q21	5742	5456	5599	5456
q22	1089	1022	991	991
Total cold run time: 65176 ms
Total hot run time: 59542 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.90% (9467/25659)
Line Coverage: 28.25% (77817/275475)
Region Coverage: 27.64% (40198/145425)
Branch Coverage: 24.25% (20422/84214)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0b05b68e14a01067765fe6035e3888d6887e9402_0b05b68e14a01067765fe6035e3888d6887e9402/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 196032 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0b05b68e14a01067765fe6035e3888d6887e9402, data reload: false

query1	919	401	393	393
query2	6461	1852	1823	1823
query3	6660	211	220	211
query4	29195	24096	24126	24096
query5	5240	548	543	543
query6	264	182	163	163
query7	4603	315	318	315
query8	280	228	227	227
query9	8472	2612	2599	2599
query10	455	315	286	286
query11	16035	15518	15590	15518
query12	163	106	108	106
query13	1692	404	394	394
query14	10501	6957	7087	6957
query15	221	172	173	172
query16	7633	485	501	485
query17	1387	588	573	573
query18	1987	292	298	292
query19	195	153	152	152
query20	125	122	111	111
query21	213	110	106	106
query22	4667	4364	4428	4364
query23	34409	33906	33732	33732
query24	10320	3097	3142	3097
query25	688	415	426	415
query26	1426	173	161	161
query27	2899	287	290	287
query28	6949	2127	2106	2106
query29	1002	427	430	427
query30	298	162	156	156
query31	997	805	805	805
query32	110	60	59	59
query33	734	318	303	303
query34	906	483	487	483
query35	884	765	764	764
query36	1075	884	893	884
query37	174	84	85	84
query38	4016	3919	3976	3919
query39	1439	1443	1437	1437
query40	290	122	118	118
query41	52	49	48	48
query42	125	101	99	99
query43	500	446	447	446
query44	1242	814	784	784
query45	203	172	177	172
query46	1120	814	799	799
query47	1890	1802	1811	1802
query48	367	301	300	300
query49	1126	472	470	470
query50	933	435	443	435
query51	6989	6923	7028	6923
query52	105	91	91	91
query53	266	190	189	189
query54	813	476	470	470
query55	79	77	78	77
query56	304	286	281	281
query57	1228	1083	1086	1083
query58	264	238	251	238
query59	2980	2732	2621	2621
query60	322	290	287	287
query61	131	221	103	103
query62	923	681	671	671
query63	220	189	185	185
query64	5318	717	709	709
query65	3285	3211	3191	3191
query66	1407	301	303	301
query67	15857	15535	15747	15535
query68	3438	868	850	850
query69	439	328	325	325
query70	1181	1133	1141	1133
query71	363	348	342	342
query72	6071	3492	3345	3345
query73	591	597	577	577
query74	9260	9115	9022	9022
query75	3168	3001	3048	3001
query76	1925	847	846	846
query77	451	416	407	407
query78	9476	9315	9370	9315
query79	910	884	869	869
query80	848	834	807	807
query81	453	269	263	263
query82	269	262	257	257
query83	195	193	190	190
query84	237	112	107	107
query85	652	454	429	429
query86	327	312	317	312
query87	4507	4363	4298	4298
query88	4158	4097	4056	4056
query89	372	377	375	375
query90	1291	326	309	309
query91	125	124	124	124
query92	79	77	73	73
query93	1041	1061	1040	1040
query94	661	368	388	368
query95	470	436	427	427
query96	471	478	471	471
query97	3154	3132	3117	3117
query98	231	226	230	226
query99	1558	1318	1320	1318
Total cold run time: 282405 ms
Total hot run time: 196032 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0b05b68e14a01067765fe6035e3888d6887e9402, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.07	0.08
query5	0.50	0.49	0.49
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.57	0.52	0.50
query10	0.56	0.59	0.56
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.61	0.60
query14	1.49	1.46	1.46
query15	0.92	0.88	0.88
query16	0.37	0.36	0.36
query17	1.00	1.03	1.03
query18	0.22	0.21	0.20
query19	1.94	1.84	1.78
query20	0.01	0.01	0.01
query21	15.43	0.67	0.65
query22	3.85	6.81	1.80
query23	17.85	1.35	1.28
query24	2.18	0.23	0.22
query25	0.19	0.08	0.08
query26	0.28	0.18	0.18
query27	0.09	0.07	0.08
query28	13.20	1.13	1.11
query29	12.57	3.39	3.33
query30	0.25	0.06	0.05
query31	2.87	0.42	0.41
query32	3.23	0.50	0.50
query33	3.06	3.09	3.12
query34	15.46	4.35	4.34
query35	4.36	4.38	4.33
query36	0.69	0.49	0.50
query37	0.19	0.17	0.17
query38	0.17	0.16	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 108.08 s
Total hot run time: 31.64 s

@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch 2 times, most recently from 156ff24 to 9e47eaf Compare September 14, 2024 09:54
@jacktengg
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.31% (9577/25669)
Line Coverage: 28.68% (79088/275735)
Region Coverage: 28.15% (40971/145535)
Branch Coverage: 24.78% (20882/84264)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9e47eaf9499b7939b860beddd597caa4c91261fe_9e47eaf9499b7939b860beddd597caa4c91261fe/report/index.html

@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch from 9e47eaf to 5b65cd7 Compare September 14, 2024 13:04
@jacktengg jacktengg force-pushed the fix-0913-hash-build-cancel branch from 5b65cd7 to d99ad86 Compare September 14, 2024 13:07
@jacktengg
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.31% (9576/25668)
Line Coverage: 28.70% (79136/275750)
Region Coverage: 28.15% (40972/145540)
Branch Coverage: 24.78% (20885/84266)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d99ad8614cec9105fff5007a10155fb0f0b78874_d99ad8614cec9105fff5007a10155fb0f0b78874/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 42089 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d99ad8614cec9105fff5007a10155fb0f0b78874, data reload: false

------ Round 1 ----------------------------------
q1	17580	7262	7167	7167
q2	2045	162	159	159
q3	10618	1163	1336	1163
q4	10418	864	867	864
q5	7762	3109	3150	3109
q6	233	152	143	143
q7	1012	610	588	588
q8	9426	2012	2069	2012
q9	6840	6425	6426	6425
q10	7057	2331	2335	2331
q11	438	248	241	241
q12	396	219	211	211
q13	17770	2982	3010	2982
q14	237	206	209	206
q15	573	511	503	503
q16	522	421	407	407
q17	981	929	945	929
q18	7468	6802	6757	6757
q19	1393	1244	1205	1205
q20	582	289	288	288
q21	3943	3402	3478	3402
q22	1105	1019	997	997
Total cold run time: 108399 ms
Total hot run time: 42089 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7143	7103	7099	7099
q2	335	232	231	231
q3	3050	3095	3032	3032
q4	2072	1982	1972	1972
q5	5585	5577	5670	5577
q6	240	139	150	139
q7	2190	1753	1768	1753
q8	3301	3394	3396	3394
q9	8709	8940	8729	8729
q10	3549	3463	3550	3463
q11	569	494	486	486
q12	811	632	633	632
q13	8558	3191	3194	3191
q14	307	272	270	270
q15	575	529	535	529
q16	510	470	476	470
q17	1778	1705	1752	1705
q18	8516	8058	8003	8003
q19	1740	1749	1732	1732
q20	2125	1873	1886	1873
q21	5455	5342	5307	5307
q22	1182	1070	1024	1024
Total cold run time: 68300 ms
Total hot run time: 60611 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198394 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d99ad8614cec9105fff5007a10155fb0f0b78874, data reload: false

query1	1267	884	901	884
query2	6431	1767	1715	1715
query3	10819	3967	3797	3797
query4	55096	25961	24204	24204
query5	5351	539	493	493
query6	414	174	171	171
query7	5743	316	302	302
query8	296	222	235	222
query9	7200	2617	2628	2617
query10	448	280	277	277
query11	16714	15599	15460	15460
query12	166	97	97	97
query13	1494	439	410	410
query14	10024	6720	7262	6720
query15	208	191	191	191
query16	7043	539	505	505
query17	1142	605	596	596
query18	1797	339	321	321
query19	191	151	148	148
query20	127	110	111	110
query21	224	102	99	99
query22	4583	4577	4689	4577
query23	34670	33633	33854	33633
query24	6150	3123	3081	3081
query25	536	397	398	397
query26	624	159	155	155
query27	1595	291	282	282
query28	2819	2113	2084	2084
query29	678	432	425	425
query30	226	151	148	148
query31	950	795	797	795
query32	72	54	56	54
query33	442	302	306	302
query34	902	480	467	467
query35	962	740	737	737
query36	1024	868	906	868
query37	130	79	72	72
query38	3980	4017	3926	3926
query39	1465	1425	1384	1384
query40	199	93	92	92
query41	46	53	44	44
query42	114	94	95	94
query43	466	449	443	443
query44	1148	771	766	766
query45	194	167	165	165
query46	1095	842	798	798
query47	1882	1809	1798	1798
query48	433	362	346	346
query49	679	393	393	393
query50	924	421	439	421
query51	7008	6863	6767	6767
query52	100	86	84	84
query53	249	177	177	177
query54	538	448	455	448
query55	76	72	75	72
query56	290	257	248	248
query57	1207	1107	1072	1072
query58	217	219	232	219
query59	3024	2698	2854	2698
query60	279	272	277	272
query61	107	100	97	97
query62	766	647	673	647
query63	211	183	179	179
query64	1430	675	605	605
query65	3261	3122	3127	3122
query66	662	286	289	286
query67	15924	15611	15436	15436
query68	2011	858	846	846
query69	450	339	353	339
query70	1149	1134	1137	1134
query71	335	331	324	324
query72	5076	3407	3265	3265
query73	585	574	575	574
query74	9239	9125	9154	9125
query75	3080	2959	2898	2898
query76	943	833	827	827
query77	478	348	350	348
query78	9496	9292	9291	9291
query79	895	870	867	867
query80	586	628	565	565
query81	446	237	236	236
query82	191	185	197	185
query83	153	152	151	151
query84	212	94	102	94
query85	647	387	352	352
query86	305	316	325	316
query87	4495	4365	4349	4349
query88	4110	4043	4043	4043
query89	357	360	351	351
query90	1473	295	297	295
query91	159	161	161	161
query92	73	74	73	73
query93	1002	988	988	988
query94	563	371	386	371
query95	452	418	417	417
query96	467	469	467	467
query97	3142	3141	3120	3120
query98	227	223	211	211
query99	1666	1322	1296	1296
Total cold run time: 290995 ms
Total hot run time: 198394 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d99ad8614cec9105fff5007a10155fb0f0b78874, data reload: false

query1	0.05	0.05	0.04
query2	0.06	0.02	0.02
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.50	0.50	0.51
query6	1.16	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.52	0.51
query10	0.56	0.60	0.55
query11	0.15	0.10	0.10
query12	0.13	0.10	0.10
query13	0.60	0.59	0.59
query14	1.42	1.44	1.43
query15	0.88	0.85	0.85
query16	0.38	0.37	0.39
query17	1.01	1.05	1.07
query18	0.19	0.18	0.19
query19	1.89	1.85	1.77
query20	0.01	0.01	0.01
query21	15.40	0.57	0.56
query22	3.89	3.40	2.72
query23	17.70	1.20	1.04
query24	2.34	0.28	0.28
query25	0.33	0.10	0.06
query26	0.15	0.13	0.13
query27	0.04	0.03	0.03
query28	12.20	1.02	0.99
query29	12.52	3.27	3.26
query30	0.24	0.06	0.06
query31	2.88	0.38	0.37
query32	3.27	0.47	0.46
query33	2.98	3.06	3.02
query34	15.43	4.30	4.27
query35	4.31	4.30	4.30
query36	0.70	0.48	0.48
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.15	0.12	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 106.3 s
Total hot run time: 31.31 s

@github-actions
Copy link
Contributor

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

@github-actions github-actions bot added the Stale label Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants