Skip to content

[feature](mtmv)(6)implement cancel method#27541

Merged
morningman merged 6 commits intoapache:masterfrom
zddr:mv_6
Nov 27, 2023
Merged

[feature](mtmv)(6)implement cancel method#27541
morningman merged 6 commits intoapache:masterfrom
zddr:mv_6

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Nov 24, 2023 •
edited
Loading

Proposed changes

Issue Number: close #xxx

1.implement cancel task method
2.fix show create table not display comment

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@zddr
Copy link
Contributor Author

zddr commented Nov 24, 2023

run buildall

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 0fe0530f537d948a531e9e08eca0782931f22af5, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4909	4654	4628	4628
q2	360	155	162	155
q3	2052	1923	1898	1898
q4	1398	1268	1238	1238
q5	3966	3945	4042	3945
q6	256	131	134	131
q7	1443	895	880	880
q8	2783	2782	2783	2782
q9	9677	9452	9432	9432
q10	3458	3530	3505	3505
q11	369	244	240	240
q12	436	284	284	284
q13	4615	3817	3780	3780
q14	307	290	282	282
q15	582	528	520	520
q16	660	581	587	581
q17	1136	964	941	941
q18	7978	7479	7501	7479
q19	1664	1677	1692	1677
q20	564	318	310	310
q21	4477	4037	4087	4037
q22	472	365	365	365
Total cold run time: 53562 ms
Total hot run time: 49090 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4590	4566	4554	4554
q2	335	224	244	224
q3	4031	4020	4019	4019
q4	2775	2724	2723	2723
q5	9589	9601	9602	9601
q6	256	124	125	124
q7	3023	2466	2489	2466
q8	4486	4469	4483	4469
q9	12970	12827	12860	12827
q10	4006	4127	4127	4127
q11	812	672	695	672
q12	1000	813	814	813
q13	4278	3553	3557	3553
q14	377	341	336	336
q15	572	519	519	519
q16	730	698	687	687
q17	3889	3823	3825	3823
q18	9617	9104	9236	9104
q19	1825	1798	1813	1798
q20	2431	2082	2046	2046
q21	8870	8570	8546	8546
q22	886	774	817	774
Total cold run time: 81348 ms
Total hot run time: 77805 ms

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.52 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17099269424 Bytes

@zddr
Copy link
Contributor Author

zddr commented Nov 24, 2023

run buildall

morrySnow
morrySnow reviewed Nov 24, 2023
View reviewed changes
@doris-robot
Copy link

doris-robot commented Nov 24, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 55a9f22156519540ec00efd7f37035c425b33230, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4912	4639	4644	4639
q2	358	141	142	141
q3	2042	1923	1918	1918
q4	1394	1280	1258	1258
q5	3957	3930	4047	3930
q6	253	129	132	129
q7	1400	894	882	882
q8	2785	2807	2781	2781
q9	9633	9423	9848	9423
q10	3453	3521	3533	3521
q11	396	242	252	242
q12	439	288	288	288
q13	4548	3801	3808	3801
q14	310	294	279	279
q15	579	539	526	526
q16	665	589	583	583
q17	1145	984	951	951
q18	7955	7540	7536	7536
q19	1675	1679	1689	1679
q20	549	300	297	297
q21	4451	3974	4057	3974
q22	475	384	379	379
Total cold run time: 53374 ms
Total hot run time: 49157 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4542	4542	4585	4542
q2	341	237	298	237
q3	4034	4010	4023	4010
q4	2698	2695	2696	2695
q5	9809	9750	9769	9750
q6	247	123	123	123
q7	3026	2486	2462	2462
q8	4508	4508	4511	4508
q9	12912	12840	12733	12733
q10	4057	4187	4142	4142
q11	796	648	646	646
q12	980	820	783	783
q13	4284	3588	3562	3562
q14	382	342	377	342
q15	574	518	522	518
q16	726	668	686	668
q17	3862	3852	3861	3852
q18	9597	9102	8970	8970
q19	1792	1816	1782	1782
q20	2407	2058	2045	2045
q21	8837	8515	8512	8512
q22	843	845	844	844
Total cold run time: 81254 ms
Total hot run time: 77726 ms

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.83 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17157894487 Bytes

@zddr
Copy link
Contributor Author

zddr commented Nov 24, 2023

run buildall

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.64 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17099180141 Bytes

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit bebc6a8bd0d15515abd101b039b6c7e17ce99b8b, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4934	4667	4702	4667
q2	355	148	158	148
q3	2029	1948	1937	1937
q4	1406	1249	1246	1246
q5	3941	3954	4052	3954
q6	248	127	125	125
q7	1434	872	878	872
q8	2772	2802	2774	2774
q9	9844	9489	9426	9426
q10	3474	3554	3541	3541
q11	386	251	249	249
q12	435	287	286	286
q13	4567	3776	3736	3736
q14	316	282	282	282
q15	579	545	519	519
q16	664	583	588	583
q17	1136	956	969	956
q18	7885	7526	7558	7526
q19	1666	1651	1686	1651
q20	591	299	310	299
q21	4424	4006	4027	4006
q22	476	370	365	365
Total cold run time: 53562 ms
Total hot run time: 49148 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4591	4598	4594	4594
q2	352	222	253	222
q3	4036	4019	4024	4019
q4	2714	2701	2718	2701
q5	9955	9947	9950	9947
q6	252	122	121	121
q7	3026	2489	2493	2489
q8	4465	4504	4447	4447
q9	13042	12887	12942	12887
q10	4051	4186	4209	4186
q11	799	647	643	643
q12	972	800	807	800
q13	4264	3573	3577	3573
q14	380	343	361	343
q15	570	521	518	518
q16	754	673	661	661
q17	3871	3828	3905	3828
q18	9633	9162	9017	9017
q19	1831	1793	1754	1754
q20	2371	2063	2063	2063
q21	8809	8638	8546	8546
q22	968	836	777	777
Total cold run time: 81706 ms
Total hot run time: 78136 ms

@zddr zddr requested a review from morrySnow November 24, 2023 13:54
@zddr
Copy link
Contributor Author

zddr commented Nov 24, 2023

run buildall

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.98 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 32.7 seconds inserted 10000000 Rows, about 305K ops/s
storage size: 17098990485 Bytes

@doris-robot
Copy link

doris-robot commented Nov 24, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 68a43acf51b529a84f84af134e8d691beb5dd212, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4965	4667	4661	4661
q2	361	155	159	155
q3	2019	1963	1873	1873
q4	1385	1271	1283	1271
q5	3956	3938	4043	3938
q6	254	127	131	127
q7	1433	888	898	888
q8	2793	2785	2768	2768
q9	9667	9494	9376	9376
q10	3530	3499	3538	3499
q11	401	251	251	251
q12	432	286	298	286
q13	4607	3814	3806	3806
q14	320	285	288	285
q15	568	534	519	519
q16	661	582	590	582
q17	1122	947	901	901
q18	7841	7523	7540	7523
q19	1667	1669	1677	1669
q20	578	323	289	289
q21	4403	3968	3960	3960
q22	477	388	372	372
Total cold run time: 53440 ms
Total hot run time: 48999 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4618	4582	4577	4577
q2	338	221	283	221
q3	4026	4018	4018	4018
q4	2718	2710	2711	2710
q5	9799	9720	9737	9720
q6	246	123	124	123
q7	3025	2479	2500	2479
q8	4504	4451	4445	4445
q9	12892	12862	12853	12853
q10	4071	4170	4179	4170
q11	770	687	668	668
q12	969	810	795	795
q13	4288	3573	3562	3562
q14	371	333	344	333
q15	580	518	522	518
q16	736	666	670	666
q17	3950	3965	3926	3926
q18	9591	9206	9047	9047
q19	1796	1786	1810	1786
q20	2388	2063	2081	2063
q21	8940	8757	8676	8676
q22	860	785	776	776
Total cold run time: 81476 ms
Total hot run time: 78132 ms

morningman
morningman approved these changes Nov 26, 2023
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2023

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 26, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2023

PR approved by anyone and no changes requested.

@morningman morningman merged commit 3791de3 into apache:master Nov 27, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@zddr @seawinde
[feature](mtmv)(6)implement cancel method (apache#27541)
bc02090 
1.implement cancel task method
2.fix `show create table ` not display `comment`
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@zddr
[feature](mtmv)(6)implement cancel method (apache#27541)
691190e 
1.implement cancel task method
2.fix `show create table ` not display `comment`
@zddr zddr deleted the mv_6 branch March 28, 2024 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@morrySnow morrySnow Awaiting requested review from morrySnow

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zddr @doris-robot @morningman @CalvinKirs @morrySnow