Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[optimize](cooldown)Improved filtering conditions for cooldown tablet #27338

Merged
merged 1 commit into from
Jan 5, 2024

Conversation

xingyingone
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@xingyingone
Copy link
Contributor Author

run buildall

Copy link
Contributor

@ByteYue ByteYue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.53% (8448/23125)
Line Coverage: 28.87% (68682/237939)
Region Coverage: 27.83% (35521/127640)
Branch Coverage: 24.58% (18112/73684)
Coverage Report: http://coverage.selectdb-in.cc/coverage/23c556a6373166744530a0178a1993169fcb04e0_23c556a6373166744530a0178a1993169fcb04e0/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 23c556a6373166744530a0178a1993169fcb04e0, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4913	4662	4670	4662
q2	381	175	160	160
q3	2043	1927	1950	1927
q4	1401	1262	1254	1254
q5	3951	3938	3987	3938
q6	251	132	131	131
q7	1453	885	890	885
q8	2742	2764	2750	2750
q9	46411	13297	9671	9671
q10	11247	3544	3541	3541
q11	385	252	233	233
q12	1096	293	292	292
q13	8408	3792	3794	3792
q14	321	290	313	290
q15	583	539	521	521
q16	668	584	581	581
q17	1130	975	934	934
q18	7772	7397	7417	7397
q19	1697	1665	1676	1665
q20	569	293	293	293
q21	6513	3964	3972	3964
q22	477	373	377	373
Total cold run time: 104412 ms
Total hot run time: 49254 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4629	4601	4578	4578
q2	353	234	263	234
q3	4036	3996	3999	3996
q4	2695	2678	2679	2678
q5	9723	9781	9767	9767
q6	243	125	127	125
q7	2996	2464	2466	2464
q8	4405	4455	4440	4440
q9	13162	13126	13114	13114
q10	4085	4184	4195	4184
q11	772	641	654	641
q12	981	812	822	812
q13	4275	3571	3550	3550
q14	378	348	354	348
q15	577	519	523	519
q16	729	677	675	675
q17	3891	3874	3937	3874
q18	9459	8911	9065	8911
q19	1820	1762	1772	1762
q20	2398	2060	2039	2039
q21	8853	8664	8713	8664
q22	956	852	786	786
Total cold run time: 81416 ms
Total hot run time: 78161 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.28 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17099649555 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.52% (8446/23125)
Line Coverage: 28.85% (68653/237939)
Region Coverage: 27.82% (35504/127640)
Branch Coverage: 24.56% (18099/73684)
Coverage Report: http://coverage.selectdb-in.cc/coverage/23c556a6373166744530a0178a1993169fcb04e0_23c556a6373166744530a0178a1993169fcb04e0/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 23c556a6373166744530a0178a1993169fcb04e0, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4880	4668	4670	4668
q2	365	159	159	159
q3	2037	1928	1874	1874
q4	1381	1251	1236	1236
q5	3983	3961	4010	3961
q6	248	125	130	125
q7	1410	900	902	900
q8	2737	2756	2750	2750
q9	10197	9734	9556	9556
q10	3462	3540	3538	3538
q11	380	248	245	245
q12	442	295	298	295
q13	4624	3823	3809	3809
q14	320	287	278	278
q15	593	539	529	529
q16	658	590	580	580
q17	1132	967	918	918
q18	7722	7386	7489	7386
q19	1673	1691	1649	1649
q20	533	295	314	295
q21	4376	3932	3964	3932
q22	470	380	378	378
Total cold run time: 53623 ms
Total hot run time: 49061 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4595	4553	4568	4553
q2	337	229	273	229
q3	4031	4037	3994	3994
q4	2686	2679	2692	2679
q5	9871	9691	9786	9691
q6	243	123	126	123
q7	3020	2470	2492	2470
q8	4425	4442	4446	4442
q9	13217	13151	13157	13151
q10	4138	4216	4199	4199
q11	797	682	662	662
q12	993	812	807	807
q13	4296	3585	3567	3567
q14	380	355	347	347
q15	579	528	533	528
q16	744	680	670	670
q17	3850	3915	3891	3891
q18	9684	8918	9048	8918
q19	1823	1752	1776	1752
q20	2389	2068	2061	2061
q21	8798	8559	8646	8559
q22	933	769	803	769
Total cold run time: 81829 ms
Total hot run time: 78062 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.43 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17100131291 Bytes

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 5, 2024
Copy link
Contributor

github-actions bot commented Jan 5, 2024

PR approved by at least one committer and no changes requested.

@yiguolei yiguolei merged commit d86277c into apache:master Jan 5, 2024
27 of 29 checks passed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
wsjz pushed a commit to wsjz/incubator-doris that referenced this pull request Feb 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants