Skip to content

[feat](stats) Prioritize critical column for auto analyze#27811

Closed
Kikyou1997 wants to merge 1 commit intoapache:masterfrom
Kikyou1997:predicate
Closed

[feat](stats) Prioritize critical column for auto analyze#27811
Kikyou1997 wants to merge 1 commit intoapache:masterfrom
Kikyou1997:predicate

Conversation

@Kikyou1997
Copy link
Contributor

add table to slot reference

predicate

name -> id

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.94 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17163962455 Bytes

@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2023

clang-tidy review says "All clean, LGTM! 👍"

@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.86 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17164079710 Bytes

@Kikyou1997 Kikyou1997 force-pushed the predicate branch 2 times, most recently from ef0c6f5 to 4bf594a Compare December 5, 2023 09:23
@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

clang-tidy review says "All clean, LGTM! 👍"

@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

clang-tidy review says "All clean, LGTM! 👍"

add table to slot reference

predicate

name -> id
@Kikyou1997
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.99 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17164161484 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit d1fc7cfedc32ed04270ac46eae78669ca519cb97, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4760	4544	4551	4544
q2	394	162	158	158
q3	1473	1240	1224	1224
q4	1116	943	948	943
q5	3169	3191	3178	3178
q6	253	143	131	131
q7	1023	511	505	505
q8	2206	2195	2194	2194
q9	6723	6764	6659	6659
q10	3235	3291	3283	3283
q11	338	227	217	217
q12	370	228	228	228
q13	4546	3794	3816	3794
q14	250	226	221	221
q15	566	521	517	517
q16	447	391	403	391
q17	1007	601	577	577
q18	8650	7648	7391	7391
q19	1517	1380	1415	1380
q20	939	364	323	323
q21	3111	2723	2699	2699
q22	366	314	320	314
Total cold run time: 46459 ms
Total hot run time: 40871 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4448	4488	4445	4445
q2	280	174	178	174
q3	3566	3517	3527	3517
q4	2385	2372	2361	2361
q5	5734	5728	5757	5728
q6	241	122	124	122
q7	2414	1880	1909	1880
q8	3520	3525	3549	3525
q9	9074	9061	9022	9022
q10	3917	3995	4004	3995
q11	515	406	385	385
q12	782	624	608	608
q13	4314	3579	3577	3577
q14	292	261	254	254
q15	567	509	521	509
q16	501	445	459	445
q17	1889	1884	1856	1856
q18	9652	8218	8140	8140
q19	1731	1748	1735	1735
q20	2246	1940	1939	1939
q21	6554	6203	6214	6203
q22	514	455	443	443
Total cold run time: 65136 ms
Total hot run time: 60863 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants