Skip to content

Conversation

@Kikyou1997
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Kikyou1997
Copy link
Contributor Author

run buildall

analysisTasks.values().forEach(analysisTaskExecutor::submitTask);
for (BaseAnalysisTask task : analysisTasks.values()) {
try {
task.execute();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a block. But it's better to write the successfully executed tasks' result to column statistics table, instead of fail the whole job when one task failed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree with you and the logic you mentioned has been implemented in AnalysisJob

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.09 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 26 seconds loaded 2358488459 Bytes, about 86 MB/s
stream load orc: 71 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17100488403 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit b29546b5a34f1a30c18e0f50f2ee5763e6a0489b, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4899	4703	4665	4665
q2	366	162	144	144
q3	2044	1944	1957	1944
q4	1413	1291	1276	1276
q5	4004	3989	4015	3989
q6	258	138	137	137
q7	1425	876	886	876
q8	2803	2816	2783	2783
q9	9809	9643	9603	9603
q10	3455	3643	3519	3519
q11	386	256	261	256
q12	440	290	290	290
q13	4612	3837	3846	3837
q14	325	300	301	300
q15	588	532	523	523
q16	664	588	592	588
q17	1146	979	940	940
q18	7901	7393	7595	7393
q19	1680	1667	1691	1667
q20	577	319	323	319
q21	4444	4012	4032	4012
q22	473	378	382	378
Total cold run time: 53712 ms
Total hot run time: 49439 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4582	4585	4574	4574
q2	339	239	260	239
q3	4036	4011	4018	4011
q4	2726	2730	2716	2716
q5	9609	9555	9604	9555
q6	249	126	126	126
q7	3063	2510	2486	2486
q8	4484	4442	4429	4429
q9	13200	13119	13166	13119
q10	4070	4159	4177	4159
q11	766	675	638	638
q12	988	806	819	806
q13	4271	3618	3584	3584
q14	375	357	364	357
q15	571	526	524	524
q16	746	686	687	686
q17	3911	3843	3865	3843
q18	9751	9146	9042	9042
q19	1832	1840	1775	1775
q20	2402	2066	2048	2048
q21	8977	8515	8819	8515
q22	902	842	790	790
Total cold run time: 81850 ms
Total hot run time: 78022 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants