Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](statistics)Fix auto job start time incorrect bug. #27402

Merged
merged 1 commit into from
Nov 22, 2023
Merged

[Fix](statistics)Fix auto job start time incorrect bug. #27402

merged 1 commit into from
Nov 22, 2023

Conversation

Jibing-Li
Copy link
Contributor

@Jibing-Li Jibing-Li commented Nov 22, 2023 •
edited
Loading

Before, the auto analyze job start time was the job creation time, not the start to execute time, which is inaccurate. This pr is to change the start time to the first task start to execute time.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Jibing-Li Jibing-Li marked this pull request as ready for review November 22, 2023 08:51
@Jibing-Li
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Jibing-Li Jibing-Li mentioned this pull request Nov 22, 2023
Merged
morrySnow
morrySnow previously approved these changes Nov 22, 2023
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@Jibing-Li
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 22, 2023
morningman
morningman approved these changes Nov 22, 2023
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@xiaokang xiaokang added the p0_b label Nov 22, 2023
@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.73 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17098927546 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit 63db93ecd19d5459dbbf6978bd165b1944aa7da0, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4863	4659	4659	4659
q2	363	156	159	156
q3	2027	1905	1921	1905
q4	1439	1300	1258	1258
q5	3940	3937	4007	3937
q6	246	130	128	128
q7	1464	877	902	877
q8	2735	2755	2743	2743
q9	9914	9697	9649	9649
q10	3467	3516	3507	3507
q11	378	236	248	236
q12	436	288	292	288
q13	4506	3825	3790	3790
q14	323	285	286	285
q15	586	543	521	521
q16	665	589	589	589
q17	1126	956	924	924
q18	7889	7465	7306	7306
q19	1639	1655	1652	1652
q20	522	289	317	289
q21	4351	3906	3956	3906
q22	470	371	370	370
Total cold run time: 53349 ms
Total hot run time: 48975 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4593	4561	4595	4561
q2	350	218	253	218
q3	3999	3980	3986	3980
q4	2697	2667	2678	2667
q5	9725	9599	9654	9599
q6	244	123	125	123
q7	3018	2456	2471	2456
q8	4447	4443	4443	4443
q9	13184	13054	13049	13049
q10	4084	4178	4180	4178
q11	827	665	648	648
q12	988	798	814	798
q13	4273	3634	3559	3559
q14	375	357	338	338
q15	576	526	534	526
q16	721	666	707	666
q17	3890	3860	3864	3860
q18	9483	9019	9079	9019
q19	1800	1772	1759	1759
q20	2404	2054	2053	2053
q21	8752	8432	8470	8432
q22	907	776	794	776
Total cold run time: 81337 ms
Total hot run time: 77708 ms

@morningman morningman merged commit 19c36dc into apache:master Nov 22, 2023
26 of 27 checks passed
morningman pushed a commit that referenced this pull request Nov 22, 2023
@Jibing-Li
[Fix](statistics)Fix auto job start time incorrect bug #27402 (#27413)
854ef33 
backport #27402
@Jibing-Li Jibing-Li deleted the time branch November 22, 2023 14:08
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@Jibing-Li @seawinde
[Fix](statistics)Fix auto job start time incorrect bug (apache#27402)
05c4094 
Before, the auto analyze job start time was the job creation time, not the start to execute time, which is inaccurate. This pr is to change the start time to the first task start to execute time.
gnehil pushed a commit to gnehil/doris that referenced this pull request Dec 4, 2023
@Jibing-Li
[Fix](statistics)Fix auto job start time incorrect bug apache#27402 (a…
a02c7fc 
…pache#27413)

backport apache#27402
@xiaokang xiaokang mentioned this pull request Dec 4, 2023
Open
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@Jibing-Li
[Fix](statistics)Fix auto job start time incorrect bug (apache#27402)
8d08815 
Before, the auto analyze job start time was the job creation time, not the start to execute time, which is inaccurate. This pr is to change the start time to the first task start to execute time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Kikyou1997 Kikyou1997 Kikyou1997 approved these changes

@morningman morningman morningman approved these changes

@morrySnow morrySnow morrySnow left review comments

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.3-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Jibing-Li @doris-robot @morningman @Kikyou1997 @morrySnow @xiaokang