Skip to content

[opt](ci) adjust threshold for perfmance test #53650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
hello-stephen merged 1 commit into from
Jul 23, 2025
Merged

[opt](ci) adjust threshold for perfmance test #53650

hello-stephen merged 1 commit into from
Jul 23, 2025

Conversation

@hello-stephen
Copy link
Contributor

@hello-stephen hello-stephen commented Jul 21, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor Author

hello-stephen commented Jul 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hello-stephen
Copy link
Contributor Author

hello-stephen commented Jul 21, 2025

run buildall

shuke987
shuke987 approved these changes Jul 21, 2025
View reviewed changes
Copy link
Collaborator

@shuke987 shuke987 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jul 21, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

TPC-H: Total hot run time: 33984 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f197c56422d282ff899475dde5858e77d81acb1b, data reload: false

------ Round 1 ----------------------------------
q1	17613	5252	5078	5078
q2	1943	287	196	196
q3	10287	1273	699	699
q4	10226	1012	517	517
q5	7614	2489	2279	2279
q6	191	169	144	144
q7	928	804	624	624
q8	9345	1301	1069	1069
q9	7284	5081	5172	5081
q10	6909	2403	1954	1954
q11	490	299	319	299
q12	359	363	231	231
q13	17787	3682	3057	3057
q14	255	247	227	227
q15	603	518	513	513
q16	442	442	398	398
q17	624	844	368	368
q18	7543	7221	7030	7030
q19	1735	973	599	599
q20	338	348	230	230
q21	3805	3206	2383	2383
q22	1084	1062	1008	1008
Total cold run time: 107405 ms
Total hot run time: 33984 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5208	5142	5121	5121
q2	259	343	241	241
q3	2166	2657	2300	2300
q4	1375	1791	1350	1350
q5	4384	4414	4419	4414
q6	225	176	140	140
q7	2093	1963	1785	1785
q8	2626	2484	2566	2484
q9	7256	7350	7206	7206
q10	3091	3277	2902	2902
q11	576	515	528	515
q12	740	819	671	671
q13	3559	4145	3526	3526
q14	300	324	297	297
q15	548	499	543	499
q16	457	508	462	462
q17	1189	1599	1410	1410
q18	7963	7973	7951	7951
q19	832	888	937	888
q20	2056	2120	1929	1929
q21	5006	4602	4286	4286
q22	1090	1070	990	990
Total cold run time: 52999 ms
Total hot run time: 51367 ms

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

TPC-DS: Total hot run time: 190820 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f197c56422d282ff899475dde5858e77d81acb1b, data reload: false

query1	997	401	413	401
query2	6581	1680	1671	1671
query3	6749	232	238	232
query4	26868	23456	23685	23456
query5	4903	726	585	585
query6	336	248	227	227
query7	4654	505	313	313
query8	377	301	294	294
query9	8674	2856	2876	2856
query10	538	378	300	300
query11	15949	15078	14799	14799
query12	176	127	130	127
query13	1675	541	442	442
query14	9327	6085	5989	5989
query15	227	211	196	196
query16	7321	649	508	508
query17	1230	721	595	595
query18	2005	436	346	346
query19	197	200	169	169
query20	134	129	130	129
query21	239	144	134	134
query22	4144	4215	4123	4123
query23	34138	33407	33362	33362
query24	8137	2411	2378	2378
query25	551	514	420	420
query26	752	280	162	162
query27	2691	517	359	359
query28	4346	2188	2175	2175
query29	703	593	459	459
query30	330	226	208	208
query31	976	880	800	800
query32	94	88	85	85
query33	581	412	363	363
query34	805	871	546	546
query35	810	875	807	807
query36	1052	1013	948	948
query37	136	119	93	93
query38	4230	4150	4164	4150
query39	1524	1468	1481	1468
query40	244	142	135	135
query41	107	103	104	103
query42	133	121	128	121
query43	508	513	480	480
query44	1350	896	871	871
query45	189	185	191	185
query46	847	1014	660	660
query47	1800	1866	1784	1784
query48	402	451	354	354
query49	772	577	477	477
query50	669	713	440	440
query51	5546	5609	5385	5385
query52	129	119	111	111
query53	260	279	208	208
query54	620	643	586	586
query55	98	95	94	94
query56	354	350	341	341
query57	1237	1215	1176	1176
query58	321	302	319	302
query59	2573	2687	2506	2506
query60	372	377	354	354
query61	156	147	166	147
query62	827	724	719	719
query63	247	223	212	212
query64	3364	1090	743	743
query65	4303	4231	4249	4231
query66	1196	593	514	514
query67	15924	15554	15533	15533
query68	7899	905	553	553
query69	526	346	312	312
query70	1255	1148	1150	1148
query71	495	355	338	338
query72	5755	4772	4882	4772
query73	710	630	375	375
query74	9000	8865	8921	8865
query75	3946	3272	2701	2701
query76	3643	1136	735	735
query77	859	443	389	389
query78	10085	10099	9365	9365
query79	2832	807	597	597
query80	705	596	544	544
query81	517	274	249	249
query82	464	146	118	118
query83	314	279	277	277
query84	315	125	96	96
query85	832	413	363	363
query86	405	325	310	310
query87	4467	4409	4409	4409
query88	3675	2396	2393	2393
query89	422	336	302	302
query90	1847	242	243	242
query91	155	152	125	125
query92	99	81	80	80
query93	2125	942	595	595
query94	697	416	310	310
query95	422	388	332	332
query96	511	584	289	289
query97	2760	2766	2669	2669
query98	256	228	223	223
query99	1478	1444	1297	1297
Total cold run time: 279176 ms
Total hot run time: 190820 ms

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

ClickBench: Total hot run time: 32.86 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f197c56422d282ff899475dde5858e77d81acb1b, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.24	0.08	0.07
query4	1.63	0.11	0.11
query5	0.43	0.44	0.40
query6	1.16	0.66	0.68
query7	0.03	0.02	0.02
query8	0.06	0.04	0.05
query9	0.60	0.53	0.52
query10	0.58	0.57	0.58
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.63	0.62	0.61
query14	0.81	0.81	0.82
query15	0.88	0.88	0.89
query16	0.38	0.38	0.40
query17	1.06	1.06	1.03
query18	0.23	0.22	0.21
query19	1.96	1.83	1.94
query20	0.01	0.02	0.02
query21	15.38	0.91	0.55
query22	0.76	1.12	0.98
query23	14.68	1.38	0.65
query24	6.60	1.05	0.63
query25	0.52	0.21	0.06
query26	0.48	0.19	0.15
query27	0.06	0.06	0.05
query28	9.84	0.93	0.45
query29	12.54	3.94	3.30
query30	3.17	3.15	3.08
query31	2.82	0.59	0.39
query32	3.24	0.55	0.49
query33	3.04	3.04	3.15
query34	16.19	5.44	4.79
query35	4.95	4.87	4.89
query36	0.67	0.54	0.50
query37	0.10	0.08	0.07
query38	0.06	0.05	0.05
query39	0.04	0.04	0.03
query40	0.19	0.16	0.15
query41	0.09	0.04	0.04
query42	0.04	0.03	0.04
query43	0.04	0.05	0.04
Total cold run time: 106.63 s
Total hot run time: 32.86 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 23, 2025

PR approved by at least one committer and no changes requested.

@hello-stephen hello-stephen merged commit e4e299a into apache:master Jul 23, 2025
29 of 30 checks passed
@hello-stephen hello-stephen deleted the 0721-perf branch July 23, 2025 08:21
github-actions bot pushed a commit that referenced this pull request Jul 23, 2025
@hello-stephen
[opt](ci) adjust threshold for perfmance test (#53650)
08511e5
@github-actions github-actions bot mentioned this pull request Jul 23, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jul 23, 2025
@hello-stephen
[opt](ci) adjust threshold for perfmance test (#53650)
a8d9620
@github-actions github-actions bot mentioned this pull request Jul 23, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jul 23, 2025
@github-actions @hello-stephen
branch-3.0: [opt](ci) adjust threshold for perfmance test #53650 (#53777
3e30aa6 
)

Cherry-picked from #53650

Co-authored-by: Dongyang Li <lidongyang@selectdb.com>
morrySnow pushed a commit that referenced this pull request Jul 24, 2025
@github-actions @hello-stephen
branch-3.1: [opt](ci) adjust threshold for perfmance test #53650 (#53778
05f16f9 
)

Cherry-picked from #53650

Co-authored-by: Dongyang Li <lidongyang@selectdb.com>
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request Jul 30, 2025
@hello-stephen @w41ter
[opt](ci) adjust threshold for perfmance test (apache#53650)
0317dc3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhangstar333 zhangstar333 zhangstar333 approved these changes

+1 more reviewer

@shuke987 shuke987 shuke987 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @shuke987 @zhangstar333 @dataroaring @morrySnow