Skip to content

[test](feut) fix testShouldWarmUpPerformanceRepeatedCycles200k - #66081

Merged
bobhan1 merged 1 commit into
apache:masterfrom
bobhan1:agent/relax-cache-hotspot-performance-time-limit
Jul 27, 2026
Merged

[test](feut) fix testShouldWarmUpPerformanceRepeatedCycles200k#66081
bobhan1 merged 1 commit into
apache:masterfrom
bobhan1:agent/relax-cache-hotspot-performance-time-limit

Conversation

@bobhan1

@bobhan1 bobhan1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: #65153

Problem Summary: The repeated-cycle cache hotspot performance test used a 1-second average threshold even though the equivalent 200K-table single-cycle test allows 1.5 seconds. CI hosts occasionally exceeded 1 second while preserving correct matching results. This change aligns the repeated-cycle threshold with the existing 1.5-second allowance and updates the assertion message.

Release note

None

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

Test command:

./run-fe-ut.sh --run org.apache.doris.cloud.CacheHotspotManagerTableFilterTest#testShouldWarmUpPerformanceRepeatedCycles200k
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: apache#65153

Problem Summary: The repeated-cycle cache hotspot performance test used a 1-second average threshold even though the equivalent 200K-table single-cycle test allows 1.5 seconds. CI hosts occasionally exceeded 1 second while preserving correct matching results. Align the repeated-cycle threshold with the existing 1.5-second allowance and update the assertion message.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.cloud.CacheHotspotManagerTableFilterTest#testShouldWarmUpPerformanceRepeatedCycles200k
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1
bobhan1 marked this pull request as ready for review July 27, 2026 03:42
@bobhan1
bobhan1 requested a review from gavinchou as a code owner July 27, 2026 03:42
@bobhan1

bobhan1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@bobhan1 bobhan1 changed the title [test](fe) Relax cache hotspot performance threshold [test](feut) fix testShouldWarmUpPerformanceRepeatedCycles200k Jul 27, 2026
@bobhan1

bobhan1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review opinion: approve.

The threshold alignment looks good. The five-cycle test still verifies exact matching for all 1,000,000 calls, and its 1.5-second per-cycle average now matches the existing single-cycle 200K-table oracle. Because OnTablesFilter is immutable and each call uses the same precompiled rule set, the average remains a useful sustained-throughput guard while reducing sensitivity to isolated CI scheduling or GC pauses.

Critical checkpoint conclusions:

  • Goal and proof: the focused test-only change accomplishes the stated anti-flakiness goal without altering correctness coverage.
  • Scope: minimal and clear; only the threshold and matching assertion message change.
  • Concurrency, lifecycle, and locking: no production or shared-state behavior changes; not applicable to this patch.
  • Configuration, compatibility, persistence, data writes, and FE-BE propagation: no changes; not applicable.
  • Parallel paths and conditions: the revised bound is consistent with the neighboring single-cycle 200K path; production callers remain unchanged.
  • Tests and results: the exact match-count assertion and broader 10K/50K/200K/500K, selective, and multi-rule coverage remain intact. No result files change. Per the review-run instructions, I did not execute builds or tests.
  • Performance: the 1.5-second bound retains a meaningful regression signal and is well below the default 60-second refresh interval. The microbenchmark's omission of catalog traversal/logging and its averaging behavior predate this patch.
  • Observability and other concerns: no runtime observability change is needed, and I found no additional correctness or performance issue.

User focus: no additional review focus was provided; I reviewed the whole PR.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jul 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29239 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ebd5fdfe257efff658ff21eed1d37db32ee90de, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17601	4108	4093	4093
q2	2030	319	198	198
q3	10297	1408	810	810
q4	4684	474	346	346
q5	7521	848	568	568
q6	179	175	138	138
q7	793	804	618	618
q8	9326	1516	1472	1472
q9	5631	4324	4324	4324
q10	6756	1722	1454	1454
q11	509	360	322	322
q12	718	572	442	442
q13	18085	3365	2765	2765
q14	277	265	251	251
q15	q16	788	782	714	714
q17	1014	874	1036	874
q18	6840	5756	5500	5500
q19	1309	1189	1166	1166
q20	803	664	560	560
q21	5939	2608	2326	2326
q22	424	349	298	298
Total cold run time: 101524 ms
Total hot run time: 29239 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4391	4278	4275	4275
q2	296	318	212	212
q3	4571	4981	4450	4450
q4	2048	2138	1370	1370
q5	4426	4286	4293	4286
q6	236	178	127	127
q7	1728	1657	1957	1657
q8	2664	2217	2163	2163
q9	7930	8091	7709	7709
q10	4728	4660	4154	4154
q11	577	405	378	378
q12	765	795	558	558
q13	3260	3597	2883	2883
q14	299	318	280	280
q15	q16	723	733	685	685
q17	1373	1310	1321	1310
q18	8036	7350	7198	7198
q19	1169	1127	1122	1122
q20	2200	2193	1950	1950
q21	5216	4520	4372	4372
q22	546	447	399	399
Total cold run time: 57182 ms
Total hot run time: 51538 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 178103 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ebd5fdfe257efff658ff21eed1d37db32ee90de, data reload: false

query5	4352	621	485	485
query6	473	229	209	209
query7	4880	614	352	352
query8	337	187	172	172
query9	8779	4052	3994	3994
query10	478	379	295	295
query11	5897	2360	2151	2151
query12	158	100	98	98
query13	1345	580	426	426
query14	6232	5188	4868	4868
query14_1	4206	4236	4206	4206
query15	224	201	183	183
query16	999	498	463	463
query17	931	721	595	595
query18	2450	478	347	347
query19	216	197	159	159
query20	111	106	109	106
query21	234	158	134	134
query22	13595	13523	13415	13415
query23	17354	16576	16198	16198
query23_1	16311	16260	16268	16260
query24	7454	1765	1265	1265
query24_1	1308	1282	1288	1282
query25	569	484	397	397
query26	1327	345	211	211
query27	2621	627	385	385
query28	4487	1995	2013	1995
query29	1121	643	512	512
query30	341	265	226	226
query31	1116	1093	972	972
query32	117	68	62	62
query33	548	334	264	264
query34	1169	1189	654	654
query35	783	790	686	686
query36	1228	1185	1070	1070
query37	163	111	92	92
query38	1867	1739	1649	1649
query39	880	878	843	843
query39_1	843	815	851	815
query40	255	157	145	145
query41	64	62	65	62
query42	90	90	91	90
query43	322	327	282	282
query44	1417	768	762	762
query45	187	188	176	176
query46	1097	1234	758	758
query47	2165	2141	2031	2031
query48	396	431	279	279
query49	580	432	308	308
query50	1113	418	362	362
query51	11182	10787	10893	10787
query52	88	83	74	74
query53	271	272	202	202
query54	278	235	220	220
query55	74	70	65	65
query56	290	293	283	283
query57	1322	1287	1217	1217
query58	293	240	261	240
query59	1588	1671	1386	1386
query60	322	274	252	252
query61	153	155	205	155
query62	564	497	437	437
query63	245	206	210	206
query64	2834	1061	868	868
query65	4741	4644	4632	4632
query66	1827	518	385	385
query67	29411	29293	29044	29044
query68	3279	1586	970	970
query69	411	305	260	260
query70	1085	976	954	954
query71	375	340	313	313
query72	3063	2658	2352	2352
query73	816	801	420	420
query74	5066	4925	4768	4768
query75	2531	2499	2159	2159
query76	2367	1180	768	768
query77	345	381	282	282
query78	12008	12033	11317	11317
query79	1344	1132	757	757
query80	765	554	458	458
query81	496	339	297	297
query82	572	161	123	123
query83	401	327	295	295
query84	325	156	135	135
query85	1033	615	536	536
query86	384	302	282	282
query87	1846	1839	1761	1761
query88	3709	2771	2769	2769
query89	445	390	328	328
query90	1839	205	195	195
query91	208	193	162	162
query92	62	58	55	55
query93	1607	1593	970	970
query94	574	352	278	278
query95	777	578	472	472
query96	1089	803	324	324
query97	2672	2610	2537	2537
query98	213	207	204	204
query99	1089	1111	977	977
Total cold run time: 263819 ms
Total hot run time: 178103 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ebd5fdfe257efff658ff21eed1d37db32ee90de, data reload: false

query1	0.01	0.01	0.01
query2	0.09	0.04	0.05
query3	0.26	0.13	0.13
query4	1.62	0.14	0.14
query5	0.26	0.23	0.22
query6	1.23	1.09	1.04
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.38	0.32	0.31
query10	0.53	0.54	0.55
query11	0.20	0.14	0.13
query12	0.18	0.14	0.14
query13	0.47	0.47	0.48
query14	1.02	1.00	0.99
query15	0.62	0.60	0.59
query16	0.33	0.32	0.34
query17	1.07	1.08	1.10
query18	0.20	0.20	0.21
query19	1.99	1.96	1.97
query20	0.02	0.01	0.01
query21	15.42	0.22	0.12
query22	4.85	0.06	0.05
query23	16.14	0.31	0.11
query24	2.99	0.41	0.30
query25	0.12	0.05	0.04
query26	0.73	0.21	0.16
query27	0.04	0.04	0.03
query28	3.51	0.92	0.51
query29	12.47	4.14	3.34
query30	0.28	0.15	0.16
query31	2.77	0.58	0.32
query32	3.22	0.59	0.48
query33	3.27	3.19	3.23
query34	15.57	4.21	3.53
query35	3.54	3.50	3.54
query36	0.54	0.43	0.44
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.16
query41	0.09	0.04	0.03
query42	0.03	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.56 s
Total hot run time: 25.02 s

@bobhan1
bobhan1 merged commit faa6aec into apache:master Jul 27, 2026
34 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.1.x dev/4.1.x-conflict reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants