Skip to content

Report per operation timings in the multi editor and perspective tests - #4232

Merged
vogella merged 1 commit into
eclipse-platform:masterfrom
vogella:perf-tests-multi-editor-perspective
Aug 14, 2026
Merged

Report per operation timings in the multi editor and perspective tests#4232
vogella merged 1 commit into
eclipse-platform:masterfrom
vogella:perf-tests-multi-editor-perspective

Conversation

@vogella

@vogella vogella commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Continues #4230 for the two tests that cover the operations users feel most: opening many editors and switching perspectives. Both drop org.eclipse.test.performance, warm up before measuring, time the individual operations and print the distribution, so a change in one operation can be attributed instead of disappearing into a single number that nothing recorded.

OpenMultipleEditorTest also reports the first and the last ten editors separately, which turns it into the only test we have that shows how cost scales with the number of open editors. Opening the 90th editor currently costs 20 to 67 percent more than the 10th, and closing an editor while 90 are open costs about three times as much as when 10 remain. Because both ends come from the same run, that comparison is unaffected by machine load, which the absolute numbers very much are.

While adding assertions I found that the test never measured what it claimed: the workbench recycles the oldest editor once REUSE_EDITORS are open, defaulting to 99, so the hundredth open silently reused the first editor. The count is now 90 with an assertion explaining the threshold. PerspectiveSwitchTest had a related problem, silently returning and printing to stdout when a perspective is missing, which reported a pass for something never measured; it now skips through an assumption, so the three JDT cases show up as skipped rather than green.

Sample output of a local run:

openMultiple[perf_outline, closeEach] open, first 10  n=10  min= 35.52  p50= 43.01  p90= 67.45 (ms)
openMultiple[perf_outline, closeEach] open, last 10   n=10  min= 54.62  p50= 67.08  p90= 76.15 (ms)
openMultiple[perf_outline, closeEach] close, first 10 n=10  min= 25.48  p50= 30.01  p90= 39.69 (ms)
openMultiple[perf_outline, closeEach] close, last 10  n=10  min=  6.22  p50=  9.46  p90= 17.50 (ms)
openMultiple[perf_text, closeAll] closeAllEditors     n=1   1011.07 (ms, 90 editors)
PerspectiveSwitch to [org.eclipse.ui.tests.performancePerspective1] n=50 min= 16.74 p50= 18.60 (ms)

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Test Results

   858 files     858 suites   50m 21s ⏱️
 8 157 tests  7 914 ✅ 243 💤 0 ❌
20 373 runs  19 719 ✅ 654 💤 0 ❌

Results for commit 07fd8a9.

♻️ This comment has been updated with latest results.

@vogella

vogella commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Change in test code, performance tests are not part of the CI verification (only compiled)

Migrates PerspectiveSwitchTest the same way as OpenCloseEditorTest,
EditorSwitchTest and OpenMultipleEditorTest: drop
org.eclipse.test.performance, warm up before measuring, time the individual
operations and print the distribution through
UIPerformanceTestUtil.reportTimings. The performance database it reported to
has not been configured for years, so commitMeasurements and
assertPerformance were no-ops and the test could neither fail nor report
anything.

Each switch direction is now reported separately, since entering the two
perspectives does not cost the same. When a perspective is missing the test
skips through an assumption instead of printing to stdout and returning,
which reported a pass for something never measured. Three of its four cases
reference JDT perspectives and are skipped in this target platform, which is
now visible in the test results.

OpenMultipleEditorTest opens 90 instead of 100 editors: the workbench
recycles the oldest editor once REUSE_EDITORS are open, which defaults to
99, so the hundredth open silently reused the first editor and the test
measured a reuse rather than an open. An assertion now pins that down.
@vogella
vogella force-pushed the perf-tests-multi-editor-perspective branch from 2e1e66d to 07fd8a9 Compare August 13, 2026 09:04
@vogella
vogella merged commit 5ddc27d into eclipse-platform:master Aug 14, 2026
18 checks passed
@vogella
vogella deleted the perf-tests-multi-editor-perspective branch August 14, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant