Make test harnesses the same for release and PRs#2308
Merged
Conversation
floryst
force-pushed
the
unify-testing
branch
2 times, most recently
from
520f251 to
bb05b94
Compare
floryst
force-pushed
the
unify-testing
branch
4 times, most recently
from
c47a930 to
0ecf988
Compare
added 2 commits
March 11, 2022 15:00
- Both release and PR checks will now run tests. - test result artifacts will now be published.
|
🎉 This PR is included in version 23.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
PaulHax
added a commit
to PaulHax/vtk-js
that referenced
this pull request
May 22, 2026
Four tests were skipped over the years, each during a test-harness or build migration rather than for a genuine feature failure. A skipped test is green in CI, so none produced a signal and all silently rotted. Re-enable them. testLabelmapOutlineManyRenderer Skipped 2022-03-11 in commit 50ace42 (PR Kitware#2308, "Make test harnesses the same for release and PRs"); the commit gave no reason. While skipped the code rotted: createLabelPipeline called backgroundImageData.getDimensionsByReference(), which is not a function on vtkImageData. Use getDimensions(). Passes against the existing baseline, so no baseline change. testEdgeVisibility Skipped 2021-11-12 in commit 13b9be1 (PR Kitware#2161, "Update test framework"), flagged in a pixelmatch-threshold-0 audit (PR Kitware#2098, closed unmerged) as a "bad image test" (issue Kitware#2104). The 2017 baseline predates commit a8873e5 (2021-02-19, "feat(vtkopenglpolydatamapper): add support for backface property"), which changed the edge draw path from lit (ambient = getAmbient(), diffuse = getDiffuse()) to flat (ambient = 1.0, diffuse = 0.0): edges now render at the full configured edge color, the intended behavior. The render has been stable since at least 2021-06 (a steady 1.80% from the 2017 baseline at 2021-06, 2022, and 2024). Refresh the baseline; the tolerance is unchanged at 1% (chromium ~0%, firefox passes). testAverageIntensityProjection Skipped 2021-11-12 in commit 13b9be1 (PR Kitware#2161), tracked by issue Kitware#2102. It did not drift or regress. Before the 2021 test-framework work the test was effectively a no-op: the old compareImages passed its threshold argument (here 1.5) straight to pixelmatch, whose threshold range is 0 to 1, so pixelmatch reported zero differing pixels for any image and the test passed vacuously. The pixelmatch-threshold-0 audit (PR Kitware#2098, closed unmerged) exposed the real comparison, and the test then "failed" because its baseline had never actually been checked. The committed baseline is the original 2019 image from aae2f55; commit 8a50448 only renamed it OpenGL/ to Core/ (100% rename). It is a near-blank volume the real render never matched. Rendering against old Sources shows a stable, correct head volume back to 2021-06 (~1.5% from the corrected baseline, ~0% on current master). Replace the baseline with the actual render (chromium ~0%, firefox 0.01%, within the 1% tolerance) and re-enable the test. Also release resources via .finally; gc.releaseResources had been passed as an unused 5th argument to compareImages. testSetTable Skipped 2026-05-21 by the Vite/Vitest migration (c0d9887), which rewrote test.onlyIfWebGL(...) as it.skip(...) instead of the it.skipIf(__VTK_TEST_NO_WEBGL__)(...) form used for every other migrated WebGL test, silently disabling a passing test. Restore the intended WebGL-conditional skip. Closes Kitware#2102
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Results
Changes
PR and Code Checklist
Testing
This updates the test environment. Green github action results means this works.