Skip to content

v6.0.0

Choose a tag to compare

View all tags
@pashidlos pashidlos released this 12 Nov 07:24
· 4 commits to master since this release
v6.0.0
8c6248a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The VRT web UI 6.0.0: a redesigned interface with dark mode, a post-analysis rendering-noise hint, undo for review decisions, and React 19, MUI 9 and Vite 8 underneath.

⚠️ Breaking changes

  • UI 6.0 needs API 6.0. It calls POST /test-runs/revert, GET /projects/favorites, PUT|DELETE /projects/:id/favorite, GET /statistics/results, GET /dinov2/status and reads per-run results of bulk approve, reject and delete. Against an older API, Undo and starring answer 404.
  • Live updates need a login. The socket connects with auth: { token } and sends join / leave for the open project ({ projectId }) and build ({ buildId }), again on every reconnect. A server that closes the socket (io server disconnect, for example an expired token or a deleted user) logs the user out; client-side disconnects and transport drops do not.
  • Node 26 (engines.node >=26, .nvmrc 26) is required to build the UI from source.
  • Image base: nginx:1.30-alpine-slim (was nginx:1.25-alpine3.17-slim); the builder stage is node:26-alpine, run on the build platform. Runtime configuration is unchanged: env.sh writes env-config.js from REACT_APP_API_URL, REACT_APP_STATIC_URL, REACT_APP_OIDC_AUTO_REDIRECT, VRT_VERSION and PORT (nginx still listens on 8080).
  • Project settings keep two post-analysis switches. The UI saves noiseAnalysisConfig as { "blur": { "enabled": … }, "dinov2": { "enabled": … } }; thresholds stored by an earlier release candidate are dropped on the next save. New projects created from the UI start with both switches off; a stored config without enabled reads as off.
  • Removed UI features: see Removed.
  • Changed locators. Status chips and counters show sentence-case labels (Passed, Auto-approved, Running) instead of enum values; enum values stay in API calls, URLs, storage and data-testids. src/constants/help.ts became src/constants/locators.ts. The project switcher label id is project-select-label. Custom Playwright suites that match these texts or ids need updating.

New

  • Undo approval / Undo rejection in the run dialog footer, and an Undo button on the toasts after approve or reject (run dialog, bulk actions, matching-variations dialog). Calls POST /test-runs/revert?decision=approve|reject; a 409 (status changed meanwhile) shows the server's message. A bulk action that partly failed offers Undo only for the runs that went through. Guests never see Undo.
  • Refused decisions warn: a run the server refuses (already approved, changed by someone else) shows an orange warning with the reason; Internal error in a bulk result shows as an error. Toasts name the run instead of its id.
  • Favourites: a star on each project card; starred projects are listed first, in the list and in the project switcher. Kept per user by the API (404 from GET /projects/favorites counts as no favourites).
  • Project list search by name, id or main branch; the query stays in the URL (?q=) and / focuses the field. The project switcher is an autocomplete with the same groups and search.
  • Project page with tabs: Builds, Variations, Statistics, Settings, with a breadcrumb; an unknown project shows "Project not found" with a link back.
  • Statistics tab: results per build (GET /statistics/results), with a branch filter.
  • Run list: filter chips, selection bar, table or grid view, grid "Sort by" with select all, comments visible in rows and cards.
  • Run dialog: header and info bar; view modes Side by side, Diff and Fade; ignore-area edit mode; images refit when the panes resize until you zoom by hand (Fit re-arms it).
  • Rendering-noise hint (post-analysis):
    • A Noise column with a verdict chip per unresolved run (Likely change, Likely noise, Analysing…) and a noise panel in the run dialog with Analyse / Analyse again.
    • Analyse noise from the build menu, the run list selection bar (queued and skipped runs are reported; queued rows flip to "Analysing…" at once) and the run's noise chip.
    • The column, chips and actions show only when the project has at least one post-analysis stage enabled.
    • Project settings, section Post-analysis: blur and DINOv2 switches; DINOv2 is disabled when the server has no DINOv2 service (GET /dinov2/status).
    • The hint never changes a status.
  • Build header with status counters and "Analysing N of M…" progress.
  • Variations: list with filters; variation page with baseline history.
  • Add project page; project settings in sections.
  • Profile page with API key regenerate (confirms first).
  • Users page: search, role filter, selection bar, joined date; deleting users asks for confirmation first.
  • Help menu has a Documentation link to the GitHub Wiki.

Redesign & theming

  • Design tokens in src/theme/tokens.ts: primary is the logo's cyan on slate neutrals, Inter bundled with the app (no CDN), softer radius and shadows.
  • Dark mode follows the system and remembers an explicit choice.
  • Status colours come from one theme source (statusColor) in light and dark; a test keeps their contrast at WCAG AA in both modes.
  • Dialogs share one look: BaseModal title and close button, Cancel left of a filled primary action, destructive confirms in the error colour.
  • Shared building blocks: StatusChip, FilterBar, SelectionBar, TagLine, short dates, outlined fields, empty states, inline validation messages.
  • @mui/styles is gone; styling uses sx and styled() with theme values.

Performance

  • Routes are lazy-loaded; the run dialog (Konva) and jszip load on demand. In #434 the main chunk went from 1.6 MB to 137 KB.
  • npm run check:size enforces a gzip budget in CI: initial load (entry plus preloaded chunks) 287 kB and ProjectPage 22 kB.
  • Grids and the variations dialog draw the small ingest thumbnails (loading="lazy"); the run dialog prefetches the images of neighbouring runs.
  • Memoised dialog work and stable Data Grid props; a large-build perf spec (PW_PERF=1, own CI job) measures the cost of socket updates.
  • Run and build lists no longer duplicate or lose entries on socket events or late answers; a late testRun_created never overwrites a newer testRun_updated, and an update for a deleted run does not bring it back.

Live updates & API integration

  • Socket handshake sends the user's token; join / leave follow the selected project and build, and are re-sent on connect. Events are listened to one by one, so the rooms' connect listener stays.
  • Server-initiated disconnect logs out (see Breaking changes).
  • Error handling (handleResponse):
    • non-2xx answers reject with the body's message, falling back to the status text;
    • a body that is not JSON (the proxy's HTML page while the API restarts) no longer throws SyntaxError; a 401 still logs out, and the error is always a string;
    • after login you return to the full link, query included.
  • Bulk approve, reject and delete show partial results and use each run's own merge flag; an empty answer body counts as all succeeded.
  • Registration flag: 404 from the API counts as enabled. The Gemini API key is shown as a masked hint.
  • Stale-answer guard (staleGuard) keeps only the latest request for a run, build or build list.

Removed

  • Merge branches panel. The API endpoint stays.
  • Guided tour (react-joyride, GuidedTour*, "Take a tour" in the Help menu); the Help menu links to the documentation instead.
  • Cut before 6.0.0 (these existed only in 6.0 release candidates, not in 5.6.0):
    • variation grouping in the run list ("Group by"), the View menu, tile density and the grid's diff toggle (cards always show the diff thumbnail when there is one);
    • the "Recently opened" projects group (favourites, then other projects);
    • the difference blend of the Fade view (Fade stays);
    • the browser-side Diff without noise view and the noise hint's Blur / DINOv2 area overlays;
    • the run list Noise filter and noise sort (the Noise column stays, unsortable);
    • post-analysis threshold fields (amplitude, solid share, distance);
    • the build-level noise caption and the "runs look like a renderer change" banner.

Fixed

  • Bulk delete, clear-ignore-areas and download acted on the selection of another build; they now send only selected runs that are shown, and the selection clears when another build opens.
  • Drawn, unsaved ignore areas were wiped by any update to the open run (a comment, another reviewer); a failed save no longer clears the "unsaved" flag.
  • Ignore areas stay inside the image.
  • The images zip overwrote files of the same test in several browsers or viewports; names now include the tags, repeats are numbered.
  • Left arrow on the first run opened the last one; A / X used stale callbacks.
  • Escape closes the run dialog (still asking about unsaved ignore areas); Escape inside the comment box closes only the box. Dialog hotkeys keep working after clicking the diff switch.
  • A failing comment save keeps the box open with the draft.
  • Late answers replaced current ones (run or build opened from a URL, build list after switching projects or searching).
  • Tag filter: a value shared by several fields matches in any of them.
  • Saving a compare area inverts the selected area and warns when none is selected, instead of storing [].
  • Icon buttons carry their own accessible names; the Tooltip wrapper accepts only title and children.
  • Guests no longer see project create, edit and delete controls.
  • Opening a variation full size in the matching-variations dialog blanked the page in production builds (React #130).
  • A role the server rejects reverts in the user list; a failed user load shows an error.
  • window.open for "Open an issue in GitHub" passes noopener,noreferrer.
  • Dark theme: Data Grid borders follow the theme divider; secondary colour is a neutral slate and Reject is error-coloured.

Stack & tooling

Package 5.6.0 6.0.0
Node (engines) >=20 >=26
React / React DOM 18.3 19.3
react-konva 18.2 19.3
konva 7.2 10.9
@mui/material 5.18 9.5
@mui/x-data-grid 6.20 9.17
react-hotkeys-hook 4.4 5.3
react-router-dom 7.18.4 7.18.5
@vitejs/plugin-react 6.1 6.2
TypeScript 5.2 6.0
ESLint 8 (.eslintrc.cjs) 9 (flat config)
Jest 30 + ts-jest yes replaced by Vitest 5.1
@testing-library/react 14 16
husky 8 (inactive v4-style block) 9 (pre-commit hook)
react-joyride 2.5 removed
@mui/styles, react-debounce-input, react-material-ui-form-validator yes removed (sx/styled, DebouncedTextField, own form validation)
nginx runtime image 1.25 (Alpine 3.17) 1.30 (alpine-slim)
Vite, Playwright, socket.io-client 8.3.5, 1.64.1, 4.8.4 unchanged
  • Unchanged: vite build; output is static files served by nginx.
  • Hotkeys follow the physical key (react-hotkeys-hook 5): letter hotkeys sit on the same key position on non-Latin layouts and move on AZERTY.
  • ESLint enforces the import flow (pages → components → contexts → services → types), fetch only in src/services, no MUI internals; eslint-plugin-sonarjs runs in npm run lint; knip (npm run knip) and the bundle-size check run in CI.
  • minimatch 10.x consumers are overridden to 11.0.5 (GHSA globparts DoS, dev-time tooling only).
  • Docker image: published for linux/amd64 and linux/arm64 by .github/workflows/docker.yml on a vX.Y.Z tag; the builder runs once on the build platform.
  • CI: lint, typecheck, unit tests with coverage, build, bundle-size check, Playwright (functional, Docker screenshots, perf) and the E2E smoke; actions are SHA-pinned; Dependabot covers npm, Docker and GitHub Actions.
  • Agent and contributor rules: AGENTS.md, CLAUDE.md, CONTRIBUTING.md, docs/TESTING.md.

Testing

  • Vitest replaces Jest: unit and component tests next to the code, renderWithProviders, coverage thresholds in vitest.config.ts that only go up.
  • At the final merge: 1635 unit tests and 502 Playwright tests passing (as reported in #502). At 5.6.0 there were 47 unit tests and 123 Playwright tests.
  • Visual contract: integration_tests/test/screens.spec.ts shoots every view and notable state in light and dark; baselines are produced only in Docker (npm run test:pw:docker, npm run test:pw:docker:update).
  • Specs use page objects only (integration_tests/pages); API mocks in integration_tests/utils/mocks.ts.
  • E2E smoke against the real backend stack (PW_E2E=1 npx playwright test, integration_tests/e2e): seeded admin login, a run posted through the API is approved, a run posted to the open build arrives over the socket; CI runs it also with the API in strict socket mode.
  • Perf spec (PW_PERF=1, integration_tests/perf): a large build with socket events.
  • Definition of done: npm run lint, npm run typecheck, npm test, npm run build, npm run test:pw:docker.

What's Changed

  • fix(projects): hide project create, edit and delete from guests by @pashidlos in #430
  • ci(docker): build and publish the ui image for amd64 and arm64 by @pashidlos in #431
  • fix(ignore-areas): keep ignore areas inside the image by @pashidlos in #432
  • Modernize the frontend: tests, MUI 7, agent rules, new theme with dark mode by @pashidlos in #434
  • test(e2e): cover SSO login, user deletion, the build menu and login errors; fix the build menu dialogs by @pashidlos in #435
  • fix: Escape and hotkeys in test details, merge branch encoding, failing comment save by @pashidlos in #436
  • fix: bugs from the frontend review — wrong runs in bulk actions, lost ignore areas, stale answers by @pashidlos in #438
  • refactor: address SonarCloud findings on #438 by @pashidlos in #439
  • feat(project-form): rendering-noise filter in each library's config by @pashidlos in #440
  • feat(project-form): DINOv2 check under the rendering-noise filter by @pashidlos in #441
  • feat(projects): favourites, recently opened and search in the project list and switcher by @pashidlos in #442
  • Frontend refactoring, review and live-update fixes, joint e2e smoke by @pashidlos in #444
  • feat(test-runs): undo an approval or rejection from its toast (U5) by @pashidlos in #446
  • feat(test-variations): merge branches with POST /test-variations/merge (A1) by @pashidlos in #447
  • feat(socket): join the open project's and build's rooms (A3) by @pashidlos in #448
  • build(deps): safe updates for 6.0 — Node 26, nginx 1.30, pinned actions by @pashidlos in #449
  • refactor: behaviour-neutral cleanup before 6.0.0 by @pashidlos in #450
  • ci: run the e2e backend stack with a generated JWT_SECRET by @pashidlos in #455
  • chore: require Node 26 by @pashidlos in #454
  • feat(project-form)!: one "Ignore rendering noise" switch by @pashidlos in #456
  • feat(socket): log out when the server closes the socket by @pashidlos in #457
  • refactor: move off GridLegacy to the MUI 7 Grid by @pashidlos in #459
  • refactor: ready for MUI 9 — slotProps, sx, own form validation by @pashidlos in #461
  • refactor: split TestDetailsModal and TestRunList under 300 lines by @pashidlos in #463
  • chore(lint): warnings become errors, strict integration_tests by @pashidlos in #464
  • fix: tag filter across fields, compare area, icon button names by @pashidlos in #465
  • refactor: minimal 6.0 cleanup by @pashidlos in #466
  • docs: AGENTS.md without library versions by @pashidlos in #467
  • fix: undo sends the decision it reverts; refused decisions warn by @pashidlos in #468
  • feat: rendering-noise hint on unresolved runs, replacing the noise filter by @pashidlos in #469
  • F0: shared building blocks (StatusChip, FilterBar, SelectionBar, TagLine, short dates) by @pashidlos in #470
  • feat(noise): browser solid-mask core for Diff without noise (P6a) by @pashidlos in #471
  • feat: project shell (breadcrumb, tabs, Settings, Statistics) by @pashidlos in #472
  • feat: Statistics tab with results per build (F8) by @pashidlos in #473
  • feat: build list rows, selection bar and build header with status counters and noise line by @pashidlos in #474
  • feat: project list redesign (Copy id, settings icon, search hint) by @pashidlos in #475
  • feat: users page with search, role filter, selection bar and joined date by @pashidlos in #476
  • feat: project settings as sections (F14) by @pashidlos in #477
  • feat: variations list and baseline history (F9, F10) by @pashidlos in #478
  • feat: test-run dialog header, info bar and noise chip (F5a) by @pashidlos in #479
  • feat: run list filter chips, View menu and selection bar (F4) by @pashidlos in #481
  • feat: profile page redesign (F13) by @pashidlos in #482
  • feat: test-run dialog view modes, area editing and footer undo (F5b, F6) by @pashidlos in #483
  • feat: Diff without noise view in the test-run dialog (F7 UI) by @pashidlos in #484
  • test: reconcile tour, screens, coverage and thresholds (F12) by @pashidlos in #485
  • build: override minimatch 10.x to 11.0.5 (Dependabot alert 185) by @pashidlos in #486
  • Show sentence-case labels for run and build statuses by @pashidlos in #487
  • fix: redesign tails by @pashidlos in #488
  • chore: remove the guided tour by @pashidlos in #489
  • Clearer project switcher groups and visible comments by @pashidlos in #490
  • test: noise column and filter hidden when the noise check is off by @pashidlos in #491
  • feat: bulk Analyse rendering noise for selected runs by @pashidlos in #492
  • feat: noise analysis off by default, documentation link in help menu by @pashidlos in #493
  • feat: consistent page layout and an Add project page by @pashidlos in #494
  • feat: restyle dialogs to the 6.0 look by @pashidlos in #495
  • refactor: split oversized files, share stale-answer guard, theme overlay colours by @pashidlos in #496
  • test: specs use page objects only by @pashidlos in #497
  • fix: UI consistency from the rc7 review by @pashidlos in #498
  • chore: socket rooms comment without the removed WS_BROADCAST flag by @pashidlos in #499
  • refactor!: remove the build-level noise caption and renderer-change banner by @pashidlos in #500
  • refactor!: cut Diff without noise, noise hint areas, filter and sort, post-analysis thresholds by @pashidlos in #501
  • refactor!: cut variation grouping, tile density/diff toggle, recent projects and the Fade blend by @pashidlos in #502
  • Dependency updates (konva 9.x #433, React 19 with react-joyride 3 and react-konva 19 #437, npm minor/patch group #445, GitHub Actions group #452, Data Grid 9 / konva 10 / TypeScript 6 / react-hotkeys-hook 5 #458, MUI 9 #462) by @pashidlos and @dependabot in #433, #437, #445, #452, #458, #462

Full Changelog: v5.6.0...v6.0.0

Contributors

pashidlos and dependabot
Assets 2
Loading