Skip to content

benchmark: complete dashboard production QA and accessibility gate #92

Description

@TheHiddenObserver

Context

The benchmark dashboard has automated Python, schema, TypeScript, build, staleness, and Chromium E2E coverage. PR #113 completes the product-level production-path QA pass for PR #76, covering nested asset loading, cross-browser behavior, interaction semantics, and accessibility.

Roadmap: PR #89. Branch synchronization #90 and canonical CV data #91 are complete.

Goal

Establish and execute a reproducible final QA gate for the deployed dashboard under docs/assets/benchmarks/.

Production-path checks

The permanent production Playwright configuration serves the repository root and opens the committed deployment at:

/docs/assets/benchmarks/

The suite verifies nested-path JS/CSS/JSON loading, console/runtime errors, refresh behavior, metadata/data links, and explicit empty states.

Interaction checks

The deployed suite exercises:

Environment → Category → Metric scope → Model → Variant → Penalty → Solver → Scale → Backend → External framework

It verifies context-derived options, deterministic upstream reset, backend/external-framework filtering, scale selection, overview identities, explicit CV failure disclosure, sorting, metric-panel visibility, and filter-synchronized chart-data tables.

Browser matrix

The production suite contains 5 product-level cases and runs all of them in:

  • Chromium/Chrome;
  • Firefox;
  • WebKit/Safari-compatible engine.

Validated result: 15/15 production browser cases passed.

Accessibility checks

Completed coverage includes:

  • skip navigation and a main landmark;
  • keyboard reachability and Enter/Space activation for generated scale/sort controls;
  • select accessible names;
  • native metric-panel disclosure buttons with aria-expanded/aria-controls;
  • visible focus indication;
  • explicit text-contrast checks;
  • chart image semantics plus exact, filter-synchronized timing/speedup tables with full labels;
  • reduced-motion fallback;
  • no primary interaction requiring pointer-only input.

Documentation integration

  • the interactive dashboard remains linked from repository documentation navigation;
  • English/Chinese benchmark guidance points live source/run counts to inventory instead of undated fixed prose;
  • a dated 2026-08-07 snapshot records 9 registered/available/parsed sources and 1,796 normalized runs;
  • frontend/e2e-production/README.md documents the final production smoke procedure;
  • no unstable screenshots were committed.

Non-goals

  • no URL-persisted filter state;
  • no full mobile redesign;
  • no table virtualization or data-bundle partitioning without measured need;
  • no new benchmark family;
  • no schema change unrelated to a verified QA defect.

Acceptance criteria

  • The production build loads correctly from the nested documentation path.
  • Chromium, Firefox, and WebKit/Safari smoke results are recorded.
  • The complete filter cascade passes the documented interaction checks.
  • Timing, speedup, overview, and domain panels agree on displayed values and identities.
  • Empty states and unsupported combinations remain understandable.
  • Keyboard navigation and visible focus pass for all primary controls.
  • Chart content has an accessible tabular or textual path.
  • The user guide is integrated into documentation navigation.
  • TypeScript, production build, deterministic staleness, and Playwright checks remain green.
  • PR feat: benchmark dashboard frontend and canonical data pipeline #76 body contains the final QA evidence and known limits.

Completion evidence

PR #113 was squash-merged into feature/benchmark-frontend-dashboard as:

c815b89974f529d5da13f4d8522c81746c4d20a3

PR #113 clean-head validation:

Benchmark Frontend CI #489 — success
6/6 jobs successful
production QA — Chromium + Firefox + WebKit

Post-merge staging validation on the exact PR #76 head:

Tests #1419 — success
Benchmark Frontend CI #490 — success
Maintenance compatibility #432 — success
Release notes validation #432 — success
Release package validation #438 — success

Benchmark Frontend CI #490 includes successful Python 3.9/3.11 data gates, TypeScript/build, deterministic staleness, Chromium regression E2E, and committed-deployment production QA in Chromium/Firefox/WebKit.

PR #76 body now records this final QA evidence and the deliberate scope limits.

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions