Repository navigation
Testing and Verification
PixlPrfct's regression suite is overwhelmingly JavaScript: 198 node:test files in tests/ (1,053 tests in the alpha.21 release run) that exercise the compiled Rust/WASM engine, the editor modules, the MCP server and the server/cloud code, plus 22 Playwright specs that drive a real Chromium editor. The Rust crate itself has only 3 unit tests; engine behaviour is tested through the WASM bindings from Node. The single most important rule: in this project "verified" means a named check was actually run and its evidence recorded — local tests passing is reported separately from installed builds, shipped releases and interface verification, and none of those is implied by the others.
| Key files | Entry points | Depends on | Used by | Related docs |
|---|---|---|---|---|
tests/*.test.mjs, tests/cloud-db.mjs, tests/helpers/*.mjs, tests/fixtures/figma/*.fig, tests/browser/*.spec.mjs, tests/browser/fixtures.mjs, playwright.config.mjs, scripts/acceptance/*.mjs, src/lib.rs (mod tests), inspector-parity.json
|
npm test, npm run test:browser, npm run mcp:check, npm run desktop:check, npm run test:idle, npm run test:sleep, npm run benchmark:editor
|
Built web/pkg (wasm-pack), Node node:test + node:sqlite, Rust toolchain, @playwright/test + Chromium, @modelcontextprotocol/sdk client |
Developers before PRs; release preparation; the Windows prerelease workflow runs a subset | RELEASE_ACCEPTANCE.md, CAPABILITY_MATRIX.md, PERFORMANCE.md, PILOT_PLAN.md, AGENTS.md, CONTRIBUTING.md |
flowchart TB
subgraph Fast["Unit / integration (npm test)"]
R["cargo test<br/>src/lib.rs mod tests (3)"]
N["node --test tests/*.test.mjs<br/>198 files"]
end
subgraph Gates["Static gates"]
L[npm run lint]
T[npm run typecheck]
F["cargo fmt --all --check<br/>cargo clippy -D warnings"]
C[generate-editor-api.mjs --check]
end
subgraph Browser["Browser (npm run test:browser)"]
P["Playwright, Chromium<br/>tests/browser/*.spec.mjs<br/>port 4198, dist/ build"]
end
subgraph Accept["Acceptance (manual, long-running)"]
I["test:idle (40 min real time)"]
S["test:sleep (real macOS sleep)"]
B[benchmark:editor]
PR[pilot-report.mjs]
end
W[(web/pkg<br/>wasm-pack build)] --> N
W --> P
W --> Accept
| Requirement | Why |
|---|---|
Node.js 24 (CI and docs use 24; package.json engines is >=22.13.0) |
node:sqlite (DatabaseSync) is used by tests/cloud-db.mjs, server/database.mjs, scripts/self-host/backup.mjs. It is available unflagged from 22.13; the project documents and tests on 24 |
Rust stable + wasm32-unknown-unknown, wasm-pack
|
cargo test; building web/pkg
|
npm ci |
Dependencies from the committed package-lock.json
|
npm run build:editor before npm test
|
126 of the Node test files import ../web/pkg/pixlprfct_core.js and read pixlprfct_core_bg.wasm; web/pkg/ is git-ignored |
npx playwright install chromium |
Browser suite |
| macOS for screenshot comparison |
snapshot() in tests/browser/fixtures.mjs calls toHaveScreenshot only when process.platform === 'darwin'
|
| Command | What it runs |
|---|---|
npm test |
cargo test && node --test tests/*.test.mjs |
node --test tests/grid-layout.test.mjs |
One file (any subset of tests/*.test.mjs) |
node --test --test-name-pattern "atomic" tests/mcp-runtime.test.mjs |
Filter by test name |
npm run mcp:check |
Catalog drift check (generate-editor-api.mjs --check) + node --test tests/mcp*.test.mjs
|
npm run desktop:check |
node --test tests/desktop.test.mjs |
npm run lint / npm run typecheck
|
ESLint (ignores dist, .next) / tsc --noEmit --incremental false
|
cargo fmt --all --check; cargo clippy --all-targets --all-features -- -D warnings
|
Rust format/lint gates (listed in CONTRIBUTING and CAPABILITY_MATRIX; not npm scripts) |
npm run test:browser |
playwright test (builds the editor, then serves dist/) |
npm run test:browser -- tests/browser/daily-workflow.spec.mjs --grep responsiveness --workers=1 |
Latency gate alone (documented in RELEASE_ACCEPTANCE step 4) |
npm run test:browser -- --grep-invert responsiveness --workers=1 |
Functional browser suite without the latency gate |
npm run test:browser:update |
Rewrites screenshot baselines. Inspect every changed image before committing |
npm run test:idle |
scripts/acceptance/mcp-idle.mjs 40 |
npm run test:sleep -- 40 |
scripts/acceptance/desktop-sleep.mjs (macOS only) |
npm run benchmark:editor [-- file.pixl] |
scripts/acceptance/benchmark.mjs |
node scripts/acceptance/pilot-report.mjs results.json |
Pilot metrics |
The full pre-PR list in CONTRIBUTING.md: npm run build, npm test, npm run lint, npm run typecheck, npm run test:browser, node scripts/generate-editor-api.mjs --check, cargo fmt --all --check, cargo clippy --all-targets --all-features -- -D warnings.
Note: There is no CI workflow that runs these checks. CONTRIBUTING.md states "Automatic CI and required status checks are disabled". CAPABILITY_MATRIX.md still says "CI never updates baselines" and "The visual job uses GitHub's macos-15 Apple Silicon runner", but no such job exists in
.github/workflows/. Only the Windows prerelease workflow runs tests (a desktop/import subset). See CI-CD-and-Releases.
Only src/lib.rs has a #[cfg(test)] mod tests block, with three tests:
| Test | Covers |
|---|---|
history_restores_edits_and_invalidates_redo |
Undo/redo stack semantics |
load_is_atomic_and_roundtrips |
load failure leaves state unchanged; snapshot round trip |
hit_testing_respects_ellipses_visibility_and_order |
hit_test geometry, hidden layers, z-order |
None of the other 26 modules (grid.rs, layout.rs, components.rs, variants.rs, slots.rs, variables.rs, …) has Rust-side tests. Their behaviour is covered by Node tests calling the compiled Editor (see tests/wasm.test.mjs, tests/grid-*.test.mjs, tests/components.test.mjs, etc.). Consequence: a Rust change is only meaningfully tested after wasm-pack build and node --test, not by cargo test alone.
Counts are files and test( call sites per area (approximate; some files generate tests in loops). Documentation reports 994 JavaScript tests at alpha.14 (RELEASE_ACCEPTANCE.md).
| Area | Files | Tests | Files (tests/<name>.test.mjs) |
|---|---|---|---|
| Layout: auto layout, grid, constraints, frames | 41 | ~247 |
auto-layout-inspector, breakpoint-preview, clipping, constraints, container-rotation, device-chrome, frame-enclosure, frame-selection, frames, grid-* (14 files), layout, layout-* (14 files), masonry, parent-alignment, resizing, responsive-grid-areas, scaling
|
| Canvas tools, vectors, selection, pages, inspector chrome | 33 | ~196 |
arrangement, clipboard, context-menu, contextual-actions, curves, design-system-gaps, dropdown-control, groups, guide-snap, input-base, inspector-disclosure, layer-drop, layer-style, locking, multi-frame-paste, native-pinch, page-drop, page-typeahead, pages, pan-inertia, paste-replace, path-join, path-split, pencil, popover-position, rectangle-actions, selection, snapping, toolbar-tools, tooltip-position, vector-edit, vectors, zoom-inertia
|
| Styles, variables, paints, strokes, effects | 30 | ~120 |
appearance, background-promotion, box-variables, color-picker-cancel, color-styles, corners, effect-editor, effect-styles, glass-preset, gradient-edit, gradients, guide-styles, inner-shadows, layer-blur, page-background, paint-* (5), shadow-variables, shadows, stroke-* (3), strokes, text-styles, variable-color-input, variable-workspace, variables
|
| Cloud API, self-host, teams, libraries, persistence | 23 | ~82 |
cloud-automatic, cloud-bodies, cloud-documents, cloud-history, cloud-session, libraries, library-lifecycle, library-overrides, local-libraries, local-drafts, local-persistence, recovery-store, project-file, project-navigation, loading-screen, static-build, self-host, self-host-preview, team-activity, team-collaboration, team-onboarding-operations, team-session, pilot-report
|
| Components, variants, slots, instances, assets | 12 | ~65 |
asset-placement, assets, component-properties, component-selection, component-sort, component-swapping, components, instance-context, nested-components, push-overrides, slots, variants
|
| Starter controls | 12 | ~54 |
button-, calendar-, checkbox-, file-, number-, password-, radio-, segmented-, switch-, textarea-starter, controls, starter-defaults
|
Import (.fig, SVG) |
6 | ~53 |
figma-import, figma-layout-parity, svg-import, svg-use, svg-radial, svg-gradients
|
| Prototyping and behaviors | 9 | ~50 |
prototype, prototype-flows, -hover, -multiple, -overlays, -press, -timers, -transitions, behaviors
|
| MCP and agent bridge | 6 | ~45 |
mcp-server, mcp-runtime, mcp-discovery, mcp-installation, agent-transport, live-editing
|
| SVG Animator | 2 | ~36 |
svg-animation, svg-animation-keyframes
|
| Text and typography | 7 | ~26 |
base-text, font-weights, google-fonts, text-editing, text-sizing, typography, typography-controls
|
| Rendering and export | 7 | ~26 |
export, batch-export, code-export, css-renderer, export-chunks, images, viewport-renderer
|
| Desktop (Electron boundaries, signing, updates, preferences) | 4 | ~13 |
desktop, desktop-signing, desktop-updates, desktop-preferences
|
| Engine bindings | 1 | 8 | wasm |
| Icon library | 2 | 3 |
icon-library, territory-icon-coverage
|
| Total | 195 | ~1,024 |
Notable suites:
-
wasm.test.mjs— complete editing sessions through the rawEditorbindings: add/update/duplicate/undo/redo, pages, atomicload. -
mcp-server.test.mjs— workspace confinement, symlink rejection, atomic save; a real MCPClientoverInMemoryTransportand overStdioClientTransport(asserting no non-protocol stdout); bridge Origin/token/Host rejection; desktop-origin pairing; session-ID preservation without replay. -
mcp-runtime.test.mjs— catalog excludes lifecycle methods; atomic builds; all native controls; Grid; components; bounded reads; reload invalidation; chunked export reconstruction (usesscripts/acceptance/complex-fixture.mjs). -
mcp-discovery.test.mjs— private registry, restart on a different port, auto-port fallback, recovery after a simulated 40-minute sleep while discovery hangs. -
mcp-installation.test.mjs— redaction, secret preservation, cross-session conflicts, symlinks, Docker credentials only in environment, a real stdio client provisioning a SQLite trial.
| Helper | Use |
|---|---|
tests/cloud-db.mjs cloudDatabase()
|
A Cloudflare D1-compatible shim over node:sqlite DatabaseSync(':memory:'). It executes every drizzle/*.sql migration in order, then exposes prepare(q).bind(...).all()/first()/run(), batch() (wrapped in BEGIN/COMMIT/ROLLBACK), exec, close. Used by cloud-automatic, cloud-bodies, cloud-documents, cloud-history to test lib/documents.mjs with real SQL |
tests/helpers/team-fixture.mjs teamFixture({server, baseURL, sqlitePath})
|
Builds a self-host config, connects the database, runs migrateAuth/migrateDocuments, optionally startServer, with synthetic accounts and in-memory mail |
tests/helpers/export.mjs completeExport(runtime, args)
|
Follows pixlprfct_export cursors until complete, asserting revision stability |
tests/helpers/figma-library.mjs figmaLibraryFixture()
|
Synthetic imported library data |
tests/fixtures/figma/*.fig |
Three small MIT-licensed .fig binaries from OpenFig (deflate + Zstandard archives, open vector network) — see its README |
scripts/acceptance/complex-fixture.mjs complexFixture()
|
Public, deterministic 5-page / 727-layer document with linked components, variants, a slot, text/effect/color styles and mode-dependent variables |
mcp/projects.mjs initializeEngine(), createHeadlessEditor(), headlessOptions()
|
Load WASM in Node with deterministic text metrics; used by tests, Playwright fixtures and acceptance scripts |
-
testDir: ./tests/browser,testMatch: **/*.spec.mjs,workers: 1,fullyParallel: true,retries: CI ? 1 : 0,forbidOnlyon CI, 30 s timeout,list+htmlreporters. -
use:baseURL http://127.0.0.1:4198, 1440×1000 viewport, DPR 1, dark colour scheme,en-US,UTC,reducedMotion: 'reduce', trace retained on failure, screenshot on failure. -
webServer:npm run build:editor && node scripts/acceptance/serve.mjswithPIXL_ACCEPTANCE_PORT=4198; readiness URL/editor/project-home.html; never reuses an existing server. - Screenshot tolerance:
maxDiffPixelRatio: 0.002, animations disabled.
scripts/acceptance/serve.mjs is a minimal static server for PIXL_ACCEPTANCE_ROOT (default dist) with cache-control: no-store and path-traversal protection; / maps to /editor/project-home.html.
tests/browser/fixtures.mjs extends Playwright's test: it aborts every request whose host is not 127.0.0.1 (offline, deterministic) and fails the test if any pageerror occurred. Helpers: fixture({components}) builds a document with a headless engine; seed(page, data) writes it to localStorage['pixlprfct.drafts.v1']; open(page) loads /editor/index.html?draft=acceptance; saved, rename, field, select; snapshot(page, name, locator) compares a screenshot on macOS only.
| Spec | Tests | Scope |
|---|---|---|
editor.spec.mjs |
13 | Projects create/reopen; stale-tab and conflict recovery; quota failure; Pages; text styles; shadow variables; slots; Motion dock; variants; set labels; layer rows |
svg-animation.spec.mjs |
42 | SVG Animator workflows |
libraries.spec.mjs |
4 | Publish/add/update/reopen; release file import; native design-archive library import; live MCP library operations over a real WebSocket |
daily-workflow.spec.mjs |
2 | 727-layer complex round trip (lossless JSON after normalized import); responsiveness gate (p95 < 100 ms, PIXL_PROFILE=1 attaches a CPU profile) |
styles.spec.mjs |
2 | Page Styles editors |
library-overrides, live-editing, team, team-activity, team-onboarding, product-site
|
1 each | Author-approved overrides; two simulated co-editors; simulated team with restart/restore; activity; onboarding; site/ page |
Screenshot baselines live in tests/browser/<spec>-snapshots/. The snapshot() names in specs include pages-and-page-paint.png, type-settings.png, text-styles.png, shadow-variables.png, motion.png, set-labels.png, library-add-to-existing-project.png, and the three page-*-style-editor/page-document-styles images. The PNGs are binary and are not part of this documentation snapshot. Linux runs the functional assertions only; pixels are compared against macOS 15 baselines.
| Script | Purpose | Output / pass criterion |
|---|---|---|
serve.mjs |
Static server for Playwright (port PIXL_ACCEPTANCE_PORT, default 4197; root PIXL_ACCEPTANCE_ROOT, default dist) |
— |
benchmark.mjs |
Headless engine timings on a synthetic 727-layer, 5-page document (or a .pixl path argument): load, snapshot, textContentEdit, textResizePreview, pageSwitch, plus a 20-update MCP build batch |
JSON with medians and final document SHA-256. Measurements, not FPS claims |
complex-fixture.mjs |
Exports complexFixture() (see above) |
— |
mcp-idle.mjs [minutes] |
Real socket transport + headless editor connected via discovery to createPixlMcp; waits N real minutes; reads; restarts the server; wake(); reads again |
Prints {phase:'passed', …, osSleepTested:false}
|
desktop-sleep.mjs [minutes] |
macOS only. Spawns the MCP server over stdio, waits for the operator to put the Mac to sleep ≥N minutes, reads real pmset Sleep/Wake events, reconnects, compares document digests |
Writes .local/acceptance/desktop-sleep.json with osSleepTested:true only on a real sleep |
pilot-report.mjs [input.json] |
Aggregates participant pilot sessions (completion, unassisted %, day-7 return, fallback, data loss) | JSON report; tested by tests/pilot-report.test.mjs
|
Pick the lowest layer that exercises the behaviour.
Engine behaviour (most common). Import the WASM package directly:
import test from 'node:test';
import assert from 'node:assert/strict';
import {readFile} from 'node:fs/promises';
import init,{Editor} from '../web/pkg/pixlprfct_core.js';
await init({module_or_path:await readFile(new URL('../web/pkg/pixlprfct_core_bg.wasm',import.meta.url))});
test('duplicate keeps the layer on its page',()=>{
const e=new Editor();
try{
const id=e.add('rectangle',0,0);
const copy=e.duplicate(id);
const shapes=JSON.parse(e.snapshot()).shapes;
assert.equal(shapes.find(s=>s.id===copy).page_id,shapes.find(s=>s.id===id).page_id);
}finally{e.free();}
});Alternatively import {initializeEngine,createHeadlessEditor} from '../mcp/projects.mjs' for deterministic text measurement. Always free() engines.
Editor modules. Most web/*.js modules export pure functions (layoutText, svgDocument, compileCssScene, codeExportFiles, toolGroups…); import them directly. Where a module touches the DOM, tests install minimal stubs (e.g. globalThis.document={createElement:()=>new Element()} in color-picker-cancel.test.mjs). A few tests assert against CSS/HTML source text.
MCP behaviour. Use createAgentRuntime({editor, createEditor:createHeadlessEditor, options:headlessOptions}) and runtime.request(...) for runtime logic; use createPixlMcp({workspace, live:false}) with InMemoryTransport.createLinkedPair() and an SDK Client for tool-level tests; use createEditorBridge({port:0}) and ws for bridge tests. Create temp dirs with mkdtemp and clean up in t.after(() => rm(dir,{recursive:true,force:true})).
Cloud / D1 routes. const db = cloudDatabase(); build Request objects with the identity header the Sites host supplies (oai-authenticated-user-id) and call the API (e.g. documentApi from lib/documents.mjs). Add schema through a new drizzle/*.sql migration — the shim applies all of them.
Self-host / teams. teamFixture({server:true}) from tests/helpers/team-fixture.mjs.
Browser. Import {test, expect, fixture, seed, open, snapshot} from ./fixtures.mjs, seed a draft, open the editor, interact through accessible labels (page.getByLabel). Add a snapshot() only for reviewed interface states; generate it with npm run test:browser:update on macOS and inspect the image.
New files named tests/<area>.test.mjs are picked up automatically by npm test; MCP tests should start with mcp to be included in npm run mcp:check.
The rules come from AGENTS.md, CAPABILITY_MATRIX.md and RELEASE_ACCEPTANCE.md:
- "Implemented" = code exists and the cited local checks pass. It does not mean complete workflow coverage, an installed update, a merged branch or a public release (CAPABILITY_MATRIX header).
- Visual checks do not establish feature completeness. Do not present placeholders as finished or promise complete equivalence with another product. Screenshot baselines detect later regressions; they do not establish compatibility or legal clearance.
- Keep evidence classes separate: source change, local test pass, packaged build, locally installed build, published release. The capability matrix has separate columns (Engine/MCP, Interactive UI, Visual evidence, Installed/released).
-
Record PASS / FAIL / NOT RUN / NOT CERTIFIED explicitly, with test counts and the location of raw logs (typically under git-ignored
.local/…). A failing gate is reported as failing, not relaxed (e.g. the 2026-10-30 responsiveness FAIL at 199.2 ms p95 was recorded before the later 58.4 ms PASS). -
Gap tally discipline:
docs/TT-Design-System-Gaps.mdkeeps completed items struck through with a dated resolution and evidence; local fixes are distinguished from shipped changes. -
Proxies are named as proxies: the timer-only idle test (
osSleepTested:false) is not a sleep test; the latency test is "a frame-opportunity proxy, not input-to-photon"; Windows CI startup is "a startup check, not a full interactive test".
inspector-parity.json is a data record, not a test input: it lists inspector sections observed per selection type (observedSelections for rectangle/arrow/text/image/frame/instance), what each pass added, remainingObservedGaps and stillToInspect, with status "Partial". No test reads it.
From RELEASE_ACCEPTANCE.md: (1) build + all gates + browser suite; (2) 727-layer complex round trip; (3) recovery simulations; (4) responsiveness p95 < 100 ms; (5) manual interface checks at the recorded panel width and scale; (6) packaged desktop build + real OS sleep probe; (7) participant pilot per PILOT_PLAN.md.
- Stale
web/pkgsilently tests old engine code. Rebuild withnpm run build:editorafter anysrc/*.rschange. -
web/editor-api.jsis committed;mcp:checkfails if it differs from what the currentweb/pkgwould generate. - Network tests bind loopback ports (
port:0); a sandbox without loopback sockets will failmcp-*andagent-transporttests. - Browser tests block all non-127.0.0.1 traffic, so features depending on Google Fonts or remote assets must have local fallbacks in tests.
-
playwright.config.mjsrebuilds the editor on every run (reuseExistingServer:false), which is slow but guarantees a fresh build. - Snapshot comparison is skipped off macOS without failing — a Linux run passing does not mean visuals are unchanged.
- Test counts in docs drift (924 → 966 → 978 → 985 → 994 over successive passes); quote the number from the run you made.
- Rust-level coverage is minimal (3 tests); most modules rely on JS-through-WASM tests.
- No automated CI for tests, lint, typecheck or browser suites.
- Real OS sleep/wake reconnection has not been accepted; Docker deployment is contract-tested only; Windows has only install/startup/uninstall automation.
- Performance: the 100 ms p95 gate passes; the broader 16.7 ms frame objective (B33) remains open (PERFORMANCE.md).
- CI-CD-and-Releases — which checks run in workflows
- MCP-Server — the subsystem behind the MCP, idle and sleep tests
-
Build-System —
build:editor,web/pkg,dist/ - Rust-Document-Engine
- Cloud-API-and-Database, Self-Hosted-Server
- Interface-Design-Guidelines — the reference/verification rules
- Contributing-and-Conventions
- Config and helpers: package.json scripts, playwright.config.mjs, tests/cloud-db.mjs, tests/browser/fixtures.mjs,
tests/helpers/,scripts/acceptance/ - Rust tests:
src/lib.rs(mod tests) - Docs: RELEASE_ACCEPTANCE.md, CAPABILITY_MATRIX.md, PERFORMANCE.md, PILOT_PLAN.md, AGENTS.md, CONTRIBUTING.md, inspector-parity.json
The full browser suite includes Design and SVG parent panels, collapsed icons and side flyouts, cursor-following tear-outs, restoration, direct stage picking/movement, Pen/Pencil, path topology, Add/Replace import, and independent gradient/object animation. See tests/browser/design-panels.spec.mjs and tests/browser/svg-animation-*.spec.mjs. Pure drawing/gradient/topology model checks live under tests/svg-animation-*.test.mjs.
Native window acceptance uses scripts/acceptance/panels-desktop.mjs with a temporary profile. Release logs are retained locally under .local/release-alpha21/; public source/build provenance and package SHA-256 hashes accompany the GitHub release. Installed Windows startup verification does not imply full interactive native Windows coverage, real sleep/wake, or a human pilot.
Start
Core
- Rust Document Engine
- Document Model & File Format
- Web Editor Architecture
- Rendering & Export
- Build System
Editor features
- Canvas Tools & Editing
- Layout, Auto Layout & Grid
- Components, Variants & Slots
- Styles, Variables & Paints
- Prototyping, Behaviors & Controls
- Import: Design Files & SVG
- SVG Animator
- Panel Workspace — tear-outs, icons, side flyouts and saved layouts
- Icon Library
Platforms
- Hosting Modes
- Self-Hosted Server
- Cloud API & Database
- Teams, Libraries & Collaboration
- Desktop App
- MCP Server
Practice