Skip to content

Commit 12dfe87

Browse files
committed
feat(engine): promote dense scatters to canvas automatically
Flips AUTO_ENABLED, so point marks past 1,000 compiled points resolve to canvas without a `render` field. Explicit `render: 'svg'` still opts out, and every existing visual baseline is byte-identical (no story crosses the threshold), which is the regression gate for the flip. Adds two gallery demos: a 2,900-point keyed year morph on the auto path (the transition the SVG 500-mark cap used to refuse outright) and a canvas-vs-SVG parity pair at 200 points. Documents a rendering difference the parity demo surfaced: canvas paints every fill and then every stroke -- two batched draw calls rather than two per point, which is what makes thousands of dots cheap -- so overlapping dots can show a stroke over a neighbour's fill where SVG would not. Sparse clouds are indistinguishable; tight clusters show it. Perf gate (e2e/perf, real browser): a 5,000-point canvas update samples 62 frames at mean 18.6ms / p95 18.6ms against a 33/50ms budget.
1 parent d7ff0c1 commit 12dfe87

18 files changed

Lines changed: 563 additions & 26 deletions

.claude/rules/visual-regression.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,14 @@ Don't rely on it for:
2222

2323
Before each screenshot the harness disables animations via an injected stylesheet and hides the Ladle dev overlays, then waits for fonts to settle, so timing and layout are deterministic.
2424

25+
## Canvas mark mode stories
26+
27+
The injected stylesheet kills CSS animations. It cannot touch the JS scheduler that drives the canvas entrance, so any baseline-captured story rendering points on canvas must set `animation: false` in its spec, or the screenshot lands mid-entrance at a nondeterministic alpha.
28+
29+
Do NOT reach for `emulateMedia({ reducedMotion: 'reduce' })` instead: `reduced-motion.css` carries rules that would shift the existing baselines.
30+
31+
Interactive-only canvas stories (the year-toggle morph demo) are deliberately left out of the baseline set for the same reason.
32+
2533
## Platform-locked baselines
2634

2735
Baseline PNGs are committed per-platform (`-chromium-darwin.png`, `-chromium-linux.png`, etc.). Font rendering and antialiasing differ across operating systems, so a baseline captured on macOS won't match pixel-for-pixel on Linux. If you're on a platform without committed baselines, run `bun run test:visual:update` to generate your own — they'll land alongside the existing ones rather than replacing them.

docs/architecture.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,8 @@ Visual features are computed per-column based on the column config: heatmap buil
188188

189189
The vanilla adapter renders charts by building a fresh SVG element from the `ChartLayout`. It doesn't diff or patch. On every update or resize, it tears down the old SVG and creates a new one.
190190

191+
Scatter charts past a point-count threshold render their dots on a `<canvas>` instead (see [canvas mark mode](spec-reference.md#canvas-mark-mode)). The canvas is a sibling inserted *before* the SVG and painting the background, gridlines and marks; the SVG keeps axes, trendline, annotations and chrome. The engine only records the decision as `ChartLayout.markRenderMode` -- the renderer keys strictly on its own `canvasMarks` option, so any caller that omits it (SSR, exports) gets a complete SVG regardless of what the layout says.
192+
191193
For tables, it creates HTML elements (table, thead, tbody, etc.) with interactivity wired up: sort headers, search input, pagination controls, keyboard navigation.
192194

193195
Both adapters use `ResizeObserver` to detect container size changes and trigger recompilation at the new dimensions. The responsive system uses breakpoints to adjust layout strategy (label density, legend position, annotation placement).

docs/codebase-map.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -124,14 +124,21 @@
124124
| Graph mount (lifecycle, events, tooltips, legend, search, camera, highlight, update) | `packages/vanilla/src/graph-mount.ts``createGraph()`. Public API: `GraphInstance` (search/zoomToFit/zoomToNode/flyTo/centerAt/getCamera/selectNode/highlight/clearHighlight/getHighlight/setActiveCategories/getActiveCategories/getLegend/update/updateVisuals). `suppressEntrance` mount option skips the reveal on a theme/darkMode-only remount (instant fit, warmup still runs). Physics-feel gates: `SPRINGY_DRAG_MAX_NODES` (5000), `CURSOR_FORCE_MAX_NODES` (2000). |
125125
| Graph force simulation (d3-force wrapper, warmup, tick streaming, sync + worker) | `packages/vanilla/src/graph/simulation.ts``SimulationManager`. Worker protocol: `graph/worker-protocol.ts`; worker URL: `graph/simulation-worker-url.ts`. |
126126
| Graph seeded layout (deterministic initial positions) | `packages/vanilla/src/graph/seed.ts``seedNodePositions()`. Start position is a pure function of `(id, seed, community)`, so the settled layout is reproducible across data reshapes (not d3's order-dependent phyllotaxis). |
127-
| Graph animation scheduler (rAF loop, first-frame arming) | `packages/vanilla/src/graph/scheduler.ts``AnimationScheduler`. Owns running `GraphAnimation`s; the mount's render loop ticks it once/frame and re-arms rAF only while animations are active (zero cost when idle). |
128-
| Graph motion primitives (easings, reduced-motion, generic tween) | `packages/vanilla/src/graph/motion.ts``createTween()`, `resolveEase()`, `prefersReducedMotion()`. Canvas rAF, NOT CSS keyframes; only the easing vocabulary is shared with charts. |
127+
| Animation scheduler (rAF loop, first-frame arming) | `packages/vanilla/src/motion/scheduler.ts``AnimationScheduler`, `Animation`. Shared by graph and the scatter canvas layer; `graph/scheduler.ts` is a re-export shim (`GraphAnimation` is the old alias). Owns the running animations; the host's render loop ticks it once/frame and re-arms rAF only while animations are active (zero cost when idle). |
128+
| Motion primitives (easings, reduced-motion, generic tween) | `packages/vanilla/src/motion/tween.ts``createTween()`, `resolveEase()`, `prefersReducedMotion()`. `graph/motion.ts` is a re-export shim. Canvas rAF, NOT CSS keyframes; only the easing vocabulary is shared with charts. |
129129
| Graph entrance choreography (reveal alpha, staggered start) | `packages/vanilla/src/graph/entrance.ts`. Turns the mount's 0→1 `entranceProgress` into a per-node reveal alpha, quantized so the canvas renderer still batches fills. The camera pull-back + flight lives in `graph-mount.ts` `startEntrance()`. |
130130
| Graph camera flights (animated zoom/pan) | `packages/vanilla/src/graph/camera.ts``createCameraFlight()`, `createCameraFollow()`, `CameraFlightOptions`, `clampK`, `K_MIN`/`K_MAX`. Interpolates two `ZoomTransform`s along d3 `interpolateZoom`; supports a moving-target provider so a flight tracks a still-settling node. `createCameraFollow()` continuously tracks a moving target (e.g. entrance pull-back). |
131131
| Graph focus model (highlight ∩ search + hover-neighborhood, eased crossfade) | `packages/vanilla/src/graph/focus-transition.ts``FocusTransition`. Composes the three emphasis sources into a snapshot pair the renderer crossfades between (composition, not strict precedence). |
132132
| Graph update diff (unified `update()` path classification) | `packages/vanilla/src/graph/update-diff.ts``diffGraphUpdate()`. Classifies a change as `visualOnly` (same node+edge id sets AND equal simulationConfig → position-preserving) vs structural (reheat). Config equality: `graph/update-diff-config.ts` (compares the full resolved simulationConfig, not just `clustering.field` like the old React heuristic). |
133133
| Graph interactive legend (accessible built-in legend) | `packages/vanilla/src/graph/legend.ts``GraphLegendController`. Node-category rows are `aria-pressed` buttons that toggle emphasis via the focus model; edge-category rows are non-interactive swatches. Turn off with `legend: false` (do this if you render your own). |
134-
| Graph spatial index (hit-testing) | `packages/vanilla/src/graph/spatial-index.ts``SpatialIndex`. |
134+
| Spatial index (hit-testing) | `packages/vanilla/src/spatial-index.ts``SpatialIndex<T extends {x,y,radius}>`. Generic over the entry type; graph uses `SpatialIndex<PositionedNode>`, the scatter canvas layer uses `SpatialIndex<ScatterHit>`. `graph/spatial-index.ts` is a re-export shim. |
135+
| Scatter canvas mark layer (canvas-under-SVG for high-cardinality points) | `packages/vanilla/src/scatter-canvas/layer.ts``createScatterCanvasLayer()`. Owns the `<canvas>`, its dirty-flag rAF loop, an `AnimationScheduler`, and the hit-test index. Created by `mount.ts` BEFORE the SVG so DOM order stacks it underneath (`mount.ts` also sets `svg.style.position='relative'`, or the absolutely-positioned canvas would win regardless of order). |
136+
| Canvas mark painting (batched fills, DPR, clip) | `packages/vanilla/src/scatter-canvas/renderer.ts``ScatterCanvasRenderer`. Paint order: background → clip → gridlines → exit ghosts → fill pass → stroke pass → hover ring. Batches by `${fill}|${alpha}`. Second ctor arg pins DPR for export rasters. |
137+
| Canvas render state (struct-of-arrays) | `packages/vanilla/src/scatter-canvas/state.ts``buildScatterCanvasState()`, `flattenFill()`. `markIds` are built from the ORIGINAL `layout.marks` index (`point-${i}`) because a trendline is `unshift`ed onto the array and tooltip descriptors key off that index. Types in `scatter-canvas/types.ts`. |
138+
| Canvas entrance animation | `packages/vanilla/src/scatter-canvas/entrance.ts``playCanvasEntrance()`, `computeEntranceDuration()`, `clampStagger()`. Replicates the CSS point entrance (fade-only, 40% of duration, 2s total stagger budget). Its `totalMs` MUST be passed to `setupAnimationCleanup` -- the DOM-counting estimate sees no point elements in canvas mode. |
139+
| Canvas pointer interactions (tooltips, click/hover) | `packages/vanilla/src/scatter-canvas/interactions.ts``wireCanvasInteractions()`. Listeners on the canvas; the SVG is `pointer-events:none` with legend/annotations/chrome/metrics re-enabled. |
140+
| Mark render mode resolution (`auto`/`svg`/`canvas`) | `packages/engine/src/compiler/mark-render-mode.ts``resolveMarkRenderMode()`, `AUTO_CANVAS_THRESHOLD`. Sets `ChartLayout.markRenderMode` only when canvas. AUTHORITY RULE: `renderChartSVG` keys ONLY on its `opts.canvasMarks`, never on the layout flag, so SSR/exports always emit a complete SVG. |
141+
| Canvas-mode exports (materialize the missing half) | `packages/vanilla/src/export-canvas.ts``materializeCanvasModeSVG()`, `VECTOR_EXPORT_MAX_POINTS`. Vector re-render at or below the cap (byte-identical to SVG mode); above it, vector everything with the dots inlined as one raster `<image>`. Raster formats force vector. |
135142
| Graph gallery stories | `examples/src/gallery/graphs.stories.tsx` + demo specs/data in `graphs.demos.ts`. |
136143
| Framework Graph wrappers (full instance API, `suppressEntrance` on theme remount) | `packages/{react,vue,svelte}/src/Graph.*` + `.../composables|hooks/useGraph.*`. Function-valued options (`tooltip.formatter`, event callbacks) ride a ref-trampoline (stable wrapper reading the latest handler), never the dep array — pinning a formatter would stale-close; recreating on it would remount. Theme/darkMode-only recreation passes `suppressEntrance: true` so the entrance doesn't replay. |
137144
| Release script | `scripts/release.mjs` |

docs/spec-reference.md

Lines changed: 80 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ The `type` field on ChartSpec accepts either a string (`'line'`) or an object wi
119119
| `tooltip` | `boolean \| null` | `true` | all | Tooltip behavior. `null` disables tooltips. |
120120
| `clip` | `boolean` | `false` | all | Clip marks to the chart area. |
121121
| `fillPattern` | `'auto' \| 'none'` | `'none'` | bar, area, arc | `'auto'` layers a per-series SVG pattern (hatch, dots, crosshatch, vertical) over the series color. See the [accessibility guide](accessibility.md#pattern-fills). |
122+
| `render` | `'auto' \| 'svg' \| 'canvas'` | `'auto'` | point | Rendering surface for point marks. `'auto'` promotes to canvas above 1,000 compiled points. See [Canvas mark mode](#canvas-mark-mode). |
122123
| `style` | `'dumbbell' \| 'arrow' \| 'bar'` | `'dumbbell'` | range | Range mark visual: dot-connector-dot, line with arrowhead at the end, or a plain floating bar. |
123124
| `colorByDirection` | `boolean` | `false` | range | Color range marks by direction of change: increases use the theme `positive` color, decreases `negative`. A field-based `color` encoding takes precedence. |
124125
| `units` | `number` | `100` | waffle | Total cells in the grid. Categories normalize to this via largest-remainder rounding. |
@@ -710,11 +711,20 @@ Everything in `AnimationPhaseConfig`, plus:
710711

711712
| Field | Type | Default | Description |
712713
| ---------- | -------- | ------- | ------------------------------------------------------------------ |
713-
| `maxMarks` | `number` | `500` | Largest mark count that still tweens. Above it, updates swap instantly. |
714+
| `maxMarks` | `number` | `500` (SVG) / `20000` (canvas) | Largest mark count that still tweens. Above it, updates swap instantly. |
714715

715716
`maxMarks` is update-only: enter and exit animate via CSS, which does not have
716717
the same per-frame cost.
717718

719+
The default depends on the rendering surface. SVG mode writes attributes on one
720+
DOM element per mark, so its cap is deliberately low. [Canvas mark
721+
mode](#canvas-mark-mode) writes into typed arrays and issues one batched fill per
722+
color bucket, so its default is 20,000. An explicit `maxMarks` overrides both.
723+
724+
The count checked is `max(previous, next)`, not the new layout alone. Exit ghosts
725+
render on the destination surface, so a 4,000-point chart updating down to 400
726+
still has ~3,600 departing marks to animate.
727+
718728
### Data-update transitions
719729

720730
When `animation.update` is enabled and `.update(newSpec)` is called, the chart animates marks from their previous positions to the new layout instead of doing an instant swap. The engine matches marks across layouts using keys derived from data values.
@@ -742,6 +752,75 @@ When `animation.update` is enabled and `.update(newSpec)` is called, the chart a
742752

743753
**Legend-hidden series:** When series are hidden via legend toggle, they are removed from the data. On the next `.update()` call, those series re-enter with enter animations as new marks.
744754

755+
## Canvas mark mode
756+
757+
Scatter charts with thousands of points cost more in DOM elements than they do
758+
in pixels. `mark.render` moves point marks onto a `<canvas>` layered under the
759+
chart SVG, which keeps a 50,000-point scatter interactive and lets a
760+
high-cardinality keyed morph actually tween instead of snapping.
761+
762+
```ts
763+
{
764+
mark: { type: 'point', render: 'canvas' },
765+
data: campuses, // 4,341 rows
766+
encoding: {
767+
x: { field: 'lowIncomePct', type: 'quantitative' },
768+
y: { field: 'readingProficiency', type: 'quantitative' },
769+
key: { field: 'campusId' },
770+
},
771+
}
772+
```
773+
774+
| Value | Behavior |
775+
| ---------- | -------- |
776+
| `'auto'` | Default. Canvas above 1,000 compiled points, SVG at or below. |
777+
| `'canvas'` | Canvas at any point count. |
778+
| `'svg'` | Never canvas. |
779+
780+
`render` applies to point marks only. On any other mark type, and on faceted,
781+
layered or sparkline charts, it falls back to SVG; an explicit `'canvas'` there
782+
warns once per compile.
783+
784+
### What canvas mode changes
785+
786+
The canvas paints the figure background, the gridlines and the dots. The SVG
787+
keeps axes, tick labels, the trendline, annotations, legend and chrome, and
788+
still carries the chart's `role` and `aria-label`.
789+
790+
Everything below is a deliberate trade, not a limitation to work around:
791+
792+
- **The trendline draws above the dots**, even with `trendline.layer: 'below'`.
793+
The canvas is a single layer under the whole SVG, so anything the SVG draws
794+
lands on top.
795+
- **Gradient fills flatten to their first stop on screen.** Exports render the
796+
true gradient, because they materialize a full SVG (see below).
797+
- **Overlapping dots can show a stroke over a neighbour's fill.** The canvas
798+
paints every fill and then every stroke, two batched draw calls rather than
799+
two per point, which is what keeps thousands of dots cheap. SVG paints each
800+
dot's fill and stroke together. Sparse clouds are indistinguishable; tight
801+
clusters show it. Lowering `mark.opacity` for dense scatters — worth doing
802+
regardless, to keep overplotting readable — makes it disappear.
803+
- **Per-mark keyboard focus and edit-mode point selection are unavailable.**
804+
There are no per-point elements to focus. Annotation, chrome and legend
805+
editing still work, and the screen-reader data table is unaffected.
806+
- **The screen-reader table caps at 1,000 rows** with a caption naming the true
807+
total. Full data remains available via CSV export.
808+
809+
Tooltips, `onMarkClick`, `onMarkHover` and hover highlighting all work: pointer
810+
events fall through to the canvas and hit-test against a spatial index.
811+
812+
### Exports
813+
814+
Exports are full-fidelity regardless of what is on screen. Canvas-mode charts
815+
re-render to a complete SVG at export time, so gradients, gridlines and the
816+
background all come back as vector.
817+
818+
| Format | Behavior |
819+
| ------ | -------- |
820+
| `svg`, `svg-with-fonts` | Vector at or below 5,000 points -- byte-identical to exporting the SVG-mode twin. Above that, vector everything with the dot cloud inlined as one raster `<image>`, so gridlines and text stay crisp without shipping 50,000 `<circle>` elements. |
821+
| `png`, `jpg`, `gif` | Always full vector before rasterizing, so output is pixel-identical to SVG mode. |
822+
| `csv` | Unchanged. |
823+
745824
### encoding.key channel
746825

747826
The `key` channel maps a field that uniquely identifies each datum across updates. It is not visually encoded. When omitted, the engine derives keys from the x-axis value (and color field for grouped charts).

e2e/perf/canvas-scatter.spec.ts

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
/**
2+
* Canvas scatter frame-pacing gate.
3+
*
4+
* The whole point of the canvas mark layer is that a high-cardinality update
5+
* stays smooth. That claim is only checkable in a real browser with a real
6+
* compositor, so the `testing--canvas-perf--canvas-update-perf` story mounts a
7+
* 5,000-point canvas scatter, runs one `.update()`, samples inter-frame deltas,
8+
* and writes `{ frames, mean, p95 }` to `#perf-result`.
9+
*
10+
* Thresholds are deliberately CI-generous. This is a "did we regress by an
11+
* order of magnitude" tripwire, not a benchmark: shared CI runners are noisy,
12+
* and a tight bound here would flake more than it would catch. The real
13+
* low-end-device pass stays a manual per-release step.
14+
*
15+
* Run: bunx playwright test --project=perf
16+
*/
17+
18+
import { expect, test } from '@playwright/test';
19+
20+
/** Roughly two dropped frames at 60fps. */
21+
const MEAN_BUDGET_MS = 33;
22+
/** Roughly three. Tail spikes on a shared runner are expected. */
23+
const P95_BUDGET_MS = 50;
24+
25+
test('a 5,000-point canvas update holds its frame budget', async ({ page }) => {
26+
await page.goto('/?mode=preview&story=testing--canvas-perf--canvas-update-perf');
27+
28+
const resultEl = page.locator('#perf-result');
29+
await resultEl.waitFor({ state: 'visible', timeout: 15_000 });
30+
31+
await expect
32+
.poll(async () => (await resultEl.getAttribute('data-result')) ?? 'pending', {
33+
timeout: 20_000,
34+
})
35+
.not.toBe('pending');
36+
37+
const raw = (await resultEl.getAttribute('data-result')) ?? '';
38+
// eslint-disable-next-line no-console
39+
console.log('canvas scatter perf:', raw);
40+
expect(raw, `perf harness failed: ${raw}`).not.toContain('error');
41+
42+
const result = JSON.parse(raw) as { frames: number; mean: number; p95: number };
43+
44+
// A sample too small to mean anything usually means the transition never ran
45+
// (gate vetoed it) -- which would make the timings trivially perfect.
46+
expect(result.frames, 'too few frames sampled to judge pacing').toBeGreaterThan(10);
47+
expect(result.mean).toBeLessThan(MEAN_BUDGET_MS);
48+
expect(result.p95).toBeLessThan(P95_BUDGET_MS);
49+
});

examples/src/gallery/charts-scatter-distribution.demos.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,5 +29,7 @@ export const page: PageEntry = {
2929
{ id: 'arrow-plot', title: 'Arrow plot' },
3030
{ id: 'range-bar', title: 'Range bar' },
3131
{ id: 'interactive', title: 'Interactive (hover to read out)' },
32+
{ id: 'high-cardinality-canvas', title: 'Keyed morph at 3,000 points' },
33+
{ id: 'canvas-svg-parity', title: 'Canvas vs SVG, same data' },
3234
],
3335
};

0 commit comments

Comments
 (0)