fix(vanilla): composite canvas scatter points individually

Canvas-mode scatter batched every point sharing a (fill, alpha) into one
beginPath() and painted the lot with a single fill() at globalAlpha.
Circles overlapping inside one path are unioned by the fill rule, so the
whole cluster was faded ONCE instead of each dot compositing in turn. A
six-deep stack at opacity 0.35 landed on 0.65 luminance instead of 0.12.

Dense translucent scatters therefore rendered visibly washed out, while
their exports looked correct -- exports below VECTOR_EXPORT_MAX_POINTS
re-render through real SVG circles, which never had the bug. Above that
cap the raster-marks export path shares the canvas renderer, so it was
affected too.

Fills, strokes and exit ghosts are now issued per point. This is not a
correctness-for-speed trade: per-circle measured faster at every count
tested in headless Chromium (~1.5x at 4k points, ~3.7x at 50k). One
enormous multi-subpath tessellation costs more than thousands of tiny
independent fills. Redundant fillStyle/strokeStyle/lineWidth/globalAlpha
writes are elided across runs of identically-styled points, and paint
order stays mark order to match SVG.

Regression coverage is an e2e pixel test rather than a unit test: a
recording 2D-context stub cannot see this, since both call sequences are
legal and only the rasterized output differs. The new invariants spec
stacks six translucent dots and asserts the composited luminance matches
the closed-form value, which fails at ~159 with the batching restored.

- packages/vanilla/src/scatter-canvas/renderer.ts: per-point draw calls
- e2e/invariants/canvas-alpha-parity.spec.ts: pixel-level parity gate
- examples/src/testing/canvas-alpha-parity.stories.tsx: its harness
- renderer.test.ts: assert per-point calls + state-write elision
- docs + gallery story: drop the now-false "batching" explanation, keep
  the still-real fill-pass-then-stroke-pass paint-order note

Author: rileyhilliard

docs/spec-reference.md

@@ -795,11 +795,11 @@ Everything below is a deliberate trade, not a limitation to work around:
 - **Gradient fills flatten to their first stop on screen.** Exports render the
   true gradient, because they materialize a full SVG (see below).
 - **Overlapping dots can show a stroke over a neighbour's fill.** The canvas
-  paints every fill and then every stroke, two batched draw calls rather than
-  two per point, which is what keeps thousands of dots cheap. SVG paints each
-  dot's fill and stroke together. Sparse clouds are indistinguishable; tight
-  clusters show it. Lowering `mark.opacity` for dense scatters — worth doing
-  regardless, to keep overplotting readable — makes it disappear.
+  paints all fills in one pass and all strokes in a second, so a later dot's
+  stroke can sit over an earlier dot's fill. SVG paints each dot's fill and
+  stroke together. Sparse clouds are indistinguishable; tight clusters show it.
+  Lowering `mark.opacity` for dense scatters — worth doing regardless, to keep
+  overplotting readable — makes it disappear.
 - **Per-mark keyboard focus and edit-mode point selection are unavailable.**
   There are no per-point elements to focus. Annotation, chrome and legend
   editing still work, and the screen-reader data table is unaffected.

e2e/invariants/canvas-alpha-parity.spec.ts

@@ -0,0 +1,146 @@
+/**
+ * Canvas and SVG must composite translucent overlapping marks identically.
+ *
+ * Red-locked bug: the canvas renderer batched all points sharing a
+ * (fill, alpha) into one `beginPath()` and painted them with a single `fill()`
+ * at `globalAlpha`. Circles overlapping inside one path get unioned by the fill
+ * rule, so the stack was faded ONCE -- a dense cluster came out at the alpha of
+ * a single dot. SVG composites each `<circle>` separately and builds toward
+ * opaque. Dense scatters rendered washed out on screen while their exports
+ * (which re-render through real SVG) looked right.
+ *
+ * This cannot be caught by a stubbed 2D context: both call sequences are legal
+ * and the difference only exists in rasterized output. Hence a real browser.
+ *
+ * The story stacks `STACK_DEPTH` identical points at one coordinate, so the
+ * correct composite is closed-form rather than a screenshot baseline:
+ *   over white, luminance = 255 * (1 - opacity)^depth
+ * With opacity 0.35 and depth 6 that is ~19/255. The batched bug produced
+ * ~166/255 -- a gap no antialiasing tolerance can blur away.
+ */
+
+import { expect, test } from '@playwright/test';
+
+const OPACITY = 0.35;
+const STACK_DEPTH = 6;
+
+/** Ideal src-over result for N stacked black dots at OPACITY on white. */
+const EXPECTED = 255 * (1 - OPACITY) ** STACK_DEPTH;
+
+/**
+ * Generous: covers antialiasing, DPR downsampling and the canvas layer's
+ * 8-bit rounding. Still ~7x tighter than the distance to the buggy value, so
+ * the test discriminates the defect without policing sub-percent drift.
+ */
+const TOLERANCE = 20;
+
+test('canvas composites stacked translucent points like SVG does', async ({ page }) => {
+  await page.goto('/?mode=preview&story=testing--canvas-alpha-parity--stacked-translucent-points');
+
+  // `createChart` promotes the mount node itself to `.oc-root`; the chart is not
+  // nested in a further wrapper.
+  await page.waitForSelector('#parity-svg.oc-root svg.oc-chart');
+  await page.waitForSelector('#parity-canvas canvas.oc-mark-canvas');
+  await page.evaluate(() => document.fonts.ready);
+  // The canvas layer paints on a rAF tick; give it a couple of frames to land.
+  await page.waitForTimeout(200);
+
+  const sample = await page.evaluate(() => {
+    const markCanvas = document.querySelector(
+      '#parity-canvas canvas.oc-mark-canvas',
+    ) as HTMLCanvasElement | null;
+    if (!markCanvas) return { error: 'no mark canvas' };
+
+    const ctx = markCanvas.getContext('2d');
+    if (!ctx) return { error: '2d context unavailable' };
+
+    const { width, height } = markCanvas;
+    const data = ctx.getImageData(0, 0, width, height).data;
+
+    /**
+     * Composite a pixel over white, so the reading is comparable regardless of
+     * whether the layer painted an opaque background or left it transparent.
+     */
+    const lumAt = (i: number): number => {
+      const a = data[i + 3] / 255;
+      return data[i] * a + 255 * (1 - a);
+    };
+
+    // Locate the stack rather than trusting a computed coordinate.
+    //
+    // Deliberate: on the canvas side `.oc-marks` is an EMPTY svg group (the dots
+    // live on the canvas), so its bounding box collapses to zero at the
+    // container origin -- sampling there reads unpainted background and the test
+    // would "fail" on the wrong pixel. The stack is the darkest thing drawn, so
+    // searching for it is both simpler and self-validating.
+    let darkest = Number.POSITIVE_INFINITY;
+    let paintedPixels = 0;
+    for (let i = 0; i < data.length; i += 4) {
+      if (data[i + 3] > 2) paintedPixels++;
+      const lum = lumAt(i);
+      if (lum < darkest) darkest = lum;
+    }
+    if (!Number.isFinite(darkest)) return { error: 'canvas is empty' };
+
+    // Average the INTERIOR of the stack, not a box centred on the darkest pixel.
+    //
+    // The darkest pixel is frequently on the dot's antialiased rim, so a box
+    // around it straddles the edge and averages dot with background -- that read
+    // lands ~100 even on a correct render and would make this test lie in both
+    // directions. Instead: take every pixel at or very near the darkest value.
+    // On a correct render that is the flat interior of the stack; if compositing
+    // regresses, the whole interior shifts together and the mean moves with it.
+    const NEAR = 4;
+    let sum = 0;
+    let count = 0;
+    for (let i = 0; i < data.length; i += 4) {
+      const lum = lumAt(i);
+      if (lum <= darkest + NEAR) {
+        sum += lum;
+        count++;
+      }
+    }
+
+    return {
+      canvasLuminance: sum / count,
+      darkest,
+      interiorPixels: count,
+      paintedPixels,
+      // If the SVG side stopped drawing dots the comparison would be vacuous.
+      svgCircleCount: document.querySelectorAll('#parity-svg circle.oc-mark-point').length,
+      canvasCircleCount: document.querySelectorAll('#parity-canvas circle.oc-mark-point').length,
+    };
+  });
+
+  if ('error' in sample) throw new Error(`sampling failed: ${sample.error}`);
+
+  // Guard the premise: SVG really did draw the stack, canvas really did not,
+  // and the canvas actually painted something for us to measure.
+  expect(sample.svgCircleCount, 'SVG chart should render real circles').toBe(STACK_DEPTH);
+  expect(sample.canvasCircleCount, 'canvas chart should have no SVG circles').toBe(0);
+  expect(sample.paintedPixels, 'canvas painted nothing to sample').toBeGreaterThan(100);
+
+  // eslint-disable-next-line no-console
+  console.log(
+    `alpha parity: interior=${sample.canvasLuminance.toFixed(1)} darkest=${sample.darkest.toFixed(1)} ` +
+      `expected=${EXPECTED.toFixed(1)} (interiorPx=${sample.interiorPixels})`,
+  );
+
+  // The stack must cover a real area. A single dot at r=14 is ~600px; six
+  // coincident dots stay ~600. If this collapses, the renderer stopped drawing
+  // the cluster and the luminance readings below would be measuring noise.
+  expect(sample.interiorPixels, 'stack interior too small to trust').toBeGreaterThan(200);
+
+  const message =
+    `canvas luminance ${sample.canvasLuminance.toFixed(1)} should be within ${TOLERANCE} of ` +
+    `${EXPECTED.toFixed(1)}. A value near 166 means overlapping points were batched into one ` +
+    `path and faded once instead of compositing individually.`;
+
+  // Assert on BOTH the interior mean and the darkest pixel. The mean alone is
+  // weak by construction -- it averages pixels selected for being near the
+  // darkest, so it tracks `darkest` and would look tight even if the whole
+  // stack lightened. Pinning `darkest` to the expected composite is what
+  // actually catches a compositing regression.
+  expect(Math.abs(sample.darkest - EXPECTED), message).toBeLessThan(TOLERANCE);
+  expect(Math.abs(sample.canvasLuminance - EXPECTED), message).toBeLessThan(TOLERANCE);
+});

examples/src/gallery/charts-scatter-distribution.stories.tsx

@@ -709,13 +709,16 @@ function CanvasMorphScatter() {
  * Small enough that auto leaves it on SVG, so `render: 'canvas'` is explicit.
  * This is the visual-parity check: same layout, same colors, same geometry.
  *
- * Two differences are expected and permanent, both consequences of batching:
- * the trendline always draws above the dots, and where dots overlap, a stroke
- * can land on a neighbour's fill. Canvas paints every fill and then every
- * stroke (two draw calls total, which is what keeps thousands of points
- * cheap), whereas SVG paints each dot's fill and stroke together. Sparse
+ * Two differences are expected and permanent: the trendline always draws above
+ * the dots, and where dots overlap, a stroke can land on a neighbour's fill.
+ * The latter is a paint-order effect -- canvas fills every dot, then strokes
+ * every dot, whereas SVG paints each dot's fill and stroke together. Sparse
  * clouds are indistinguishable; tight clusters show it.
  *
+ * Translucent overlaps do NOT differ: each dot composites individually on
+ * canvas exactly as it does in SVG. That parity is pinned by
+ * `e2e/invariants/canvas-alpha-parity.spec.ts`.
+ *
  * `animation: false` because a baseline screenshot cannot freeze the canvas
  * entrance.
  */
@@ -1092,7 +1095,7 @@ export const ScatterAndDistribution = () => (
       <Demo
         id="canvas-svg-parity"
         title="Canvas vs SVG, same data"
-        description="An explicit render: 'canvas' on a 200-point cloud that auto would have left on SVG. Same layout, same geometry. Two differences come from batching, which is what keeps thousands of points cheap: the trendline always draws above the dots, and canvas paints every fill before every stroke, so overlapping dots can show a stroke over a neighbour's fill. Sparse clouds look identical; tight clusters show it."
+        description="An explicit render: 'canvas' on a 200-point cloud that auto would have left on SVG. Same layout, same geometry. Two differences remain: the trendline always draws above the dots, and canvas fills every dot before stroking any, so overlapping dots can show a stroke over a neighbour's fill. Sparse clouds look identical; tight clusters show it."
         specForPanel={canvasParitySpec}
         height={420}
       >

examples/src/testing/canvas-alpha-parity.stories.tsx

@@ -0,0 +1,94 @@
+/**
+ * Testing / Canvas-vs-SVG alpha compositing parity harness.
+ *
+ * Not an editorial demo -- this story exists so the Playwright `invariants`
+ * project can compare what the two mark renderers actually put on screen.
+ *
+ * The bug this guards against: the canvas renderer used to batch every point
+ * sharing a (fill, alpha) into ONE `beginPath()` and pay for it with ONE
+ * `fill()` at `globalAlpha`. Overlapping circles inside a single path are
+ * unioned by the fill rule and then faded once, so a dense cluster of
+ * translucent dots came out at the alpha of a SINGLE dot. SVG composites every
+ * `<circle>` separately, so the same cluster builds toward opaque. A 5-deep
+ * stack at opacity 0.35 landed on 0.65 luminance instead of 0.12 -- dense
+ * scatters rendered visibly washed out, while exports (which re-render through
+ * real SVG) looked correct.
+ *
+ * A stubbed 2D context cannot catch this: the call sequence is legal either
+ * way and the defect only exists in the rasterizer's output. So both charts
+ * mount here, in a real browser, over the SAME data, and the spec reads pixels
+ * out of each.
+ *
+ * Both charts are deliberately given heavily overlapping points -- a tight
+ * cluster is where the two compositing models diverge most. Sparse clouds
+ * agree even with the bug present, which is exactly why this shipped.
+ */
+
+import type { ChartSpec } from '@opendata-ai/openchart-core';
+import { createChart } from '@opendata-ai/openchart-vanilla';
+import { useEffect, useRef } from 'react';
+
+export default { title: 'Testing / Canvas Alpha Parity' };
+
+/** Fill + opacity are fixed so the expected composite is computable by hand. */
+const FILL = '#000000';
+const OPACITY = 0.35;
+/** Deep enough that per-circle compositing is unmistakably darker than one fill. */
+const STACK_DEPTH = 6;
+
+const WIDTH = 420;
+const HEIGHT = 320;
+
+/**
+ * Every point sits at the same data coordinate, so all `STACK_DEPTH` circles
+ * land on the same pixel. That makes the assertion a closed-form value rather
+ * than a screenshot: correct output is `1 - (1 - 0.35)^6` over white.
+ */
+function stackedSpec(render: 'svg' | 'canvas'): ChartSpec {
+  return {
+    // A settled chart, not a mid-fade one -- entrance alpha would poison the read.
+    animation: false,
+    width: WIDTH,
+    height: HEIGHT,
+    mark: {
+      type: 'point',
+      render,
+      size: 14,
+      opacity: OPACITY,
+      fill: FILL,
+      // No separator stroke: a stroke over the sample point would be measuring
+      // the stroke pass, not the fill compositing under test.
+      strokeWidth: 0,
+    },
+    data: Array.from({ length: STACK_DEPTH }, (_, i) => ({ id: `p${i}`, x: 50, y: 50 })),
+    encoding: {
+      x: { field: 'x', type: 'quantitative', scale: { domain: [0, 100] } },
+      y: { field: 'y', type: 'quantitative', scale: { domain: [0, 100] } },
+      key: { field: 'id', type: 'nominal' },
+    },
+  };
+}
+
+function Chart({ render, id }: { render: 'svg' | 'canvas'; id: string }) {
+  const ref = useRef<HTMLDivElement>(null);
+
+  useEffect(() => {
+    if (!ref.current) return;
+    const chart = createChart(ref.current, stackedSpec(render));
+    return () => chart.destroy();
+  }, [render]);
+
+  return <div id={id} ref={ref} style={{ width: WIDTH, height: HEIGHT }} />;
+}
+
+/**
+ * Two charts over identical data and identical opacity, differing only in
+ * which renderer draws the dots. The spec samples the centre pixel of each.
+ */
+export const StackedTranslucentPoints = () => (
+  <div style={{ padding: 16, background: '#ffffff' }}>
+    <div id="alpha-parity-ready" data-ready="true" />
+    <Chart render="svg" id="parity-svg" />
+    <Chart render="canvas" id="parity-canvas" />
+  </div>
+);

packages/vanilla/src/scatter-canvas/__tests__/renderer.test.ts

@@ -90,8 +90,30 @@ describe('ScatterCanvasRenderer DPR handling', () => {
   });
 });
 
-describe('ScatterCanvasRenderer batching', () => {
-  it('draws N same-fill points in a single fill() call', () => {
+/**
+ * `fillStyle` values assigned for point fills, i.e. everything after the
+ * full-bleed background write that `render()` performs first.
+ */
+function pointFillStyles(s: CanvasStub): unknown[] {
+  const all = s.sets.filter((set) => set.prop === 'fillStyle').map((set) => set.value);
+  // Assert rather than blind-slice: if the background write ever stops being
+  // first, dropping [0] would silently swallow a real point fill and the
+  // color-elision assertions would start passing for the wrong reason.
+  expect(all[0], 'expected the full-bleed background fill first').toBe('#ffffff');
+  return all.slice(1);
+}
+
+describe('ScatterCanvasRenderer per-point compositing', () => {
+  /**
+   * The renderer deliberately does NOT merge points into shared paths.
+   *
+   * Merging unions overlapping circles inside one path, so a single `fill()` at
+   * `globalAlpha` fades the whole cluster once instead of letting each dot
+   * composite -- dense translucent scatters rendered washed out. These tests
+   * pin the call shape; `e2e/invariants/canvas-alpha-parity.spec.ts` pins the
+   * resulting pixels, which is the part a recording stub cannot see.
+   */
+  it('issues one fill() per point rather than one per color bucket', () => {
     stub = stubCanvas2D();
     const renderer = new ScatterCanvasRenderer(document.createElement('canvas'));
     renderer.resize(400, 300);
@@ -103,11 +125,15 @@ describe('ScatterCanvasRenderer batching', () => {
     }));
     renderer.render(makeState({ marks: soa(points) }));
 
-    expect(stub.callsTo('fill')).toHaveLength(1);
+    // 50 points => 50 fills and 50 arcs. A single fill() here would mean the
+    // path batching regressed and translucent overlaps stopped accumulating.
+    expect(stub.callsTo('fill')).toHaveLength(50);
     expect(stub.callsTo('arc')).toHaveLength(50);
+    // 50 point paths + the one `render()` opens for the clip rect.
+    expect(stub.callsTo('beginPath')).toHaveLength(51);
   });
 
-  it('splits fill buckets by color', () => {
+  it('skips redundant fillStyle writes across a same-color run', () => {
     stub = stubCanvas2D();
     const renderer = new ScatterCanvasRenderer(document.createElement('canvas'));
     renderer.resize(400, 300);
@@ -120,10 +146,34 @@ describe('ScatterCanvasRenderer batching', () => {
         ]),
       }),
     );
-    expect(stub.callsTo('fill')).toHaveLength(2);
+
+    // Every point still gets its own fill()...
+    expect(stub.callsTo('fill')).toHaveLength(3);
+    // ...but the three colors here alternate, so all three assignments stand.
+    // Drop the leading background write (makeState paints '#ffffff' full-bleed).
+    expect(pointFillStyles(stub)).toEqual(['#ff0000', '#00ff00', '#ff0000']);
+  });
+
+  it('collapses fillStyle writes when consecutive points share a color', () => {
+    stub = stubCanvas2D();
+    const renderer = new ScatterCanvasRenderer(document.createElement('canvas'));
+    renderer.resize(400, 300);
+    renderer.render(
+      makeState({
+        marks: soa([
+          { x: 50, y: 100, r: 3, fill: '#ff0000' },
+          { x: 60, y: 100, r: 3, fill: '#ff0000' },
+          { x: 70, y: 100, r: 3, fill: '#00ff00' },
+        ]),
+      }),
+    );
+
+    expect(stub.callsTo('fill')).toHaveLength(3);
+    // The second red point reuses the style already set by the first.
+    expect(pointFillStyles(stub)).toEqual(['#ff0000', '#00ff00']);
   });
 
-  it('batches strokes by (stroke, width) and skips zero-width strokes', () => {
+  it('strokes each point individually and skips zero-width strokes', () => {
     stub = stubCanvas2D();
     const renderer = new ScatterCanvasRenderer(document.createElement('canvas'));
     renderer.resize(400, 300);
@@ -137,8 +187,13 @@ describe('ScatterCanvasRenderer batching', () => {
         ]),
       }),
     );
-    // Two stroke buckets; no gridlines, so every stroke() is a point stroke.
-    expect(stub.callsTo('stroke')).toHaveLength(2);
+
+    // Three stroked points => three stroke() calls; the zero-width one is
+    // skipped entirely. No gridlines here, so every stroke() is a point stroke.
+    expect(stub.callsTo('stroke')).toHaveLength(3);
+    // lineWidth changes only where the width actually changes (1 -> 2).
+    const widths = stub.sets.filter((s) => s.prop === 'lineWidth').map((s) => s.value);
+    expect(widths).toEqual([1, 2]);
   });
 
   it('culls points outside the clip rect', () => {