Skip to content

FE-1552: Reuse sweep compute, stream the selection first, and pipeline the ladder - #9474

Merged
kube merged 17 commits into
claude/fe-1552-sweep-gpu-streamingfrom
claude/fe-1552-sweep-view-fixes
Sep 3, 2026
Merged

FE-1552: Reuse sweep compute, stream the selection first, and pipeline the ladder#9474
kube merged 17 commits into
claude/fe-1552-sweep-gpu-streamingfrom
claude/fe-1552-sweep-view-fixes

Conversation

@kube

@kube kube commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Important

Experimental
Behind the Parameter sweeps and WebGPU feature flags.

Summary

Before this PR, opening the sweep drawer and moving its sliders was slow because compute was spawned, serialised and thrown away. 166 web workers filled one view, one experiment ran per surface cell, ladder rungs ran one at a time, and every batch created a fresh GPU device, shader and calibration. Terminating nets plotted an empty surface.

This PR reuses that compute with pooled workers, batched surface cells and one GPU backend per session. It streams the selected point before anything else, fills the surface in quad-tree levels and pipelines the ladder. Results are unchanged: same seeds, same trajectories.

Measured on the Bench / SweepDrawer story, base against this PR:

Case Before After
Workers to fill the view 166 20
Workers per slider commit 16–24 1
Navigator reaches 1,000 runs after page load 1,488 ms 1,312 ms (first frames stream at 927 ms)
11×11 surface complete ~15 s sequential, or empty on terminating nets 1.6 s, filling 4 → 25 → 81 → 121 cells
GPU 1,000-run ladder 1,512–1,582 ms 937 ms
GPU command encoders per commit 75–79 68–79

Links

Changes

Sweep view

  • Terminating nets no longer plot an empty surface

    sweepCellObjective reads a cell's value from its last frame that holds samples.

  • Chart data applies once per animation frame

    Latest wins, with the crossfade's epoch change latched until a frame paints.

  • Chart axes mount before any data
  • Switching the shown metric repaints without simulating

    Surface caches every metric's value per cell.

  • Adds the Bench / SweepDrawer Storybook fixture

    Numbers above are measured on it.

Compute reuse

  • createReusableWorkerFactory in core hands out leases over an idle pool

    A lease returns its worker only once every cancelled acknowledgement it owes has arrived, so no stale reply reaches the next lease.
    drain() flushes and dispose() shuts down.

  • Experiments provider routes every worker-creation site through one pool per mount
  • sampleCellsBatch turns many surface cells into one experiment

    Each cell's parameter values are laid out as per-run draws with pinned per-run seeds, so a cell's value matches the navigator's own runs whatever chunk sampled it. Cells whose initial marking the swept parameters shape keep the per-cell path.

  • One GPU backend is cached per experiment session and leased across batches

    Device, compiled shader and learned calibration are reused.
    Cache is keyed by what shapes the shader; per-run-buffered parameter values are excluded so range selections share it.
    Calibration is also keyed per marking.

  • Streamed GPU runs open with a 128-run tile

    It precedes the full-size tiles so a usable picture lands early.

Scheduling

  • whenSelectionStreamed() gates surface work behind the selection's first frames

    It re-arms on every slider move.

  • quadTreeLevels fills the surface corners first

    Each level then splits every region in two per axis: 4, 5, 16, 56, 40 cells on an 11×11 grid, with three chunks in flight.

  • Next ladder rung starts as soon as the current one streams its first frames

    At most two are live; folds stay ordered.

  • Surface chunks take one worker while the ladder computes on the CPU

    They shard across the freed pool once it idles or runs on the GPU. Shards stream partial cells as they complete.

Known issues

  • On the GPU backend the pipelined ladder holds two GPU experiments concurrently for the overlap window, doubling buffer residency then. Bounded and not observed to fail
  • Preview tile is invisible on light models, where every tile finishes inside one 100 ms publish window. It pays on heavy models

Test coverage

  • New reusable-worker-factory.test.ts:

    Reuse, listener detachment, the owed-acks gate, quarantine of a silent worker, maxIdle, drain and dispose semantics.

  • gpu-backend-cache.test.ts:

    New suite.

  • runner.test.ts, metric-windows.test.ts:

    planTiles and calibrationKey.

  • sweep-session.test.ts:

    sampleCells draw layout and pinned seeds, whenSelectionStreamed, foregroundActive, pipelining order and aborts, partial streaming.

  • contour-grid.test.ts:

    Objective scan-back and quadTreeLevels.

  • provider.test.tsx:

    One worker serves two consecutive experiments and is terminated on unmount.

How to test

  • Open Petrinaut preview on Vercel
  • Viewport controls > Settings > Simulation > Parameter sweeps, WebGPU
  • Load example > Café Queue
  • Simulate > Experiments > Create
  • Sweep two numeric scenario parameters, Run
  • Expect parameter strip, surface container and metric boxes with axes painted before any data
  • Expect charts streaming first, then the surface filling corners-first and sharpening level by level
  • Drag and release a parameter slider
  • Expect charts restreaming first, surface resuming, run counter climbing smoothly through the ladder
  • Switch the surface metric
  • Expect plot repainting from cache
  • In a browser with WebGPU, create the experiment again with Backend > GPU
  • Expect 1,000-run ladder completing in about a second and slider commits streaming without a per-batch setup pause

Demo

Screenshots pending (will be handed over for drag-drop).

@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 3, 2026 6:36pm UTC
petrinaut Ready Ready Preview Sep 3, 2026 6:36pm UTC
petrinaut-docs Ready Ready Preview Sep 3, 2026 6:36pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 3, 2026 6:36pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/apps > hash.design Affects the `hash.design` design site (app) labels Aug 31, 2026
@kube kube self-assigned this Aug 31, 2026
@kube kube changed the title FE-1552: Fix empty sweep surfaces and paint the drawer layout before its data FE-1552: Reuse sweep compute, stream the selection first, and pipeline the ladder Sep 1, 2026
@kube
kube force-pushed the claude/fe-1552-sweep-view-fixes branch from d1a91f3 to 28a8834 Compare September 1, 2026 23:17
@kube
kube force-pushed the claude/fe-1552-sweep-view-fixes branch from 28a8834 to efbf71f Compare September 2, 2026 16:12
@kube
kube marked this pull request as ready for review September 2, 2026 18:32
Copilot AI balanced review requested due to automatic review settings September 2, 2026 18:32
@cursor

cursor Bot commented Sep 2, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches worker pooling races, concurrent GPU batch leases, and sweep scheduling across CPU/GPU backends; behavior is heavily tested but regressions could affect experiment lifecycle or sweep results timing.

Overview
Sweep experiments stop tearing down compute on every batch. CPU workers are leased from a reusable pool (reset via cancel with strict ack ordering), the WebGPU path shares one cached backend and calibration per session, and streamed GPU runs can open with a small preview tile before full tiles.

Scheduling changes how work is ordered: the navigator’s selection must stream before surface sampling (whenSelectionStreamed), the refine ladder pipelines up to two rungs once the current rung has frames, and the surface fills in quad-tree levels via batched sampleSurfaceCells (many cells per experiment when marking is shared). Background CPU batches narrow while the ladder holds the pool and widen when it idles or the GPU is in use.

UI correctness/perf: surface cells read metrics from the last frame with samples (fixing terminating nets), metric charts apply plot data once per animation frame, and axes can mount before data arrives; switching surface metrics repaints from cached per-cell values.

Reviewed by Cursor Bugbot for commit 777f383. Bugbot is set up for automated code reviews on this repo. Configure here.

TimDiekmann
TimDiekmann previously approved these changes Sep 3, 2026
@kube
kube added this pull request to the merge queue Sep 3, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Sep 3, 2026
@kube
kube dismissed TimDiekmann’s stale review September 3, 2026 16:17

The merge-base changed after approval.

TimDiekmann
TimDiekmann previously approved these changes Sep 3, 2026
@hash-release hash-release Bot mentioned this pull request Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash.design Affects the `hash.design` design site (app) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

5 participants