Skip to content

Fix Studio DOM picking and timeline thumbnails#529

Merged
miguel-heygen merged 1 commit intonextfrom
fix/studio-alpha-dom-edit-picking
Apr 28, 2026
Merged

Fix Studio DOM picking and timeline thumbnails#529
miguel-heygen merged 1 commit intonextfrom
fix/studio-alpha-dom-edit-picking

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen commented Apr 28, 2026

Problem

Manual DOM editing in Studio had a few rough edges that showed up when testing the latest alpha like a video editor would:

  • Elements with identity transforms such as matrix(1, 0, 0, 1, 0, 0) were treated as transform-driven and could not be moved/resized directly, even when their layout was otherwise absolute px geometry.
  • Clicking inside an already-selected parent layer could keep selecting the parent instead of the exact child under the pointer, which made last-mile visual edits feel imprecise.
  • Timeline thumbnails for DOM clips were generated from the full composition too often, so repeated/anonymous elements were hard to distinguish.
  • Runtime manifest clips with anonymous ids like __node__index_* could lose their connection back to the real DOM node, which also blocked precise selector thumbnails and reliable timeline patch targets.
  • Generic finite-duration DOM clips were more constrained than needed in timeline drag operations.
  • The caption lint rule could false-positive on non-caption code that merely used caption-like names.

What this fixes

  • Treats none, identity matrix(...), and identity matrix3d(...) transforms as directly editable for DOM move/resize capability checks.
  • Lets selected-box clicks re-pick the exact child element under the pointer, while preserving normal drag behavior for real movement.
  • Allows finite-duration patchable DOM clips to move on the timeline, while keeping trim behavior conservative.
  • Carries selector and selectorIndex through Studio timeline thumbnails and the core thumbnail API.
  • Uses querySelectorAll(selector)[selectorIndex] when generating cropped thumbnails in both Vite Studio preview and the CLI Studio server.
  • Reconnects anonymous runtime manifest clips back to their DOM nodes using tag/start/duration/track matching and avoids generic .clip selectors when a more specific class exists.
  • Makes the thumbnail preload eager so hidden preload images actually load before the visible filmstrip is mounted.
  • Narrows the caption lint marker detection so ordinary variables/classes containing caption do not trigger caption-runtime warnings.

Root cause

The DOM editor was using a strict transform === "none" check. GSAP and browsers commonly leave identity transforms as computed matrices, so visually untransformed absolute elements were incorrectly downgraded into the layout-controlled path.

The selected-box picker also mixed two intents on the same pointer path: click-to-repick and drag-to-move. Movable boxes started a gesture on pointerdown, which could suppress the normal click path before the exact child picker ran.

For thumbnails, Studio had the backend cropper but did not consistently pass enough identity information from timeline clips to the thumbnail route. Runtime manifests can emit synthetic ids for anonymous DOM clips, so the Studio side needed a DOM fallback to recover the actual element, selector, and occurrence index.

Verification

Local checks

  • bun run --filter @hyperframes/studio test src/components/editor/domEditing.test.ts src/player/components/timelineEditing.test.ts src/player/components/CompositionThumbnail.test.ts src/player/hooks/useTimelinePlayer.test.ts -> 66 tests pass
  • bun run --filter @hyperframes/core test src/lint/rules/captions.test.ts src/studio-api/routes/thumbnail.test.ts -> 10 tests pass
  • bun run --filter @hyperframes/studio typecheck
  • bun run --filter @hyperframes/core typecheck
  • bun run --filter @hyperframes/cli typecheck
  • bunx oxlint packages/cli/src/server/studioServer.ts packages/core/src/lint/rules/captions.test.ts packages/core/src/lint/rules/captions.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/types.ts packages/studio/src/App.tsx packages/studio/src/components/editor/DomEditOverlay.tsx packages/studio/src/components/editor/domEditing.test.ts packages/studio/src/components/editor/domEditing.ts packages/studio/src/player/components/CompositionThumbnail.tsx packages/studio/src/player/components/CompositionThumbnail.test.ts packages/studio/src/player/components/timelineEditing.test.ts packages/studio/src/player/components/timelineEditing.ts packages/studio/src/player/hooks/useTimelinePlayer.test.ts packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/vite.config.ts -> 0 warnings, 0 errors
  • git diff --check
  • bun run --filter @hyperframes/studio build
  • Lefthook pre-commit -> lint, format, typecheck pass
  • Lefthook commit-msg -> commitlint pass

Browser verification

  • Started Studio preview against a scratch project for DOM/timeline edge cases.
  • Used agent-browser to select an absolute layer with computed matrix(1, 0, 0, 1, 0, 0) and verify normal X/Y/W/H controls were available.
  • Dragged that identity-transform card and verified the preview updated and Studio persisted the file through the project file API.
  • Re-picked a child text element from inside the selected parent box and verified selection changed from #identity-card to #identity-title.
  • Verified repeated DOM timeline thumbnails emitted distinct URLs with selector and selectorIndex, loaded successfully, and returned distinct cropped JPEGs.
  • Dragged a timeline DOM clip and verified data-start / track changes were written back.
  • Added two local sub-compositions (matrix-visual, matrix-code) to the scratch project and verified:
    • hyperframes lint /tmp/hf-dom-edit-fix-verify -> 0 errors, 0 warnings
    • hyperframes compositions /tmp/hf-dom-edit-fix-verify listed main, matrix-visual, and matrix-code
    • root timeline showed both composition clips
    • composition thumbnails loaded from /thumbnail/compositions/...
    • sidebar drill-in to matrix-visual showed its inner clips and selector-specific thumbnails
  • Recorded the tested Studio flow with agent-browser.

Notes

  • The scratch verification project lives at /tmp/hf-dom-edit-fix-verify and is intentionally not committed.
  • Local screenshots/recordings are kept under qa-artifacts/ and are intentionally not committed.
  • While testing sub-compositions, agent-browser dblclick did not reliably trigger timeline composition drill-in; sidebar composition navigation worked and verified the drilled-in composition path.

@miguel-heygen miguel-heygen merged commit 26d7a9a into next Apr 28, 2026
23 checks passed
@miguel-heygen miguel-heygen deleted the fix/studio-alpha-dom-edit-picking branch April 28, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant