` and animate the wrapper, not the video.**
+
+The framework forces `opacity: 1` on any element with `data-start`/`data-duration` while it's "active" — that's how it controls clip visibility. CSS `opacity: 0` on the video element is silently overwritten by the framework's clip lifecycle, so an opacity tween on the video element won't do anything. Wrap the video in a `
` that has no `data-*` attributes; the wrapper is owned entirely by your CSS/GSAP.
+
+**2. Both videos start at `data-start="0"` and decode in sync from t=0.**
+
+It's tempting to "late-mount" the cutout (`data-start="3.3"` to match the cut). Don't — Chrome does a seek + decoder warm-up at mount, which can land one frame off the base mp4 at the cut moment. With both videos mounted from t=0 and the cutout's wrapper opacity-animated, both decoders advance the same way and stay frame-accurate.
+
+### Quality preset and color match
+
+When the cutout is overlaid on its own source mp4, the encoder's CRF directly affects how visible the doubling is at edges:
+
+| `--quality` | CRF | File size (12s @ 1080p) | When to use |
+|---|---|---|---|
+| `fast` | 30 | ~2 MB | Cutout sits over an unrelated background and file size matters |
+| `balanced` *(default)* | 18 | ~6 MB | Recommended for text-behind-subject and any pattern that overlays on the source |
+| `best` | 12 | ~12 MB | Hero shots, masters, or anything you'll re-encode downstream |
+
+The encoder also writes BT.709 + limited-range color metadata so Chrome's YUV→RGB pipeline matches the source mp4's. Without those tags, the cutout would render slightly differently from the underlying mp4 even at lossless quality (visible red/skin shift).
+
+## What u²-net_human_seg is and isn't good for
+
+The model is purpose-built for **portrait / human matting**. It excels when:
+
+- ✅ The subject is a person, head-and-shoulders or full-body
+- ✅ The framing is reasonably stable (not a wide handheld shot)
+- ✅ The background contrasts with the subject
+
+It struggles or fails on:
+
+- ❌ Non-human subjects (products, animals, objects). The model will return a mostly-empty mask.
+- ❌ Very fine hair detail on a busy background. The 320×320 inference resolution means hair tips get softened — fine for most use cases, but compositors notice.
+- ❌ Frame-to-frame temporal consistency. Each frame is processed independently, so static backgrounds with moving subjects can show subtle edge flicker. For most web playback this is invisible; for high-end VFX it may matter.
+- ❌ Live streams or real-time capture. The pipeline is batch-only.
+
+If your use case hits one of these, see the alternatives below.
+
+## Alternatives — when the built-in command isn't the right tool
+
+The CLI ships **one model on purpose** — the one that's MIT-licensed, runs everywhere, and produces production-quality output for person/portrait video. The list below leads with **free, open-source tools** that pair naturally with HyperFrames. Each entry calls out the actual catch — license, install effort, hardware needs — so you can pick the right one for your situation. Full benchmarks are in the [matting eval](https://www.heygenverse.com/a/0dd5a431-1832-4858-862d-de7fb7d02654).
+
+### Free, open-source CLIs and libraries
+
+These all run locally with no account, no upload, no watermark.
+
+| Tool | When to use it | Catch |
+|------|----------------|-------|
+| [`rembg`](https://github.com/danielgatis/rembg) (Python, MIT) | You need a different subject type — `isnet-general-use` for objects/animals/products, `birefnet-portrait` for a quality ceiling on hair, `silueta` for a tiny ~40 MB footprint. Same family as our default model, more variety. | Requires Python + `pip install rembg`. Some bundled models (`birefnet-*`) need ~4 GB RAM and are CPU-only |
+| [BiRefNet](https://github.com/ZhengPeng7/BiRefNet) (PyTorch, MIT) | Highest-fidelity portrait mattes available — visibly better hair edges than u²-net | Heavy (~4 GB inference RAM), slow on CPU, broken on Apple CoreML at the time of the eval |
+| [Robust Video Matting (RVM)](https://github.com/PeterL1n/RobustVideoMatting) (PyTorch, **GPL-3.0**) | The only widely-available model with **temporal consistency** built in — no edge flicker on moving subjects. Best choice when you're matting a long talking-head clip and frame-to-frame stability matters | GPL-3.0 license is incompatible with most commercial / proprietary codebases. Read your repo's license before using |
+| [Backgroundremover](https://github.com/nadermx/backgroundremover) (Python, MIT) | Simple `pip install` wrapper around u²-net; nice if you want a Python API instead of our Node CLI | Same model family as ours, no quality difference — pick whichever fits your stack |
+| [ComfyUI](https://github.com/comfyanonymous/ComfyUI) (open-source, GPL-3.0 core) | Custom workflows: chain a segmentation model + alpha refinement + temporal smoothing. The right tool for tricky cases (multiple subjects, hair against a similar background, sports footage) | Setup is involved (Python, models, node graph). Worth it for repeat specialty work |
+
+After running any of these externally, encode the output as a HyperFrames-compatible transparent WebM with:
+
+```bash Terminal
+ffmpeg -i frames-%04d.png -c:v libvpx-vp9 \
+ -pix_fmt yuva420p \
+ -metadata:s:v:0 alpha_mode=1 \
+ -auto-alt-ref 0 -b:v 0 -crf 30 \
+ transparent.webm
+```
+
+### Free desktop / GUI tools
+
+| Tool | When to use it | Catch |
+|------|----------------|-------|
+| [DaVinci Resolve — Magic Mask](https://www.blackmagicdesign.com/products/davinciresolve) | You're already editing in Resolve, want a brush-based UI with manual refinement, and need to round-trip the alpha into a larger edit | macOS / Windows / Linux desktop install. The free tier covers Magic Mask; paid Studio version unlocks higher resolutions on some features |
+| [Backgroundremover.app](https://backgroundremover.app) (web) | One-off image cutout, no signup, no watermark | Single images only, not video. Free tier is hosted but the underlying tool is the same `rembg` model family |
+| [PhotoRoom Background Remover](https://www.photoroom.com/tools/background-remover) (web) | Quick one-off image, polished UI, no signup | Single images only, e-commerce-tuned model |
+
+### Web SaaS tools (free tiers, with strings)
+
+| Tool | When to use it | Catch |
+|------|----------------|-------|
+| [unscreen.com](https://www.unscreen.com) | Quick one-off video, no install, drag-and-drop | **Free tier is watermarked and capped at short clips** (~10s preview). Paid removes both. Run by the team behind remove.bg |
+| [RunwayML — Green Screen](https://runwayml.com) | Polished UI with brush refinement and time-aware tracking; the closest a SaaS gets to professional roto | Free tier exists but is credit-limited; serious use is a subscription |
+| [Kapwing — Background Remover](https://www.kapwing.com/tools/remove-video-background) | Browser-based, integrates with their video editor | Free tier is watermarked; paid removes it |
+
+### How to choose
+
+- **Person / portrait video, web playback, MIT-clean** → use the built-in `hyperframes remove-background` (this is what it's tuned for).
+- **Non-human subject** (product, animal, object) → `rembg` with `isnet-general-use`.
+- **Maximum portrait quality, especially hair** → `BiRefNet` via Python.
+- **Long video where edge flicker would be visible**, GPL is OK → `RVM`.
+- **One-off marketing clip, no install** → DaVinci Resolve (free) for video, Backgroundremover.app for a still image.
+- **Specialty case the off-the-shelf models can't handle** → ComfyUI with a custom graph.
+
+## Troubleshooting
+
+### Model download fails or hangs
+
+The weights live on GitHub Releases (rembg's `v0.0.0` release, ~168 MB). If your network blocks GitHub or the download is interrupted:
+
+```bash Terminal
+# Manually download and drop into the cache
+mkdir -p ~/.cache/hyperframes/background-removal/models
+curl -L -o ~/.cache/hyperframes/background-removal/models/u2net_human_seg.onnx \
+ https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net_human_seg.onnx
+```
+
+Subsequent `remove-background` runs skip the download and use your local copy.
+
+### "ffmpeg and ffprobe are required"
+
+The pipeline shells out to ffmpeg for decode + encode. Install via `brew install ffmpeg` on macOS or `sudo apt install ffmpeg` on Debian/Ubuntu. Verify with `npx hyperframes doctor`.
+
+### The output WebM looks fully opaque in the browser
+
+Chrome only reads the alpha plane when the WebM is encoded as `yuva420p` with the `alpha_mode=1` metadata tag. The CLI sets both. If you re-encode the output yourself (e.g. with another ffmpeg invocation), preserve those flags:
+
+```bash Terminal
+ffmpeg -i in.webm -c:v libvpx-vp9 \
+ -pix_fmt yuva420p \
+ -metadata:s:v:0 alpha_mode=1 \
+ -auto-alt-ref 0 \
+ out.webm
+```
+
+To verify a WebM has alpha, extract the first frame and inspect:
+
+```bash Terminal
+ffmpeg -y -c:v libvpx-vp9 -i out.webm -frames:v 1 -pix_fmt rgba -update 1 frame0.png
+```
+
+The decoded `frame0.png` should be RGBA and have non-trivial alpha values.
+
+### CoreML is "available" but inference fails to start
+
+The pipeline auto-falls-back to CPU if CoreML fails to bind, with a warning. If you want to skip the CoreML attempt entirely, force CPU:
+
+```bash Terminal
+npx hyperframes remove-background subject.mp4 -o transparent.webm --device cpu
+```
+
+### The alpha mask has rough or jagged edges
+
+That usually means the source frame is high-contrast against a similar-toned background and the model's 320×320 inference resolution is showing through. Two paths forward:
+
+1. Re-frame or re-shoot to give the subject a more contrasting background.
+2. Try `birefnet-portrait` via `rembg` (see [Other open-source models](#other-open-source-models)) — it's higher quality at hair edges but slower and heavier.
+
+## Reference
+
+- CLI: [`hyperframes remove-background`](/packages/cli#remove-background)
+- Eval: [Matting eval — v7](https://www.heygenverse.com/a/0dd5a431-1832-4858-862d-de7fb7d02654)
+- Source model: [danielgatis/rembg](https://github.com/danielgatis/rembg)
+- ONNX runtime: [`onnxruntime-node`](https://www.npmjs.com/package/onnxruntime-node)
diff --git a/docs/guides/rendering.mdx b/docs/guides/rendering.mdx
index aba45fe16..8de19627c 100644
--- a/docs/guides/rendering.mdx
+++ b/docs/guides/rendering.mdx
@@ -65,7 +65,8 @@ Render your Hyperframes [compositions](/concepts/compositions) to MP4, MOV, or W
**Pros:**
- Fast startup, no container overhead
- - Uses your system GPU for hardware-accelerated encoding (with `--gpu`)
+ - Can use your system GPU for Chrome/WebGL capture by default
+ - Can use your system GPU for hardware-accelerated encoding (with `--gpu`)
- Best for iterative development
**Cons:**
@@ -90,7 +91,8 @@ Render your Hyperframes [compositions](/concepts/compositions) to MP4, MOV, or W
**Cons:**
- Slower startup due to container initialization
- - No GPU acceleration inside the container
+ - Browser capture stays on the deterministic software-GL path
+ - GPU encoding requires Docker host GPU passthrough and is not cross-platform on Docker Desktop
Docker mode uses `chrome-headless-shell` with [BeginFrame](/concepts/determinism#how-it-works) control for frame-perfect, deterministic capture.
@@ -121,8 +123,10 @@ Render your Hyperframes [compositions](/concepts/compositions) to MP4, MOV, or W
| `--video-bitrate` | e.g. `10M`, `5000k` | — | Target bitrate encoding. Cannot combine with `--crf` |
| `--workers` | 1-8 or `auto` | auto | Parallel render workers (see [Workers](#workers) below) |
| `--max-concurrent-renders` | 1-10 | 2 | Max simultaneous renders via the producer server (see [Concurrent Renders](#concurrent-renders) below) |
-| `--gpu` | — | off | GPU encoding (NVENC, VideoToolbox, VAAPI) |
-| `--hdr` | — | off | Detect HDR sources and output HDR10 (MP4 only). See [HDR Rendering](/guides/hdr) |
+| `--gpu` | — | off | GPU encoding (NVENC, VideoToolbox, VAAPI, QSV) |
+| `--browser-gpu` / `--no-browser-gpu` | — | on locally, off in Docker | Use or opt out of host GPU acceleration for local Chrome/WebGL capture |
+| `--hdr` | — | off | Force HDR output even if no HDR sources are detected (MP4 only). See [HDR Rendering](/guides/hdr) |
+| `--sdr` | — | off | Force SDR output even if HDR sources are detected |
| `--docker` | — | off | Use Docker for [deterministic rendering](/concepts/determinism) |
| `--quiet` | — | off | Suppress verbose output |
@@ -148,6 +152,26 @@ npx hyperframes render --video-bitrate 10M --output controlled.mp4
**Tip**: The default `standard` preset (CRF 18) is visually lossless at 1080p — most people cannot distinguish it from the source. Use `--quality draft` for faster iteration, or `--quality high` / `--crf 10` when file size is no concern.
+## GPU Acceleration
+
+Hyperframes has two separate GPU acceleration surfaces:
+
+- `--gpu` uses a hardware video encoder in FFmpeg when one is available. Supported backends include VideoToolbox on macOS, NVENC on NVIDIA systems, VAAPI on Linux, and Intel QSV on supported Windows/Linux hosts.
+- Browser GPU uses the host GPU for local Chrome/WebGL capture. It is enabled automatically for local renders and disabled in Docker. Use `--no-browser-gpu` to opt out.
+
+```bash Terminal
+# Add hardware FFmpeg encoding to the default local browser-GPU render
+npx hyperframes render --gpu --output encoded-fast.mp4
+
+# Opt out of hardware Chrome/WebGL capture
+npx hyperframes render --no-browser-gpu --output software-browser.mp4
+
+# Use browser GPU plus hardware FFmpeg encoding
+npx hyperframes render --gpu --output gpu.mp4
+```
+
+Browser GPU capture is local-mode only. It maps to platform-native Chrome GPU backends: Metal on macOS, D3D11 on Windows, and EGL on Linux. Use `--no-browser-gpu` or Docker mode when exact cross-machine reproducibility matters more than local render speed.
+
## Workers
Each render worker launches a **separate Chrome browser process** to capture frames in parallel. More workers can speed up rendering, but each one consumes ~256 MB of RAM and significant CPU.
diff --git a/docs/guides/troubleshooting.mdx b/docs/guides/troubleshooting.mdx
index cceb52c4c..aa9007bb7 100644
--- a/docs/guides/troubleshooting.mdx
+++ b/docs/guides/troubleshooting.mdx
@@ -128,9 +128,10 @@ If your issue is about a specific coding mistake (animations not working, video
1. Use `--quality draft` during development for faster encoding
2. Run `npx hyperframes benchmark` to find the optimal worker count for your system
- 3. Use `--gpu` for hardware-accelerated encoding (local mode only)
- 4. Reduce `--fps` to 24 if 30fps is not needed
- 5. Check that your composition does not have unnecessary elements or overly complex animations
+ 3. Local Chrome/WebGL GPU capture is enabled automatically; compare with `--no-browser-gpu` if troubleshooting
+ 4. Use `--gpu` for hardware-accelerated encoding (local mode only)
+ 5. Reduce `--fps` to 24 if 30fps is not needed
+ 6. Check that your composition does not have unnecessary elements or overly complex animations
See [Rendering: Options](/guides/rendering#options) for all available flags.
diff --git a/docs/guides/video-editor-cheatsheet.mdx b/docs/guides/video-editor-cheatsheet.mdx
new file mode 100644
index 000000000..f8a633d2e
--- /dev/null
+++ b/docs/guides/video-editor-cheatsheet.mdx
@@ -0,0 +1,267 @@
+---
+title: Video Editor Cheatsheet
+description: "Fast reference for video editors and creative people directing agents, cutting timing, tweaking layouts, previewing, and publishing HyperFrames projects."
+---
+
+Use this as a fast reference when you are directing agents, cutting timing, making visual layout tweaks, previewing, and sharing HyperFrames projects.
+
+## The Fast Loop
+
+```bash
+npx hyperframes init my-video --example blank
+cd my-video
+npx hyperframes preview
+```
+
+Keep the preview running while your agent edits `index.html` or files in `compositions/`. The Studio updates automatically, so you can direct the agent, scrub the result, make manual visual tweaks, then repeat.
+
+Most production work should feel like this:
+
+1. Ask the agent for the first cut, scene, caption pass, transition, or cleanup.
+2. Use the Studio preview and timeline to check timing.
+3. Use manual DOM editing for Figma-like layout tweaks: select elements, move them, and adjust visual properties directly.
+4. Ask the agent to clean up or generalize anything you changed manually.
+5. Lint, validate, render, and publish.
+
+Before showing or rendering a project:
+
+```bash
+npx hyperframes lint
+npx hyperframes validate
+npx hyperframes render --quality standard --output review.mp4
+```
+
+For fast iteration renders, use draft quality:
+
+```bash
+npx hyperframes render --quality draft --output draft.mp4
+```
+
+For final delivery:
+
+```bash
+npx hyperframes render --quality high --fps 30 --output final.mp4
+```
+
+## Terminal Shortcuts
+
+Move around projects quickly:
+
+```bash
+pwd # show current folder
+ls # list files
+cd my-video # enter a project folder
+cd .. # go up one folder
+cd - # jump back to the previous folder
+open . # open the current folder in Finder on macOS
+code . # open the current folder in VS Code, if installed
+```
+
+Common HyperFrames project folders:
+
+```bash
+cd assets # source videos, images, audio
+cd compositions # reusable scenes and overlays
+cd .. # back to the project root
+```
+
+Run HyperFrames commands from the project root, where `index.html` lives. If you are not sure where you are, run `pwd` then `ls`. If you see `index.html`, you are in the right place.
+
+## Preview Shortcuts
+
+Start the Studio:
+
+```bash
+npx hyperframes preview
+```
+
+Use a different port if `3002` is already busy:
+
+```bash
+npx hyperframes preview --port 4567
+```
+
+Inside the Studio:
+
+| Shortcut | Use |
+| --- | --- |
+| `Space` | Play or pause (focus on the page body) |
+| `Left Arrow` / `Right Arrow` | Nudge seek bar by 1 second (seek bar focused) |
+| `Shift+Left Arrow` / `Shift+Right Arrow` | Nudge seek bar by 5 seconds (seek bar focused) |
+| `Shift+T` | Show or hide the timeline editor |
+| `Cmd+1` / `Ctrl+1` | Switch to Compositions |
+| `Cmd+2` / `Ctrl+2` | Switch to Assets |
+| `Delete` / `Backspace` | Delete the selected timeline clip (when not typing in an editor) |
+| `Escape` | Leave a sub-composition or close editor dialogs |
+
+
+ Preview uses the same runtime as rendering, so the visual frame matches the output. If preview stutters on a heavy frame but the render is clean, that is expected — preview plays in real time, render captures one frame at a time.
+
+
+## Agent-Led Editing
+
+Ask the agent to verify visible changes in the browser. For a user-visible edit, a good handoff is:
+
+```
+Run the preview, check it with agent-browser, take a screenshot, and render a draft MP4 to take a look at the frames with ffmpeg.
+```
+
+## Manual DOM Editing
+
+In the Studio, you can edit the DOM visually for the final 10% of creative adjustment where dragging is faster than describing.
+
+Use manual DOM editing for:
+
+- moving titles, captions, product cards, logos, and overlays into position
+- adjusting size, spacing, opacity, color, and other visual properties
+- checking composition balance at an exact timestamp
+- making Figma-like placement tweaks
+
+Use agents for:
+
+- creating scenes from scratch
+- refactoring repeated visual patterns
+- wiring GSAP timelines
+- fixing broken timing, layout overflow, or render errors
+- turning a manual visual tweak into reusable, clean HTML/CSS
+
+After manual DOM edits, ask the agent to inspect the diff and keep the source clean:
+
+```
+I moved the hero title and resized the CTA manually in Studio. Inspect the changes, clean up the CSS if needed, then run lint and validate.
+```
+
+## CLI Commands Editors Use Most
+
+| Command | Use it for |
+| --- | --- |
+| `npx hyperframes init my-video` | Create a new project |
+| `npx hyperframes init my-video --example warm-grain` | Start from a visual template |
+| `npx hyperframes init my-video --video source.mp4` | Import video and generate captions from the source audio |
+| `npx hyperframes capture https://example.com` | Capture a website as source material for a video |
+| `npx hyperframes preview` | Open the live Studio preview |
+| `npx hyperframes lint` | Catch structural mistakes before preview or render |
+| `npx hyperframes validate` | Run the composition in headless Chrome to catch runtime errors |
+| `npx hyperframes inspect` | Find text overflow and layout problems across the timeline |
+| `npx hyperframes snapshot --at 1,3,5` | Save PNG checks at exact timestamps |
+| `npx hyperframes render --output final.mp4` | Render the video |
+| `npx hyperframes publish` | Upload the project and get a shareable HyperFrames URL |
+| `npx hyperframes doctor` | Check Node.js, FFmpeg, Chrome, Docker, and other dependencies |
+| `npx hyperframes docs` | Open local CLI docs |
+| `npx hyperframes upgrade` | Check for a newer CLI version |
+
+## Timing Cheatsheet
+
+Every visible timed layer should usually be a clip:
+
+```html
+
+ Opening title
+
+```
+
+Use these attributes like timeline controls:
+
+| Attribute | Video editor meaning |
+| --- | --- |
+| `data-start` | When the layer starts |
+| `data-duration` | How long the layer stays active |
+| `data-track-index` | Timeline track number |
+| `data-media-start` | Offset into a media file |
+| `data-volume` | Audio volume for an audio or video clip |
+| `data-composition-src` | Nested scene or reusable overlay |
+
+For GSAP animation, register one paused timeline per composition:
+
+```html
+
+```
+
+
+ If a video cuts off early, check that the GSAP timeline is at least as long as the intended edit. The final `tl.set({}, {}, 5)` pattern is the fix.
+
+
+## Render Presets
+
+| Goal | Command |
+| --- | --- |
+| Fast iteration | `npx hyperframes render --quality draft --output draft.mp4` |
+| Review link | `npx hyperframes render --quality standard --output review.mp4` |
+| Final export | `npx hyperframes render --quality high --fps 30 --output final.mp4` |
+| Transparent overlay | `npx hyperframes render --format webm --output overlay.webm` |
+| Deterministic output | `npx hyperframes render --docker --output final.mp4` |
+
+Use WebM for transparent overlays, captions, and lower thirds. Use `--docker` when you need pixel-consistent output across different machines.
+
+## Publish and Share
+
+Use `publish` when you want to share the editable project, not just the rendered MP4:
+
+```bash
+npx hyperframes publish
+```
+
+Publish zips the current project, uploads it, and prints a stable `hyperframes.dev` URL. The URL includes a claim token so the recipient can open it, claim the project, and continue editing in the web app.
+
+```bash
+npx hyperframes publish ./my-video # publish a specific folder
+npx hyperframes publish --yes # skip the confirmation prompt in scripts
+```
+
+Publish expects an `index.html` at the project root. It ignores `.git`, `node_modules`, `dist`, `.next`, and `coverage`.
+
+## What Agent Browser Is
+
+`agent-browser` is a browser automation tool for AI agents. It opens Chrome, navigates to your preview, clicks controls, reads page state, and captures screenshots. It is how an agent proves the video preview actually works instead of only saying the code looks right.
+
+Typical verification flow:
+
+```bash
+agent-browser open http://localhost:3002
+agent-browser snapshot -i
+agent-browser screenshot --screenshot-dir ./qa
+```
+
+Use it when you want the agent to open the HyperFrames Studio preview, play or scrub the video, click timeline controls, inspect visible UI text, capture screenshots for review, or record proof of a tested flow.
+
+For editor-facing changes, keep `npx hyperframes preview` running, then have the agent use `agent-browser` against the local preview URL.
+
+## Quick Fixes
+
+| Problem | Command or check |
+| --- | --- |
+| Preview will not start | `npx hyperframes doctor` |
+| Port already in use | `npx hyperframes preview --port 4567` |
+| Render fails | `npx hyperframes lint` then `npx hyperframes validate` |
+| Need exact frame checks | `npx hyperframes snapshot --at 1,2.5,5` |
+| Text overflows in the frame | `npx hyperframes inspect` |
+| Final render is too slow | Try `--quality draft`, reduce image sizes, or lower `--fps` |
+| Need to share editable project | `npx hyperframes publish` |
+
+
+
+ How to direct AI agents to build better videos
+
+
+ Timing, tracks, and GSAP timeline patterns
+
+
+ Pitfalls the linter can't catch
+
+
+ Full command reference
+
+
diff --git a/docs/packages/cli.mdx b/docs/packages/cli.mdx
index 779fce12a..f9ae12f72 100644
--- a/docs/packages/cli.mdx
+++ b/docs/packages/cli.mdx
@@ -148,6 +148,9 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
# Agent mode (default) — --example is required
npx hyperframes init my-video --example blank --video video.mp4
+ # Include Tailwind CSS browser-runtime support
+ npx hyperframes init my-video --example blank --tailwind
+
# Human mode — interactive prompts
npx hyperframes init --human-friendly
```
@@ -157,6 +160,7 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
| `--example, -e` | Example to scaffold (required in default mode, interactive in `--human-friendly`) |
| `--video, -V` | Path to a video file (MP4, WebM, MOV) |
| `--audio, -a` | Path to an audio file (MP3, WAV, M4A) |
+ | `--tailwind` | Add Tailwind CSS browser-runtime support to scaffolded HTML |
| `--skip-skills` | Skip AI coding skills installation |
| `--skip-transcribe` | Skip automatic whisper transcription |
| `--model` | Whisper model for transcription (e.g. `small.en`, `medium.en`, `large-v3`) |
@@ -173,6 +177,8 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
In default (agent) mode, `--example` is required — the CLI errors with a usage example if missing. In `--human-friendly` mode, you choose interactively. When `--video` or `--audio` is provided, the CLI automatically transcribes the audio with Whisper and patches captions into the composition (use `--skip-transcribe` to disable).
+ `--tailwind` injects the pinned Tailwind v4 browser runtime into scaffolded HTML and exposes a `window.__tailwindReady` promise that renders wait on before capturing frame 0. Use the `/tailwind` skill when editing these projects so agents follow v4 CSS-first patterns instead of v3 `tailwind.config.js` and `@tailwind` directive patterns. The browser runtime is still intended for scaffolded projects and quick iteration; for fully offline or locked-down production renders, compile Tailwind to CSS and include the stylesheet directly.
+
After scaffolding, the CLI installs AI coding skills for Claude Code, Gemini CLI, and Codex CLI (use `--skip-skills` to disable). See [`skills`](#skills) command.
See [Examples](/examples) for full details.
@@ -335,6 +341,59 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
Combine `tts` with `transcribe` to generate narration and word-level timestamps for captions in a single workflow: generate the audio with `tts`, then transcribe the output with `transcribe` to get word-level timing.
+
+ ### `remove-background`
+
+ Remove the background from a video or image using a local AI model. The output is transparent media you can drop into any composition's `` or `` for HTML5-native transparent playback | ~1 MB |
+ | `.mov` (ProRes 4444) | Editing round-trip in Premiere / Resolve / DaVinci | ~50 MB |
+ | `.png` | Single-image cutout | varies |
+
+
+ The `` element in Chrome only respects the alpha plane when the WebM is encoded as `yuva420p` with the `alpha_mode=1` metadata tag. The CLI sets both automatically — if you re-encode the output yourself, preserve those flags.
+
+
+ See the [Remove Background guide](/guides/remove-background) for the full workflow — using transparent videos in compositions, performance per platform, limitations of `u2net_human_seg`, and free alternative tools when this model isn't the right fit.
+
### `capture`
Capture a website — extract screenshots, design tokens, fonts, assets, and animations for video production:
@@ -530,6 +589,12 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
# With options
npx hyperframes render --output output.mp4 --fps 60 --quality high
+
+ # Opt out of local browser GPU capture
+ npx hyperframes render --no-browser-gpu --output cpu-browser.mp4
+
+ # Add hardware FFmpeg encoding
+ npx hyperframes render --gpu --output gpu.mp4
```
| Flag | Values | Default | Description |
@@ -540,14 +605,57 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
| `--quality` | draft, standard, high | standard | Encoding quality preset (drives CRF/bitrate) |
| `--crf` | 0-51 | — | Override encoder CRF (lower = higher quality). Mutually exclusive with `--video-bitrate` |
| `--video-bitrate` | e.g. `10M`, `5000k` | — | Target video bitrate. Mutually exclusive with `--crf` |
- | `--hdr` | — | off | Detect HDR sources and output HDR10 (H.265 10-bit, BT.2020 PQ/HLG). MP4 only. SDR-only compositions are unaffected. See [HDR Rendering](/guides/hdr) |
+ | `--hdr` | — | off | Force HDR output even if no HDR sources are detected. MP4 only. See [HDR Rendering](/guides/hdr) |
+ | `--sdr` | — | off | Force SDR output even if HDR sources are detected |
| `--workers` | 1-8 | 4 | Parallel render workers |
- | `--gpu` | — | off | GPU encoding (NVENC, VideoToolbox, VAAPI) |
+ | `--gpu` | — | off | GPU encoding (NVENC, VideoToolbox, VAAPI, QSV) |
+ | `--browser-gpu` / `--no-browser-gpu` | — | on locally, off in Docker | Use or opt out of host GPU acceleration for local Chrome/WebGL capture |
| `--docker` | — | off | Use Docker for [deterministic rendering](/concepts/determinism) |
| `--quiet` | — | off | Suppress verbose output |
+ | `--variables` | JSON object | — | Variable overrides merged over `data-composition-variables` defaults. Read via `window.__hyperframes.getVariables()` |
+ | `--variables-file` | path | — | Path to a JSON file with variable overrides (alternative to `--variables`) |
+ | `--strict-variables` | — | off | Fail render if any `--variables` key is undeclared or has a wrong type vs the composition's `data-composition-variables`. Without this flag, mismatches print as warnings and the render continues. |
CRF and target bitrate default to the `--quality` preset. Use `--crf` or `--video-bitrate` for fine-grained overrides; `RenderConfig.crf` and `RenderConfig.videoBitrate` accept the same overrides programmatically.
+ #### Parametrized renders
+
+ Render the same composition with different content by declaring variables on the composition root and overriding them at render time:
+
+ ```html index.html
+
+
+
- This path requires **headed Chrome with `--enable-unsafe-webgpu`** — WebGPU is unavailable in `chrome-headless-shell`. It is *not* used by the default `--hdr` render pipeline (which extracts HDR pixels from sources via FFmpeg and composites in Node). Use it only for advanced custom pipelines that need CSS animations driving HDR pixel output.
+ This path requires **headed Chrome with `--enable-unsafe-webgpu`** — WebGPU is unavailable in `chrome-headless-shell`. It is *not* used by the default HDR-aware render pipeline (which extracts HDR pixels from sources via FFmpeg and composites in Node). Use it only for advanced custom pipelines that need CSS animations driving HDR pixel output.
## The `window.__hf` Protocol
diff --git a/docs/packages/producer.mdx b/docs/packages/producer.mdx
index 88ffb07ed..1886af13b 100644
--- a/docs/packages/producer.mdx
+++ b/docs/packages/producer.mdx
@@ -213,18 +213,31 @@ npx hyperframes render --docker --output output.mp4
The producer supports hardware-accelerated encoding for faster renders:
-| Platform | Encoder | Flag |
-|----------|---------|------|
+| Platform | Encoder | Selection |
+|----------|---------|-----------|
| NVIDIA | NVENC | Auto-detected |
| macOS | VideoToolbox | Auto-detected |
| Linux | VAAPI | Auto-detected |
+| Intel | QSV | Auto-detected |
-GPU encoding is automatically used when available. To check your system's capabilities:
+When GPU encoding is enabled, Hyperframes detects the available FFmpeg hardware encoder automatically. To check your system's capabilities:
```bash
npx hyperframes doctor
```
+The CLI enables local Chrome/WebGL GPU capture automatically and supports `--no-browser-gpu` as an opt-out. When using the producer API directly, pass an engine config override:
+
+```typescript
+import { resolveConfig } from '@hyperframes/producer';
+
+const job = createRenderJob({
+ fps: 30,
+ quality: 'standard',
+ producerConfig: resolveConfig({ browserGpuMode: 'hardware' }),
+});
+```
+
## Additional Exports
The producer also re-exports key engine functionality for convenience:
diff --git a/docs/public/catalog-index.json b/docs/public/catalog-index.json
index f2f7abd3f..2226c308f 100644
--- a/docs/public/catalog-index.json
+++ b/docs/public/catalog-index.json
@@ -12,6 +12,35 @@
"href": "/catalog/blocks/app-showcase",
"preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/app-showcase.png"
},
+ {
+ "name": "apple-money-count",
+ "type": "block",
+ "title": "Apple Money Count",
+ "description": "Apple-style finance counter that counts from $0 to $10,000, flashes green, and bursts money icons with sound.",
+ "tags": [
+ "showcase",
+ "finance",
+ "kinetic",
+ "youtube",
+ "sfx"
+ ],
+ "href": "/catalog/blocks/apple-money-count",
+ "preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/apple-money-count.png"
+ },
+ {
+ "name": "blue-sweater-intro-video",
+ "type": "block",
+ "title": "Blue Sweater Intro Video",
+ "description": "Warm AI creator intro sequence that resolves into an X follow card for @_blue_sweater_.",
+ "tags": [
+ "showcase",
+ "ai",
+ "creator",
+ "sfx"
+ ],
+ "href": "/catalog/blocks/blue-sweater-intro-video",
+ "preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/blue-sweater-intro-video.png"
+ },
{
"name": "chromatic-radial-split",
"type": "block",
@@ -201,6 +230,36 @@
"href": "/catalog/blocks/macos-notification",
"preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/macos-notification.png"
},
+ {
+ "name": "north-korea-locked-down",
+ "type": "block",
+ "title": "North Korea Locked Down",
+ "description": "Realistic map zoom into North Korea with a red scribble circle, locked-down pop-up label, and reddish editorial wash.",
+ "tags": [
+ "showcase",
+ "map",
+ "annotation",
+ "youtube",
+ "kinetic"
+ ],
+ "href": "/catalog/blocks/north-korea-locked-down",
+ "preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/north-korea-locked-down.png"
+ },
+ {
+ "name": "nyc-paris-flight",
+ "type": "block",
+ "title": "NYC Paris Flight",
+ "description": "Apple-style realistic map animation with a plane flying from New York to Paris, marker circle, landing pop, and sound effects.",
+ "tags": [
+ "showcase",
+ "travel",
+ "map",
+ "youtube",
+ "sfx"
+ ],
+ "href": "/catalog/blocks/nyc-paris-flight",
+ "preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/nyc-paris-flight.png"
+ },
{
"name": "reddit-post",
"type": "block",
@@ -483,6 +542,20 @@
"href": "/catalog/blocks/ui-3d-reveal",
"preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/ui-3d-reveal.png"
},
+ {
+ "name": "vpn-youtube-spot",
+ "type": "block",
+ "title": "VPN YouTube Spot",
+ "description": "Snappy Apple-style YouTube insert showing a phone finding and installing a friendly VPN app with sound effects.",
+ "tags": [
+ "app",
+ "showcase",
+ "youtube",
+ "sfx"
+ ],
+ "href": "/catalog/blocks/vpn-youtube-spot",
+ "preview": "https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/vpn-youtube-spot.png"
+ },
{
"name": "whip-pan",
"type": "block",
diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx
index 498816d66..6add780c9 100644
--- a/docs/quickstart.mdx
+++ b/docs/quickstart.mdx
@@ -13,7 +13,7 @@ Install the HyperFrames skills, then describe the video you want:
npx skills add heygen-com/hyperframes
```
-This teaches your agent (Claude Code, Cursor, Gemini CLI, Codex) how to write correct compositions and GSAP animations. In Claude Code the skills register as slash commands — `/hyperframes` for composition authoring, `/hyperframes-cli` for CLI commands, and `/gsap` for animation help. Invoking the slash command loads the skill context explicitly, which produces correct output the first time.
+This teaches your agent (Claude Code, Cursor, Gemini CLI, Codex) how to write correct compositions, GSAP timelines, Tailwind v4 browser-runtime styles, and first-party adapter animations. In Claude Code the skills register as slash commands — `/hyperframes` for composition authoring, `/hyperframes-cli` for the dev-loop commands (init, lint, preview, render), `/hyperframes-media` for asset preprocessing (TTS, transcription, background removal), `/tailwind` for `init --tailwind` projects, `/gsap` for timeline animation help, and `/animejs`, `/css-animations`, `/lottie`, `/three`, or `/waapi` when a composition uses those runtimes. Invoking the slash command loads the skill context explicitly, which produces correct output the first time.
Claude Design uses a different entry path. Open [`docs/guides/claude-design-hyperframes.md`](https://github.com/heygen-com/hyperframes/blob/main/docs/guides/claude-design-hyperframes.md) on GitHub, click the download button (↓) to save it, then attach to your Claude Design chat. It produces a valid first draft you can refine in any AI coding agent. See the [Claude Design guide](/guides/claude-design).
@@ -47,7 +47,7 @@ Copy any of these into your agent to get started.
The agent handles scaffolding, animation, and rendering. See the [prompting guide](/guides/prompting) for more patterns.
- Skills encode HyperFrames-specific patterns — like required `class="clip"` on timed elements, GSAP timeline registration, and `data-*` attribute semantics — that are not in generic web docs. Using skills produces correct compositions from the start.
+ Skills encode HyperFrames-specific patterns — like required `class="clip"` on timed elements, GSAP timeline registration, adapter registries such as `window.__hfLottie`, and `data-*` attribute semantics — that are not in generic web docs. Using skills produces correct compositions from the start.
## Option 2: Start a project manually
@@ -237,6 +237,6 @@ The agent handles scaffolding, animation, and rendering. See the [prompting guid
Start from built-in examples like Warm Grain and Swiss Grid
- Explore render options: quality presets, Docker mode, and GPU encoding
+ Explore render options: quality presets, Docker mode, and GPU acceleration
diff --git a/docs/schema/registry-item.json b/docs/schema/registry-item.json
index ba72da93c..a2051a5bf 100644
--- a/docs/schema/registry-item.json
+++ b/docs/schema/registry-item.json
@@ -35,6 +35,14 @@
"type": "string",
"minLength": 1
},
+ "authorUrl": {
+ "type": "string",
+ "format": "uri"
+ },
+ "sourcePrompt": {
+ "type": "string",
+ "minLength": 1
+ },
"license": {
"type": "string",
"minLength": 1,
diff --git a/package.json b/package.json
index c3d5d5d82..5310bb8a2 100644
--- a/package.json
+++ b/package.json
@@ -11,7 +11,7 @@
"type": "module",
"scripts": {
"dev": "bun run studio",
- "build": "bun run --filter '!@hyperframes/cli' build && bun run --filter @hyperframes/cli build",
+ "build": "bun run --filter @hyperframes/core build && bun run --filter '@hyperframes/{core,engine,producer,player,studio,shader-transitions}' build && bun run --filter @hyperframes/cli build",
"build:producer": "bun run --filter @hyperframes/producer build",
"studio": "bun run --filter @hyperframes/studio dev",
"build:hyperframes-runtime": "bun run --filter @hyperframes/core build:hyperframes-runtime",
diff --git a/packages/cli/package.json b/packages/cli/package.json
index cd6c2e397..d7c49870c 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -1,6 +1,6 @@
{
"name": "@hyperframes/cli",
- "version": "0.4.27",
+ "version": "0.4.45",
"description": "HyperFrames CLI — create, preview, and render HTML video compositions",
"repository": {
"type": "git",
@@ -33,6 +33,7 @@
"giget": "^3.2.0",
"hono": "^4.0.0",
"mime-types": "^3.0.2",
+ "onnxruntime-node": "^1.20.0",
"open": "^10.0.0",
"postcss": "^8.5.8",
"prettier": "^3.8.1",
@@ -56,7 +57,7 @@
"vitest": "^3.2.4"
},
"optionalDependencies": {
- "@google/genai": "^1.50.0"
+ "@google/genai": "^1.50.1"
},
"engines": {
"node": ">=22"
diff --git a/packages/cli/scripts/build-copy.mjs b/packages/cli/scripts/build-copy.mjs
index 9132c7095..926493a9d 100644
--- a/packages/cli/scripts/build-copy.mjs
+++ b/packages/cli/scripts/build-copy.mjs
@@ -82,6 +82,11 @@ async function main() {
cpSync(layoutAuditScript, join(DIST, "commands", "layout-audit.browser.js"));
}
+ const contrastAuditScript = join(CLI_ROOT, "src", "commands", "contrast-audit.browser.js");
+ if (existsSync(contrastAuditScript)) {
+ cpSync(contrastAuditScript, join(DIST, "commands", "contrast-audit.browser.js"));
+ }
+
copyMdFiles(join(CLI_ROOT, "src", "docs"), join(DIST, "docs"));
console.log("[build-copy] done");
diff --git a/packages/cli/src/background-removal/inference.test.ts b/packages/cli/src/background-removal/inference.test.ts
new file mode 100644
index 000000000..719bcab4d
--- /dev/null
+++ b/packages/cli/src/background-removal/inference.test.ts
@@ -0,0 +1,127 @@
+import { describe, expect, it } from "vitest";
+import { MEAN, STD, applyMask } from "./inference.js";
+
+// Regression: the u2net_human_seg model was trained with ImageNet
+// normalization. Drifting away from these exact values changes the input
+// tensor at every pixel and shifts the predicted alpha mask noticeably
+// (Miguel reproduced 8,317 pixel changes with delta up to 78/255 when std
+// was set to (1, 1, 1)). Reference:
+// https://github.com/danielgatis/rembg/blob/main/rembg/sessions/u2net_human_seg.py#L33
+describe("background-removal/inference — rembg u2net_human_seg parity", () => {
+ it("MEAN matches U2netHumanSegSession reference", () => {
+ expect(MEAN).toEqual([0.485, 0.456, 0.406]);
+ });
+
+ it("STD matches U2netHumanSegSession reference (ImageNet, not the base u2net's (1,1,1))", () => {
+ expect(STD).toEqual([0.229, 0.224, 0.225]);
+ });
+});
+
+// These tests pin the contract that `--background-output` is built on:
+// fg.alpha + bg.alpha === 255 per pixel, and the RGB plane is byte-identical
+// between fg and bg. A future change to the postprocess loop (different mask
+// threshold, premultiplied alpha, gamma-corrected compositing) that breaks
+// either invariant should fail here loudly.
+describe("background-removal/inference — applyMask invariants", () => {
+ function makeRgb(pixels: number): Buffer {
+ // Deterministic but non-trivial RGB so byte equality is meaningful.
+ const buf = Buffer.allocUnsafe(pixels * 3);
+ for (let i = 0; i < pixels; i++) {
+ buf[i * 3] = (i * 7) & 0xff;
+ buf[i * 3 + 1] = (i * 13 + 31) & 0xff;
+ buf[i * 3 + 2] = (i * 19 + 61) & 0xff;
+ }
+ return buf;
+ }
+
+ function makeMask(pixels: number): Buffer {
+ // Hit the saturation endpoints (0, 255) and a few mid-tone values so the
+ // 255-m inversion is exercised across the full byte range.
+ const buf = Buffer.allocUnsafe(pixels);
+ for (let i = 0; i < pixels; i++) buf[i] = (i * 37) & 0xff;
+ return buf;
+ }
+
+ it("dual-output: fg.alpha + bg.alpha === 255 for every pixel", () => {
+ const pixels = 64;
+ const rgb = makeRgb(pixels);
+ const mask = makeMask(pixels);
+ const fg = Buffer.allocUnsafe(pixels * 4);
+ const bg = Buffer.allocUnsafe(pixels * 4);
+
+ const result = applyMask(rgb, mask, fg, bg, pixels);
+
+ expect(result.fg).toBe(fg);
+ expect(result.bg).toBe(bg);
+ for (let i = 0; i < pixels; i++) {
+ const sum = fg[i * 4 + 3]! + bg[i * 4 + 3]!;
+ expect(sum).toBe(255);
+ }
+ });
+
+ it("dual-output: RGB triples are byte-identical between fg and bg", () => {
+ const pixels = 64;
+ const rgb = makeRgb(pixels);
+ const mask = makeMask(pixels);
+ const fg = Buffer.allocUnsafe(pixels * 4);
+ const bg = Buffer.allocUnsafe(pixels * 4);
+
+ applyMask(rgb, mask, fg, bg, pixels);
+
+ for (let i = 0; i < pixels; i++) {
+ expect(fg[i * 4]).toBe(bg[i * 4]);
+ expect(fg[i * 4 + 1]).toBe(bg[i * 4 + 1]);
+ expect(fg[i * 4 + 2]).toBe(bg[i * 4 + 2]);
+ // And both match the source.
+ expect(fg[i * 4]).toBe(rgb[i * 3]);
+ expect(fg[i * 4 + 1]).toBe(rgb[i * 3 + 1]);
+ expect(fg[i * 4 + 2]).toBe(rgb[i * 3 + 2]);
+ }
+ });
+
+ it("dual-output: fg.alpha equals the input mask", () => {
+ const pixels = 32;
+ const rgb = makeRgb(pixels);
+ const mask = makeMask(pixels);
+ const fg = Buffer.allocUnsafe(pixels * 4);
+ const bg = Buffer.allocUnsafe(pixels * 4);
+
+ applyMask(rgb, mask, fg, bg, pixels);
+
+ for (let i = 0; i < pixels; i++) {
+ expect(fg[i * 4 + 3]).toBe(mask[i]);
+ }
+ });
+
+ it("single-output: bg=null returns bg=null and writes only fg", () => {
+ const pixels = 32;
+ const rgb = makeRgb(pixels);
+ const mask = makeMask(pixels);
+ const fg = Buffer.allocUnsafe(pixels * 4);
+
+ const result = applyMask(rgb, mask, fg, null, pixels);
+
+ expect(result.bg).toBeNull();
+ expect(result.fg).toBe(fg);
+ for (let i = 0; i < pixels; i++) {
+ expect(fg[i * 4]).toBe(rgb[i * 3]);
+ expect(fg[i * 4 + 3]).toBe(mask[i]);
+ }
+ });
+
+ it("saturates correctly at mask=0 and mask=255", () => {
+ // mask=0 → fg.alpha=0 (transparent subject), bg.alpha=255 (fully opaque plate)
+ // mask=255 → fg.alpha=255 (fully opaque subject), bg.alpha=0 (transparent plate)
+ const rgb = Buffer.from([10, 20, 30, 40, 50, 60]);
+ const mask = Buffer.from([0, 255]);
+ const fg = Buffer.allocUnsafe(8);
+ const bg = Buffer.allocUnsafe(8);
+
+ applyMask(rgb, mask, fg, bg, 2);
+
+ expect(fg[3]).toBe(0);
+ expect(bg[3]).toBe(255);
+ expect(fg[7]).toBe(255);
+ expect(bg[7]).toBe(0);
+ });
+});
diff --git a/packages/cli/src/background-removal/inference.ts b/packages/cli/src/background-removal/inference.ts
new file mode 100644
index 000000000..605257fc3
--- /dev/null
+++ b/packages/cli/src/background-removal/inference.ts
@@ -0,0 +1,249 @@
+/**
+ * u2net_human_seg inference: RGB frame → RGBA frame (alpha = human mask).
+ *
+ * Pre/postprocessing matches rembg's u2net session
+ * (https://github.com/danielgatis/rembg/blob/main/rembg/sessions/u2net.py)
+ * so output should be pixel-equivalent to `rembg new_session("u2net_human_seg")`.
+ */
+import type { InferenceSession, Tensor } from "onnxruntime-node";
+import type sharpType from "sharp";
+import { ensureModel, selectProviders, type Device, type ModelId } from "./manager.js";
+
+const INPUT_SIZE = 320;
+const INPUT_PLANE = INPUT_SIZE * INPUT_SIZE;
+
+// Must match rembg's U2netHumanSegSession.predict — ImageNet mean/std, NOT the
+// (1.0, 1.0, 1.0) std used by the general-purpose u2net session.
+// https://github.com/danielgatis/rembg/blob/main/rembg/sessions/u2net_human_seg.py#L33
+export const MEAN = [0.485, 0.456, 0.406] as const;
+export const STD = [0.229, 0.224, 0.225] as const;
+
+type Sharp = typeof sharpType;
+interface OrtModule {
+ InferenceSession: typeof InferenceSession;
+ Tensor: typeof Tensor;
+}
+
+export interface SessionResult {
+ /** Subject opaque, background fully transparent. */
+ fg: Buffer;
+ /** Inverse-alpha plate: same RGB, alpha is `255 − mask`. Null unless `withBackground` was true. */
+ bg: Buffer | null;
+}
+
+export interface Session {
+ /**
+ * Both `fg` and `bg` (when requested) are session-owned buffers reused on the
+ * next call — drain the encoder's stdin before invoking `process` again.
+ */
+ process(
+ rgb: Buffer,
+ width: number,
+ height: number,
+ withBackground?: boolean,
+ ): Promise;
+ provider: string;
+ close(): Promise;
+}
+
+export interface CreateSessionOptions {
+ model?: ModelId;
+ device?: Device;
+ onProgress?: (message: string) => void;
+}
+
+export async function createSession(options: CreateSessionOptions = {}): Promise {
+ const ort = (await import("onnxruntime-node")) as unknown as OrtModule;
+ const sharpMod = await import("sharp");
+ const sharp = sharpMod.default as Sharp;
+
+ const choice = selectProviders(options.device ?? "auto");
+ const path = await ensureModel(options.model, { onProgress: options.onProgress });
+
+ options.onProgress?.(`Loading model on ${choice.label}...`);
+
+ const tryCreate = (providers: string[]) =>
+ ort.InferenceSession.create(path, {
+ executionProviders: providers,
+ graphOptimizationLevel: "all",
+ });
+
+ let session: InferenceSession;
+ let providerUsed = choice.label;
+ try {
+ session = await tryCreate(choice.providers);
+ } catch (err) {
+ if (choice.providers[0] === "cpu") throw err;
+ options.onProgress?.(
+ `${choice.label} provider failed (${(err as Error).message}); falling back to CPU.`,
+ );
+ session = await tryCreate(["cpu"]);
+ providerUsed = "CPU";
+ }
+
+ const inputName = session.inputNames[0];
+ const outputName = session.outputNames[0];
+ if (!inputName || !outputName) {
+ throw new Error("ONNX session is missing input or output bindings");
+ }
+
+ // Reused across calls; sized lazily on first frame. Saves ~9 MB/frame at 1080p.
+ const inputData = new Float32Array(3 * INPUT_PLANE);
+ const maskBuf = Buffer.allocUnsafe(INPUT_PLANE);
+ let rgbaBuf: Buffer | null = null;
+ let rgbaBgBuf: Buffer | null = null;
+
+ return {
+ provider: providerUsed,
+ async process(rgb, width, height, withBackground = false) {
+ const tensor = await preprocess(sharp, ort, rgb, width, height, inputData);
+ const outputs = await session.run({ [inputName]: tensor });
+ const output = outputs[outputName];
+ if (!output) throw new Error(`Model did not return output '${outputName}'`);
+ const expectedBytes = width * height * 4;
+ if (!rgbaBuf || rgbaBuf.length !== expectedBytes) {
+ rgbaBuf = Buffer.allocUnsafe(expectedBytes);
+ }
+ if (withBackground) {
+ if (!rgbaBgBuf || rgbaBgBuf.length !== expectedBytes) {
+ rgbaBgBuf = Buffer.allocUnsafe(expectedBytes);
+ }
+ }
+ return await postprocess(
+ sharp,
+ output,
+ rgb,
+ width,
+ height,
+ maskBuf,
+ rgbaBuf,
+ withBackground ? rgbaBgBuf : null,
+ );
+ },
+ async close() {
+ await session.release();
+ },
+ };
+}
+
+async function preprocess(
+ sharp: Sharp,
+ ort: OrtModule,
+ rgb: Buffer,
+ width: number,
+ height: number,
+ inputData: Float32Array,
+): Promise {
+ const resized = await sharp(rgb, { raw: { width, height, channels: 3 } })
+ .resize(INPUT_SIZE, INPUT_SIZE, { kernel: "lanczos3", fit: "fill" })
+ .raw()
+ .toBuffer();
+
+ // rembg's normalize divides by `np.max(im_ary)` (NOT 255). Match exactly so
+ // we hit the same operating point as the model's training distribution.
+ let maxPixel = 0;
+ for (let i = 0; i < resized.length; i++) {
+ if (resized[i]! > maxPixel) maxPixel = resized[i]!;
+ }
+ if (maxPixel === 0) maxPixel = 1;
+
+ for (let y = 0; y < INPUT_SIZE; y++) {
+ for (let x = 0; x < INPUT_SIZE; x++) {
+ const src = (y * INPUT_SIZE + x) * 3;
+ const dst = y * INPUT_SIZE + x;
+ inputData[dst] = (resized[src]! / maxPixel - MEAN[0]) / STD[0];
+ inputData[INPUT_PLANE + dst] = (resized[src + 1]! / maxPixel - MEAN[1]) / STD[1];
+ inputData[2 * INPUT_PLANE + dst] = (resized[src + 2]! / maxPixel - MEAN[2]) / STD[2];
+ }
+ }
+
+ return new ort.Tensor("float32", inputData, [1, 3, INPUT_SIZE, INPUT_SIZE]);
+}
+
+async function postprocess(
+ sharp: Sharp,
+ output: Tensor,
+ rgb: Buffer,
+ width: number,
+ height: number,
+ maskBuf: Buffer,
+ rgbaBuf: Buffer,
+ rgbaBgBuf: Buffer | null,
+): Promise {
+ const raw = output.data as Float32Array;
+
+ let lo = Infinity;
+ let hi = -Infinity;
+ for (let i = 0; i < INPUT_PLANE; i++) {
+ const v = raw[i]!;
+ if (v < lo) lo = v;
+ if (v > hi) hi = v;
+ }
+ const range = hi - lo || 1;
+
+ for (let i = 0; i < INPUT_PLANE; i++) {
+ const norm = (raw[i]! - lo) / range;
+ maskBuf[i] = Math.max(0, Math.min(255, Math.round(norm * 255)));
+ }
+
+ // lanczos3 keeps soft edges; nearest leaves visible jaggies on hair.
+ // Sharp upcasts the single-channel raw input to a 3-channel buffer during
+ // resize, so the output is laid out as RGB-interleaved (R0,G0,B0,R1,G1,B1,...)
+ // even though all three channels carry the same grayscale value. Force the
+ // output back to single channel with toColourspace("b-w") so we can index
+ // it linearly as a mask.
+ const fullMask = await sharp(maskBuf, {
+ raw: { width: INPUT_SIZE, height: INPUT_SIZE, channels: 1 },
+ })
+ .resize(width, height, { kernel: "lanczos3", fit: "fill" })
+ .toColourspace("b-w")
+ .raw()
+ .toBuffer();
+
+ return applyMask(rgb, fullMask, rgbaBuf, rgbaBgBuf, width * height);
+}
+
+/**
+ * Composite the RGB source frame with the segmentation mask into one or two
+ * RGBA buffers. The contract this PR is built on:
+ * - `fg`'s alpha is the mask, `bg`'s alpha (when provided) is `255 − mask`,
+ * so `fg.alpha + bg.alpha === 255` for every pixel.
+ * - RGB triples are byte-identical between `fg` and `bg`.
+ * - When `bg` is null, only `fg` is touched.
+ *
+ * Exported for direct unit testing of the invariants above without spinning
+ * up an ONNX session.
+ */
+export function applyMask(
+ rgb: Buffer,
+ mask: Buffer,
+ fg: Buffer,
+ bg: Buffer | null,
+ pixels: number,
+): SessionResult {
+ if (bg) {
+ for (let i = 0; i < pixels; i++) {
+ const r = rgb[i * 3]!;
+ const g = rgb[i * 3 + 1]!;
+ const b = rgb[i * 3 + 2]!;
+ const m = mask[i]!;
+ const o = i * 4;
+ fg[o] = r;
+ fg[o + 1] = g;
+ fg[o + 2] = b;
+ fg[o + 3] = m;
+ bg[o] = r;
+ bg[o + 1] = g;
+ bg[o + 2] = b;
+ bg[o + 3] = 255 - m;
+ }
+ return { fg, bg };
+ }
+ for (let i = 0; i < pixels; i++) {
+ fg[i * 4] = rgb[i * 3]!;
+ fg[i * 4 + 1] = rgb[i * 3 + 1]!;
+ fg[i * 4 + 2] = rgb[i * 3 + 2]!;
+ fg[i * 4 + 3] = mask[i]!;
+ }
+ return { fg, bg: null };
+}
diff --git a/packages/cli/src/background-removal/manager.test.ts b/packages/cli/src/background-removal/manager.test.ts
new file mode 100644
index 000000000..1cc24bd07
--- /dev/null
+++ b/packages/cli/src/background-removal/manager.test.ts
@@ -0,0 +1,81 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+
+describe("background-removal/manager — selectProviders", () => {
+ beforeEach(() => {
+ vi.resetModules();
+ delete process.env["HYPERFRAMES_CUDA"];
+ });
+
+ afterEach(() => {
+ vi.restoreAllMocks();
+ });
+
+ it("returns CPU explicitly when --device cpu", async () => {
+ vi.doMock("node:os", () => ({
+ platform: () => "darwin",
+ arch: () => "arm64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ const choice = selectProviders("cpu");
+ expect(choice.providers).toEqual(["cpu"]);
+ expect(choice.label).toBe("CPU");
+ });
+
+ it("auto picks CoreML on darwin-arm64", async () => {
+ vi.doMock("node:os", () => ({
+ platform: () => "darwin",
+ arch: () => "arm64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ const choice = selectProviders("auto");
+ expect(choice.providers).toEqual(["coreml", "cpu"]);
+ expect(choice.label).toBe("CoreML");
+ });
+
+ it("auto falls back to CPU on linux without HYPERFRAMES_CUDA", async () => {
+ vi.doMock("node:os", () => ({
+ platform: () => "linux",
+ arch: () => "x64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ const choice = selectProviders("auto");
+ expect(choice.providers).toEqual(["cpu"]);
+ expect(choice.label).toBe("CPU");
+ });
+
+ it("auto picks CUDA on linux when HYPERFRAMES_CUDA=1", async () => {
+ process.env["HYPERFRAMES_CUDA"] = "1";
+ vi.doMock("node:os", () => ({
+ platform: () => "linux",
+ arch: () => "x64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ const choice = selectProviders("auto");
+ expect(choice.providers).toEqual(["cuda", "cpu"]);
+ expect(choice.label).toBe("CUDA");
+ });
+
+ it("--device coreml on linux throws", async () => {
+ vi.doMock("node:os", () => ({
+ platform: () => "linux",
+ arch: () => "x64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ expect(() => selectProviders("coreml")).toThrow(/CoreML execution provider not available/);
+ });
+
+ it("--device cuda without env var throws", async () => {
+ vi.doMock("node:os", () => ({
+ platform: () => "linux",
+ arch: () => "x64",
+ homedir: () => "/tmp",
+ }));
+ const { selectProviders } = await import("./manager.js");
+ expect(() => selectProviders("cuda")).toThrow(/CUDA execution provider not available/);
+ });
+});
diff --git a/packages/cli/src/background-removal/manager.ts b/packages/cli/src/background-removal/manager.ts
new file mode 100644
index 000000000..d149c0e2e
--- /dev/null
+++ b/packages/cli/src/background-removal/manager.ts
@@ -0,0 +1,96 @@
+import { existsSync, mkdirSync } from "node:fs";
+import { homedir, platform, arch } from "node:os";
+import { join } from "node:path";
+import { downloadFile } from "../utils/download.js";
+
+export const MODELS_DIR = join(homedir(), ".cache", "hyperframes", "background-removal", "models");
+
+export const DEFAULT_MODEL = "u2net_human_seg" as const;
+export type ModelId = typeof DEFAULT_MODEL;
+
+const MODEL_URLS: Record = {
+ u2net_human_seg:
+ "https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net_human_seg.onnx",
+};
+
+export const MODEL_MEMORY_MB: Record = {
+ u2net_human_seg: 1500,
+};
+
+export const DEVICES = ["auto", "cpu", "coreml", "cuda"] as const;
+export type Device = (typeof DEVICES)[number];
+
+export function isDevice(value: unknown): value is Device {
+ return typeof value === "string" && (DEVICES as readonly string[]).includes(value);
+}
+
+export interface ProviderChoice {
+ providers: string[];
+ label: "CoreML" | "CUDA" | "CPU";
+}
+
+export function selectProviders(device: Device = "auto"): ProviderChoice {
+ if (device === "cpu") return { providers: ["cpu"], label: "CPU" };
+
+ const available = listAvailableProviders();
+ const hasCoreML = available.includes("coreml");
+ const hasCUDA = available.includes("cuda");
+
+ if (device === "coreml") {
+ if (!hasCoreML) {
+ throw new Error(
+ "CoreML execution provider not available. Install onnxruntime-node on Apple Silicon, or use --device cpu.",
+ );
+ }
+ return { providers: ["coreml", "cpu"], label: "CoreML" };
+ }
+ if (device === "cuda") {
+ if (!hasCUDA) {
+ throw new Error(
+ "CUDA execution provider not available. Use --device cpu or install an onnxruntime-node build with CUDA support.",
+ );
+ }
+ return { providers: ["cuda", "cpu"], label: "CUDA" };
+ }
+
+ if (hasCoreML && platform() === "darwin" && arch() === "arm64") {
+ return { providers: ["coreml", "cpu"], label: "CoreML" };
+ }
+ if (hasCUDA) return { providers: ["cuda", "cpu"], label: "CUDA" };
+ return { providers: ["cpu"], label: "CPU" };
+}
+
+let _cachedProviders: string[] | undefined;
+export function listAvailableProviders(): string[] {
+ if (_cachedProviders) return _cachedProviders;
+
+ // The npm onnxruntime-node ships with CPU on every platform and bundles the
+ // CoreML EP only on darwin-arm64. CUDA is opt-in via a separate gpu build —
+ // gate behind an env var so we don't try to bind to a missing EP.
+ const out: string[] = ["cpu"];
+ if (platform() === "darwin" && arch() === "arm64") out.push("coreml");
+ if (process.env["HYPERFRAMES_CUDA"] === "1") out.push("cuda");
+ _cachedProviders = out;
+ return out;
+}
+
+export function modelPath(model: ModelId = DEFAULT_MODEL): string {
+ return join(MODELS_DIR, `${model}.onnx`);
+}
+
+export async function ensureModel(
+ model: ModelId = DEFAULT_MODEL,
+ options?: { onProgress?: (message: string) => void },
+): Promise {
+ const dest = modelPath(model);
+ if (existsSync(dest)) return dest;
+
+ mkdirSync(MODELS_DIR, { recursive: true });
+ options?.onProgress?.(`Downloading ${model} weights (~168 MB)...`);
+ await downloadFile(MODEL_URLS[model], dest);
+
+ if (!existsSync(dest)) {
+ throw new Error(`Model download failed: ${model}`);
+ }
+ return dest;
+}
diff --git a/packages/cli/src/background-removal/pipeline.test.ts b/packages/cli/src/background-removal/pipeline.test.ts
new file mode 100644
index 000000000..5763efb06
--- /dev/null
+++ b/packages/cli/src/background-removal/pipeline.test.ts
@@ -0,0 +1,191 @@
+import { describe, expect, it } from "vitest";
+import { EventEmitter } from "node:events";
+import type { spawn } from "node:child_process";
+import {
+ inferOutputFormat,
+ inferInputKind,
+ buildEncoderArgs,
+ resolveRenderTargets,
+ waitForExit,
+} from "./pipeline.js";
+
+describe("background-removal/pipeline — inferOutputFormat", () => {
+ it("maps .webm → webm", () => {
+ expect(inferOutputFormat("/tmp/out.webm")).toBe("webm");
+ });
+ it("maps .mov → mov", () => {
+ expect(inferOutputFormat("/tmp/out.mov")).toBe("mov");
+ });
+ it("maps .png → png", () => {
+ expect(inferOutputFormat("/tmp/out.png")).toBe("png");
+ });
+ it("rejects unknown extensions", () => {
+ expect(() => inferOutputFormat("/tmp/out.mp4")).toThrow(/Unsupported output extension/);
+ });
+});
+
+describe("background-removal/pipeline — inferInputKind", () => {
+ it("recognizes mp4/mov/webm/mkv/avi as video", () => {
+ for (const ext of [".mp4", ".mov", ".webm", ".mkv", ".avi"]) {
+ expect(inferInputKind(`/tmp/clip${ext}`)).toBe("video");
+ }
+ });
+ it("recognizes jpg/png/webp as image", () => {
+ for (const ext of [".jpg", ".jpeg", ".png", ".webp"]) {
+ expect(inferInputKind(`/tmp/img${ext}`)).toBe("image");
+ }
+ });
+ it("rejects unknown extensions", () => {
+ expect(() => inferInputKind("/tmp/file.gif")).toThrow(/Unsupported input/);
+ });
+});
+
+describe("background-removal/pipeline — buildEncoderArgs", () => {
+ it("webm preset emits VP9 + alpha_mode metadata", () => {
+ const args = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/out.webm");
+ expect(args).toContain("libvpx-vp9");
+ expect(args).toContain("yuva420p");
+ // The alpha_mode metadata must be present; without it Chrome ignores the alpha plane.
+ const idx = args.indexOf("-metadata:s:v:0");
+ expect(idx).toBeGreaterThan(-1);
+ expect(args[idx + 1]).toBe("alpha_mode=1");
+ expect(args[args.length - 1]).toBe("/tmp/out.webm");
+ });
+
+ it("webm preset tags BT.709 colorspace + limited range", () => {
+ // Without these tags, ffmpeg's RGB→YUV conversion uses the BT.601 default,
+ // and Chrome's YUV→RGB pass on the resulting webm produces a different
+ // RGB triple than the source mp4 (visible color shift on overlay). Pin
+ // BT.709 limited-range so the cutout matches modern Rec.709 sources.
+ const args = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/out.webm");
+ const csIdx = args.indexOf("-colorspace");
+ expect(csIdx).toBeGreaterThan(-1);
+ expect(args[csIdx + 1]).toBe("bt709");
+ const rangeIdx = args.indexOf("-color_range");
+ expect(rangeIdx).toBeGreaterThan(-1);
+ expect(args[rangeIdx + 1]).toBe("tv");
+ });
+
+ it("webm quality presets map to crf 30/18/12", () => {
+ const fast = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/o.webm", "fast");
+ const balanced = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/o.webm", "balanced");
+ const best = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/o.webm", "best");
+ const crf = (args: string[]) => args[args.indexOf("-crf") + 1];
+ expect(crf(fast)).toBe("30");
+ expect(crf(balanced)).toBe("18");
+ expect(crf(best)).toBe("12");
+ });
+
+ it("webm default quality is balanced (crf 18)", () => {
+ const args = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/o.webm");
+ expect(args[args.indexOf("-crf") + 1]).toBe("18");
+ });
+
+ it("mov preset emits ProRes 4444 + yuva444p10le", () => {
+ const args = buildEncoderArgs("mov", 1920, 1080, 30, "/tmp/out.mov");
+ expect(args).toContain("prores_ks");
+ expect(args).toContain("4444");
+ expect(args).toContain("yuva444p10le");
+ });
+
+ it("png preset emits a single RGBA frame", () => {
+ const args = buildEncoderArgs("png", 1920, 1080, 30, "/tmp/out.png");
+ expect(args).toContain("-frames:v");
+ expect(args).toContain("rgba");
+ });
+
+ it("threads input dimensions and fps into raw video header", () => {
+ const args = buildEncoderArgs("webm", 640, 480, 24, "/tmp/o.webm");
+ const sIdx = args.indexOf("-s");
+ expect(args[sIdx + 1]).toBe("640x480");
+ const rIdx = args.indexOf("-r");
+ expect(args[rIdx + 1]).toBe("24");
+ });
+});
+
+describe("background-removal/pipeline — resolveRenderTargets", () => {
+ it("resolves a normal video → webm render", () => {
+ const t = resolveRenderTargets("/tmp/clip.mp4", "/tmp/cutout.webm");
+ expect(t.format).toBe("webm");
+ expect(t.inputKind).toBe("video");
+ expect(t.bgFormat).toBeUndefined();
+ });
+
+ it("resolves an image → png render", () => {
+ const t = resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.png");
+ expect(t.format).toBe("png");
+ expect(t.inputKind).toBe("image");
+ });
+
+ it("rejects image input with a video output extension", () => {
+ expect(() => resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.webm")).toThrow(
+ /Image input requires a \.png output/,
+ );
+ });
+
+ it("rejects video input with a .png output", () => {
+ expect(() => resolveRenderTargets("/tmp/clip.mp4", "/tmp/cutout.png")).toThrow(
+ /Video input requires a \.webm or \.mov output/,
+ );
+ });
+
+ it("threads background-output format through when valid", () => {
+ const t = resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.webm");
+ expect(t.bgFormat).toBe("webm");
+ const tMov = resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.mov");
+ expect(tMov.bgFormat).toBe("mov");
+ });
+
+ it("rejects --background-output for image inputs (no temporal pairing to do)", () => {
+ expect(() =>
+ resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.png", "/tmp/bg.png"),
+ ).toThrow(/--background-output is not supported for image inputs/);
+ });
+
+ it("rejects .png as the --background-output extension", () => {
+ // .png is only valid for single-image inputs, and image inputs themselves
+ // can't have a background-output anyway. So .png here is always a misuse.
+ expect(() => resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.png")).toThrow(
+ /--background-output must be \.webm or \.mov/,
+ );
+ });
+});
+
+// Regression: a previous version of waitForExit treated `code === null` as
+// success. Per Node's child_process docs, that's the signal-killed case —
+// reporting it as success means a SIGTERM/SIGKILL'd ffmpeg encoder produces
+// a "successful" render with a missing or truncated output file.
+describe("background-removal/pipeline — waitForExit signal handling", () => {
+ function fakeProc(): ReturnType {
+ return new EventEmitter() as unknown as ReturnType;
+ }
+
+ it("resolves on a clean exit (code=0, signal=null)", async () => {
+ const proc = fakeProc();
+ const promise = waitForExit(proc, "ffmpeg encoder", () => "");
+ proc.emit("exit", 0, null);
+ await expect(promise).resolves.toBeUndefined();
+ });
+
+ it("rejects when killed by signal (code=null, signal='SIGTERM')", async () => {
+ const proc = fakeProc();
+ const promise = waitForExit(proc, "ffmpeg encoder", () => "tail of stderr");
+ proc.emit("exit", null, "SIGTERM");
+ await expect(promise).rejects.toThrow(/killed by SIGTERM/);
+ await expect(promise).rejects.toThrow(/tail of stderr/);
+ });
+
+ it("rejects on non-zero exit code", async () => {
+ const proc = fakeProc();
+ const promise = waitForExit(proc, "ffmpeg encoder", () => "");
+ proc.emit("exit", 1, null);
+ await expect(promise).rejects.toThrow(/exited with code 1/);
+ });
+
+ it("rejects on SIGKILL", async () => {
+ const proc = fakeProc();
+ const promise = waitForExit(proc, "ffmpeg encoder", () => "");
+ proc.emit("exit", null, "SIGKILL");
+ await expect(promise).rejects.toThrow(/killed by SIGKILL/);
+ });
+});
diff --git a/packages/cli/src/background-removal/pipeline.ts b/packages/cli/src/background-removal/pipeline.ts
new file mode 100644
index 000000000..a0a0f3bfc
--- /dev/null
+++ b/packages/cli/src/background-removal/pipeline.ts
@@ -0,0 +1,470 @@
+/**
+ * Background-removal rendering pipeline.
+ *
+ * Decode source frames via ffmpeg → run inference per frame → encode the RGBA
+ * stream via a second ffmpeg process. Output formats:
+ * .webm → VP9 with alpha (HTML5-native, ~1 MB / 4s @ 1080p)
+ * .mov → ProRes 4444 with alpha (editing round-trip)
+ * .png → single RGBA still (only when input is also a single image)
+ *
+ * The encode flags for VP9-with-alpha mirror the `chunkEncoder.ts` pattern in
+ * @hyperframes/engine — `-pix_fmt yuva420p` plus the
+ * `-metadata:s:v:0 alpha_mode=1` tag are what make Chrome's `` element
+ * decode the alpha plane.
+ */
+import { spawn } from "node:child_process";
+import { extname } from "node:path";
+import { hasFFmpeg, hasFFprobe } from "../whisper/manager.js";
+import { createSession, type Session } from "./inference.js";
+import { type Device, type ModelId } from "./manager.js";
+
+export type OutputFormat = "webm" | "mov" | "png";
+
+export const QUALITY_CRF = {
+ fast: 30,
+ balanced: 18,
+ best: 12,
+} as const;
+
+export type Quality = keyof typeof QUALITY_CRF;
+
+export const QUALITIES = Object.keys(QUALITY_CRF) as readonly Quality[];
+
+export const DEFAULT_QUALITY: Quality = "balanced";
+
+export const isQuality = (v: unknown): v is Quality =>
+ typeof v === "string" && (QUALITIES as readonly string[]).includes(v);
+
+export interface RenderOptions {
+ inputPath: string;
+ outputPath: string;
+ /**
+ * Optional second output: an inverse-alpha background plate (same source
+ * RGB, transparent where the subject was). Only valid for video inputs and
+ * .webm/.mov outputs — not allowed alongside a .png output. The plate's
+ * format is inferred from this path independently of the foreground's.
+ *
+ * NOTE: this is a hole-cut plate, not an inpainted clean plate. Composite
+ * something opaque (graphics, blur, scene) under it to fill the hole.
+ */
+ backgroundOutputPath?: string;
+ device?: Device;
+ model?: ModelId;
+ /** Encoder CRF preset for `.webm`. See `QUALITY_CRF`. Ignored for `.mov`/`.png`. */
+ quality?: Quality;
+ onProgress?: (event: ProgressEvent) => void;
+}
+
+export type ProgressEvent =
+ | { kind: "info"; message: string }
+ | { kind: "metadata"; width: number; height: number; fps: number; frameCount: number }
+ | { kind: "frame"; index: number; total: number; avgMsPerFrame: number };
+
+export interface RenderResult {
+ outputPath: string;
+ /** Present only when `backgroundOutputPath` was set. */
+ backgroundOutputPath?: string;
+ framesProcessed: number;
+ durationSeconds: number;
+ avgMsPerFrame: number;
+ provider: string;
+ format: OutputFormat;
+}
+
+const VIDEO_EXTENSIONS = new Set([".mp4", ".mov", ".webm", ".mkv", ".avi"]);
+const IMAGE_EXTENSIONS = new Set([".jpg", ".jpeg", ".png", ".webp"]);
+
+interface MediaInfo {
+ width: number;
+ height: number;
+ fps: number;
+ frameCount: number;
+}
+
+export function inferOutputFormat(outputPath: string): OutputFormat {
+ const ext = extname(outputPath).toLowerCase();
+ if (ext === ".webm") return "webm";
+ if (ext === ".mov") return "mov";
+ if (ext === ".png") return "png";
+ throw new Error(
+ `Unsupported output extension: ${ext}. Use .webm (VP9 alpha), .mov (ProRes 4444), or .png.`,
+ );
+}
+
+export function inferInputKind(inputPath: string): "video" | "image" {
+ const ext = extname(inputPath).toLowerCase();
+ if (VIDEO_EXTENSIONS.has(ext)) return "video";
+ if (IMAGE_EXTENSIONS.has(ext)) return "image";
+ throw new Error(
+ `Unsupported input: ${ext}. Use a video (mp4/mov/webm/mkv/avi) or image (jpg/png/webp).`,
+ );
+}
+
+interface EngineMetadata {
+ width: number;
+ height: number;
+ fps: number;
+ durationSeconds: number;
+}
+
+async function probeMedia(inputPath: string): Promise {
+ const isImage = inferInputKind(inputPath) === "image";
+ const engine = (await import("@hyperframes/engine")) as {
+ extractMediaMetadata: (path: string) => Promise;
+ };
+ const meta = await engine.extractMediaMetadata(inputPath);
+
+ if (isImage) {
+ return { width: meta.width, height: meta.height, fps: 0, frameCount: 1 };
+ }
+
+ const fps = meta.fps || 30;
+ const frameCount = meta.durationSeconds ? Math.round(meta.durationSeconds * fps) : 0;
+ return { width: meta.width, height: meta.height, fps, frameCount };
+}
+
+export function buildEncoderArgs(
+ format: OutputFormat,
+ width: number,
+ height: number,
+ fps: number,
+ outputPath: string,
+ quality: Quality = DEFAULT_QUALITY,
+): string[] {
+ const base = [
+ "-y",
+ "-f",
+ "rawvideo",
+ "-pix_fmt",
+ "rgba",
+ "-s",
+ `${width}x${height}`,
+ "-r",
+ String(fps || 30),
+ "-i",
+ "-",
+ ];
+
+ if (format === "webm") {
+ return [
+ ...base,
+ "-c:v",
+ "libvpx-vp9",
+ "-b:v",
+ "0",
+ "-crf",
+ String(QUALITY_CRF[quality]),
+ "-deadline",
+ "good",
+ "-row-mt",
+ "1",
+ "-auto-alt-ref",
+ "0",
+ "-pix_fmt",
+ "yuva420p",
+ // Tag the output as BT.709 limited range so browsers use the same
+ // YUV→RGB matrix the source video was encoded with. Without these tags
+ // ffmpeg's default RGB→YUV conversion is BT.601, which causes a visible
+ // color shift (red/skin tones in particular) when the matted overlay is
+ // composited over the original mp4.
+ "-colorspace",
+ "bt709",
+ "-color_primaries",
+ "bt709",
+ "-color_trc",
+ "bt709",
+ "-color_range",
+ "tv",
+ "-metadata:s:v:0",
+ "alpha_mode=1",
+ "-an",
+ outputPath,
+ ];
+ }
+ if (format === "mov") {
+ return [
+ ...base,
+ "-c:v",
+ "prores_ks",
+ "-profile:v",
+ "4444",
+ "-vendor",
+ "apl0",
+ "-pix_fmt",
+ "yuva444p10le",
+ "-an",
+ outputPath,
+ ];
+ }
+ return [...base, "-frames:v", "1", "-pix_fmt", "rgba", "-update", "1", outputPath];
+}
+
+async function* readFrames(
+ stream: NodeJS.ReadableStream,
+ frameBytes: number,
+): AsyncGenerator {
+ let buffered: Buffer = Buffer.alloc(0);
+ for await (const chunk of stream) {
+ buffered =
+ buffered.length === 0 ? (chunk as Buffer) : Buffer.concat([buffered, chunk as Buffer]);
+ while (buffered.length >= frameBytes) {
+ // Copy because the next concat would clobber the underlying memory.
+ yield Buffer.from(buffered.subarray(0, frameBytes));
+ buffered = buffered.subarray(frameBytes);
+ }
+ }
+}
+
+export interface RenderTargets {
+ format: OutputFormat;
+ inputKind: "video" | "image";
+ bgFormat: OutputFormat | undefined;
+}
+
+/**
+ * Resolve and validate the input/output combination before any I/O. Pure;
+ * exported so unit tests can pin the error messages without spawning ffmpeg.
+ */
+export function resolveRenderTargets(
+ inputPath: string,
+ outputPath: string,
+ backgroundOutputPath?: string,
+): RenderTargets {
+ const format = inferOutputFormat(outputPath);
+ const inputKind = inferInputKind(inputPath);
+
+ if (inputKind === "image" && format !== "png") {
+ throw new Error(
+ `Image input requires a .png output (got ${extname(outputPath)}). Use a video input for .webm/.mov.`,
+ );
+ }
+ if (inputKind === "video" && format === "png") {
+ throw new Error(
+ `Video input requires a .webm or .mov output (got .png). Use an image input for .png.`,
+ );
+ }
+
+ let bgFormat: OutputFormat | undefined;
+ if (backgroundOutputPath) {
+ if (inputKind === "image") {
+ throw new Error(
+ "--background-output is not supported for image inputs. Use a video input (mp4/mov/webm) to produce both a cutout and a background plate.",
+ );
+ }
+ bgFormat = inferOutputFormat(backgroundOutputPath);
+ if (bgFormat === "png") {
+ throw new Error(
+ "--background-output must be .webm or .mov; .png is only valid for single-image inputs.",
+ );
+ }
+ }
+
+ return { format, inputKind, bgFormat };
+}
+
+export async function render(options: RenderOptions): Promise {
+ if (!hasFFmpeg() || !hasFFprobe()) {
+ throw new Error("ffmpeg and ffprobe are required. Install: brew install ffmpeg");
+ }
+
+ const { format, bgFormat } = resolveRenderTargets(
+ options.inputPath,
+ options.outputPath,
+ options.backgroundOutputPath,
+ );
+
+ const media = await probeMedia(options.inputPath);
+
+ options.onProgress?.({
+ kind: "metadata",
+ width: media.width,
+ height: media.height,
+ fps: media.fps,
+ frameCount: media.frameCount,
+ });
+
+ const session = await createSession({
+ model: options.model,
+ device: options.device,
+ onProgress: (msg) => options.onProgress?.({ kind: "info", message: msg }),
+ });
+
+ try {
+ const start = Date.now();
+ const framesProcessed = await runPipeline(options, session, media, format, bgFormat);
+ const durationSeconds = (Date.now() - start) / 1000;
+ const avgMsPerFrame = framesProcessed ? (durationSeconds * 1000) / framesProcessed : 0;
+
+ return {
+ outputPath: options.outputPath,
+ backgroundOutputPath: options.backgroundOutputPath,
+ framesProcessed,
+ durationSeconds,
+ avgMsPerFrame,
+ provider: session.provider,
+ format,
+ };
+ } finally {
+ await session.close();
+ }
+}
+
+const RECENT_WINDOW = 30;
+
+interface FfmpegProc {
+ proc: ReturnType;
+ exit: Promise;
+ /** Tail of stderr, captured for inclusion in error messages. */
+ getStderr: () => string;
+}
+
+type StdioFd = "ignore" | "pipe";
+type StdioTuple = [StdioFd, StdioFd, StdioFd];
+
+function spawnFfmpeg(args: string[], label: string, stdio: StdioTuple): FfmpegProc {
+ const proc = spawn("ffmpeg", args, { stdio });
+ let stderrBuf = "";
+ proc.stderr?.on("data", (d: Buffer) => {
+ stderrBuf += d.toString();
+ });
+ // If the encoder dies mid-render, the next .write() to its stdin emits an
+ // 'error' event on the writable. Without a listener, Node treats it as
+ // unhandled and crashes the CLI before waitForExit's reject path can
+ // surface the real cause (encoder stderr tail). Swallowing here is safe —
+ // the process exit is the source of truth.
+ proc.stdin?.on("error", () => {});
+ const exit = waitForExit(proc, label, () => stderrBuf);
+ return { proc, exit, getStderr: () => stderrBuf };
+}
+
+async function runPipeline(
+ options: RenderOptions,
+ session: Session,
+ media: MediaInfo,
+ format: OutputFormat,
+ bgFormat: OutputFormat | undefined,
+): Promise {
+ const { inputPath, outputPath, backgroundOutputPath } = options;
+ const { width, height, fps, frameCount } = media;
+ const frameBytes = width * height * 3;
+ const quality = options.quality ?? DEFAULT_QUALITY;
+
+ const decoder = spawnFfmpeg(
+ ["-loglevel", "error", "-i", inputPath, "-f", "rawvideo", "-pix_fmt", "rgb24", "-an", "-"],
+ "ffmpeg decoder",
+ ["ignore", "pipe", "pipe"],
+ );
+
+ const fg = spawnFfmpeg(
+ buildEncoderArgs(format, width, height, fps || 30, outputPath, quality),
+ "ffmpeg encoder",
+ ["pipe", "ignore", "pipe"],
+ );
+
+ const bg =
+ backgroundOutputPath && bgFormat
+ ? spawnFfmpeg(
+ buildEncoderArgs(bgFormat, width, height, fps || 30, backgroundOutputPath, quality),
+ "ffmpeg background encoder",
+ ["pipe", "ignore", "pipe"],
+ )
+ : null;
+
+ let processed = 0;
+ const total = frameCount;
+
+ const recentMs = new Array(RECENT_WINDOW).fill(0);
+ let recentSum = 0;
+ let recentSlot = 0;
+ let recentCount = 0;
+
+ try {
+ for await (const rgb of readFrames(decoder.proc.stdout!, frameBytes)) {
+ const t0 = Date.now();
+ const result = await session.process(rgb, width, height, bg !== null);
+ const elapsed = Date.now() - t0;
+
+ recentSum += elapsed - recentMs[recentSlot]!;
+ recentMs[recentSlot] = elapsed;
+ recentSlot = (recentSlot + 1) % RECENT_WINDOW;
+ if (recentCount < RECENT_WINDOW) recentCount++;
+
+ // Issue both writes before any await so a slow encoder doesn't block
+ // the other. Drain anything that returned false before the next
+ // session.process() — its output buffers are reused per frame.
+ //
+ // Subtlety: write() returning true means "highWaterMark not exceeded,"
+ // NOT "libuv has flushed the chunk." The buffer reference is held by
+ // libuv until the underlying syscall completes. Reusing the session's
+ // output buffer is safe because the next session.process() call takes
+ // ~10–50ms (ORT inference) — plenty of event-loop turns for libuv to
+ // drain. If that ever stops being true, we'd need to copy here.
+ const fgWroteFully = fg.proc.stdin!.write(result.fg);
+ const bgWroteFully = bg && result.bg ? bg.proc.stdin!.write(result.bg) : true;
+ if (!fgWroteFully || !bgWroteFully) {
+ const drains: Promise[] = [];
+ if (!fgWroteFully) {
+ drains.push(
+ new Promise((resolve) => fg.proc.stdin!.once("drain", () => resolve())),
+ );
+ }
+ if (!bgWroteFully && bg) {
+ drains.push(
+ new Promise((resolve) => bg.proc.stdin!.once("drain", () => resolve())),
+ );
+ }
+ await Promise.all(drains);
+ }
+
+ processed++;
+ options.onProgress?.({
+ kind: "frame",
+ index: processed,
+ total,
+ avgMsPerFrame: recentSum / recentCount,
+ });
+ }
+ } catch (err) {
+ decoder.proc.kill("SIGKILL");
+ fg.proc.kill("SIGKILL");
+ bg?.proc.kill("SIGKILL");
+ throw err;
+ }
+
+ fg.proc.stdin!.end();
+ bg?.proc.stdin!.end();
+ const exits: Promise[] = [decoder.exit, fg.exit];
+ if (bg) exits.push(bg.exit);
+ await Promise.all(exits);
+
+ if (processed === 0) {
+ throw new Error(
+ `No frames produced from ${inputPath}. Decoder stderr:\n${decoder.getStderr().slice(-400)}`,
+ );
+ }
+
+ return processed;
+}
+
+export function waitForExit(
+ proc: ReturnType,
+ label: string,
+ getStderr: () => string,
+): Promise {
+ return new Promise((resolve, reject) => {
+ proc.on("error", reject);
+ // Per Node docs the exit callback is (code, signal): on a normal exit
+ // `code` is the numeric exit status and `signal` is null; on a
+ // signal-killed exit `code` is null and `signal` is the signal name.
+ // Treating null-code as success would silently report SIGTERM/SIGKILL
+ // as a successful render.
+ proc.on("exit", (code, signal) => {
+ if (code === 0 && !signal) {
+ resolve();
+ return;
+ }
+ const cause = signal ? `killed by ${signal}` : `exited with code ${code}`;
+ reject(new Error(`${label} ${cause}: ${getStderr().slice(-400)}`));
+ });
+ });
+}
diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts
index 64f6c6bfc..5bf2a2a4c 100644
--- a/packages/cli/src/cli.ts
+++ b/packages/cli/src/cli.ts
@@ -38,6 +38,7 @@ const subCommands = {
compositions: () => import("./commands/compositions.js").then((m) => m.default),
benchmark: () => import("./commands/benchmark.js").then((m) => m.default),
browser: () => import("./commands/browser.js").then((m) => m.default),
+ "remove-background": () => import("./commands/remove-background.js").then((m) => m.default),
transcribe: () => import("./commands/transcribe.js").then((m) => m.default),
tts: () => import("./commands/tts.js").then((m) => m.default),
"el-tts": () => import("./commands/el-tts.js").then((m) => m.default),
diff --git a/packages/cli/src/commands/add.test.ts b/packages/cli/src/commands/add.test.ts
index 87b82f525..6836b20d0 100644
--- a/packages/cli/src/commands/add.test.ts
+++ b/packages/cli/src/commands/add.test.ts
@@ -181,9 +181,9 @@ describe("runAdd (integration, mocked registry)", () => {
expect(result.type).toBe("hyperframes:block");
expect(result.written).toHaveLength(1);
expect(existsSync(join(dir, "compositions/my-block.html"))).toBe(true);
- expect(readFileSync(join(dir, "compositions/my-block.html"), "utf-8")).toContain(
- "my-block.html",
- );
+ const installed = readFileSync(join(dir, "compositions/my-block.html"), "utf-8");
+ expect(installed).toContain("");
+ expect(installed).toContain("my-block.html");
expect(result.snippet).toContain("compositions/my-block.html");
} finally {
rmSync(dir, { recursive: true, force: true });
diff --git a/packages/cli/src/commands/compositions.test.ts b/packages/cli/src/commands/compositions.test.ts
new file mode 100644
index 000000000..50c4456ff
--- /dev/null
+++ b/packages/cli/src/commands/compositions.test.ts
@@ -0,0 +1,49 @@
+import { describe, expect, it, beforeEach } from "vitest";
+import { ensureDOMParser } from "../utils/dom.js";
+import { parseSubComposition } from "./compositions.js";
+
+describe("parseSubComposition", () => {
+ beforeEach(() => {
+ ensureDOMParser();
+ });
+
+ it("reads template-wrapped sub-composition contents", () => {
+ const html = `
+
+
+ `;
+
+ expect(parseSubComposition(html, "foo", 1280, 720)).toEqual({
+ id: "foo",
+ duration: 1.5,
+ width: 1920,
+ height: 1080,
+ elementCount: 1,
+ });
+ });
+
+ it("counts visual-only template content and estimates simple script durations", () => {
+ const html = `
+
+
+ `;
+
+ expect(parseSubComposition(html, "foo", 1280, 720)).toEqual({
+ id: "foo",
+ duration: 0.5,
+ width: 1920,
+ height: 1080,
+ elementCount: 1,
+ });
+ });
+});
diff --git a/packages/cli/src/commands/compositions.ts b/packages/cli/src/commands/compositions.ts
index 302ed303e..4dfee7284 100644
--- a/packages/cli/src/commands/compositions.ts
+++ b/packages/cli/src/commands/compositions.ts
@@ -21,7 +21,31 @@ interface CompositionInfo {
source?: string;
}
-function parseCompositions(html: string, baseDir: string): CompositionInfo[] {
+const NON_RENDERED_TAGS = new Set(["script", "style", "link", "meta", "template"]);
+
+function countRenderableDescendants(root: Element): number {
+ return Array.from(root.querySelectorAll("*")).filter(
+ (el) => !NON_RENDERED_TAGS.has(el.tagName.toLowerCase()),
+ ).length;
+}
+
+function estimateDurationFromScripts(root: ParentNode): number {
+ let duration = 0;
+ for (const script of Array.from(root.querySelectorAll("script"))) {
+ const content = script.textContent ?? "";
+ const durationPattern = /\bduration\s*:\s*(\d+(?:\.\d+)?)/g;
+ let match: RegExpExecArray | null;
+ while ((match = durationPattern.exec(content)) !== null) {
+ const value = Number.parseFloat(match[1] ?? "");
+ if (Number.isFinite(value) && value > duration) {
+ duration = value;
+ }
+ }
+ }
+ return duration;
+}
+
+export function parseCompositions(html: string, baseDir: string): CompositionInfo[] {
const parser = new DOMParser();
const doc = parser.parseFromString(html, "text/html");
@@ -81,7 +105,7 @@ function parseCompositions(html: string, baseDir: string): CompositionInfo[] {
return compositions;
}
-function parseSubComposition(
+export function parseSubComposition(
html: string,
fallbackId: string,
fallbackWidth: number,
@@ -89,20 +113,23 @@ function parseSubComposition(
): CompositionInfo {
const parser = new DOMParser();
const doc = parser.parseFromString(html, "text/html");
+ const template = doc.querySelector("template");
+ const searchDocument = template?.content ?? doc;
// Sub-compositions may use wrappers or direct divs
- const compDiv =
- doc.querySelector("[data-composition-id]") ??
- doc.querySelector("template [data-composition-id]");
+ const compDiv = searchDocument.querySelector("[data-composition-id]");
const id = compDiv?.getAttribute("data-composition-id") ?? fallbackId;
const width = parseInt(compDiv?.getAttribute("data-width") ?? String(fallbackWidth), 10);
const height = parseInt(compDiv?.getAttribute("data-height") ?? String(fallbackHeight), 10);
// Count timed elements inside the sub-composition
- const searchRoot = compDiv ?? doc;
+ const searchRoot = compDiv ?? searchDocument;
const timedChildren = searchRoot.querySelectorAll("[data-start], .clip, .caption-group");
let elementCount = timedChildren.length;
+ if (elementCount === 0 && compDiv) {
+ elementCount = countRenderableDescendants(compDiv);
+ }
// Parse duration from the composition's own data-duration attribute
let duration = 0;
@@ -133,6 +160,9 @@ function parseSubComposition(
}
});
}
+ if (duration <= 0) {
+ duration = estimateDurationFromScripts(searchRoot);
+ }
return { id, duration, width, height, elementCount };
}
diff --git a/packages/cli/src/commands/contrast-audit.browser.js b/packages/cli/src/commands/contrast-audit.browser.js
index db506cc14..d1f5fe147 100644
--- a/packages/cli/src/commands/contrast-audit.browser.js
+++ b/packages/cli/src/commands/contrast-audit.browser.js
@@ -90,6 +90,7 @@ window.__contrastAudit = async function (imgBase64, time) {
if (parseFloat(cs.opacity) <= 0.01) continue;
var rect = el.getBoundingClientRect();
if (rect.width < 8 || rect.height < 8) continue;
+ if (rect.right <= 0 || rect.bottom <= 0 || rect.left >= w || rect.top >= h) continue;
var fg = parseColor(cs.color);
if (fg[3] <= 0.01) continue;
@@ -103,6 +104,7 @@ window.__contrastAudit = async function (imgBase64, time) {
var y0 = Math.max(0, Math.floor(rect.y) - 4);
var y1 = Math.min(h - 1, Math.ceil(rect.y + rect.height) + 4);
var sample = function (sx, sy) {
+ if (sx < 0 || sx >= w || sy < 0 || sy >= h) return;
var idx = (sy * w + sx) * 4;
rr.push(px[idx]);
gg.push(px[idx + 1]);
diff --git a/packages/cli/src/commands/init.test.ts b/packages/cli/src/commands/init.test.ts
index 132958fa1..1e905509b 100644
--- a/packages/cli/src/commands/init.test.ts
+++ b/packages/cli/src/commands/init.test.ts
@@ -1,11 +1,14 @@
import { describe, expect, it } from "vitest";
import { spawnSync } from "node:child_process";
-import { existsSync, mkdtempSync, rmSync } from "node:fs";
+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
+import { injectTailwindBrowserScript } from "./init.js";
const cliEntry = resolve(fileURLToPath(import.meta.url), "..", "..", "cli.ts");
+const tailwindScript =
+ '';
// Spawns `bun` directly because the CLI entry is a .ts file that needs a
// TypeScript-aware runtime. vitest runs under node, so `process.execPath`
@@ -24,18 +27,111 @@ function runInit(args: string[]): { status: number; stdout: string; stderr: stri
}
describe("hyperframes init flag rename", () => {
- it("--example blank scaffolds a bundled project", () => {
+ it("--example blank scaffolds a bundled project with npm scripts", () => {
const dir = mkdtempSync(join(tmpdir(), "hf-init-test-"));
const target = join(dir, "proj");
try {
const res = runInit([target, "--example", "blank", "--non-interactive", "--skip-skills"]);
expect(res.status).toBe(0);
expect(existsSync(join(target, "index.html"))).toBe(true);
+ expect(res.stdout).toContain("npm run dev");
+ expect(res.stdout).toContain("npm run check");
+ expect(res.stdout).toContain("npm run render");
+
+ const pkg = JSON.parse(readFileSync(join(target, "package.json"), "utf-8")) as {
+ private?: boolean;
+ type?: string;
+ scripts?: Record;
+ };
+ expect(pkg.private).toBe(true);
+ expect(pkg.type).toBe("module");
+ expect(pkg.scripts).toMatchObject({
+ dev: "npx --yes hyperframes preview",
+ check:
+ "npx --yes hyperframes lint && npx --yes hyperframes validate && npx --yes hyperframes inspect",
+ render: "npx --yes hyperframes render",
+ publish: "npx --yes hyperframes publish",
+ });
+ expect(Object.keys(pkg.scripts ?? {}).sort()).toEqual(["check", "dev", "publish", "render"]);
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
+
+ it("--tailwind enables Tailwind utilities in scaffolded HTML", () => {
+ const dir = mkdtempSync(join(tmpdir(), "hf-init-test-"));
+ const target = join(dir, "proj");
+ try {
+ const res = runInit([
+ target,
+ "--example",
+ "blank",
+ "--tailwind",
+ "--non-interactive",
+ "--skip-skills",
+ ]);
+ expect(res.status).toBe(0);
+
+ const html = readFileSync(join(target, "index.html"), "utf-8");
+ expect(html).toContain(tailwindScript);
+ expect(html).toContain("window.__tailwindReady");
+
+ const pkg = JSON.parse(readFileSync(join(target, "package.json"), "utf-8")) as {
+ scripts?: Record;
+ };
+ expect(pkg.scripts).toMatchObject({
+ dev: "npx --yes hyperframes preview",
+ check:
+ "npx --yes hyperframes lint && npx --yes hyperframes validate && npx --yes hyperframes inspect",
+ render: "npx --yes hyperframes render",
+ publish: "npx --yes hyperframes publish",
+ });
+ expect(Object.keys(pkg.scripts ?? {}).sort()).toEqual(["check", "dev", "publish", "render"]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
+ it("inserts Tailwind before uppercase closing head tags", () => {
+ const html = [
+ "",
+ "",
+ "",
+ ' ',
+ "",
+ "",
+ ].join("\n");
+
+ const injected = injectTailwindBrowserScript(html);
+ expect(injected.indexOf(' ')).toBeLessThan(
+ injected.indexOf(tailwindScript),
+ );
+ expect(injected.indexOf(tailwindScript)).toBeLessThan(injected.indexOf(""));
+ });
+
+ it("inserts Tailwind into single-line HTML heads", () => {
+ const html = "x ";
+
+ expect(injectTailwindBrowserScript(html)).toContain(`${tailwindScript}\n`);
+ });
+
+ it("does not duplicate Tailwind support when it is already present", () => {
+ const html = ["", "", "", tailwindScript, "", ""].join(
+ "\n",
+ );
+
+ expect(injectTailwindBrowserScript(html)).toBe(html);
+ });
+
+ it("keeps the readiness shim free of render-loop APIs", () => {
+ const html = "";
+ const injected = injectTailwindBrowserScript(html);
+
+ expect(injected).not.toContain("Date.now");
+ expect(injected).not.toContain("requestAnimationFrame");
+ expect(injected).not.toContain("setTimeout");
+ });
+
it("--template prints a rename hint and exits non-zero", () => {
const dir = mkdtempSync(join(tmpdir(), "hf-init-test-"));
const target = join(dir, "proj");
diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts
index 43a0d5f48..798d20763 100644
--- a/packages/cli/src/commands/init.ts
+++ b/packages/cli/src/commands/init.ts
@@ -6,6 +6,7 @@ export const examples: Example[] = [
["Pick a starter example", "hyperframes init my-video --example warm-grain"],
["Start from an existing video file", "hyperframes init my-video --video clip.mp4"],
["Start from an audio file", "hyperframes init my-video --audio track.mp3"],
+ ["Scaffold with Tailwind CSS", "hyperframes init my-video --example blank --tailwind"],
["Non-interactive mode (for CI or AI agents)", "hyperframes init my-video --non-interactive"],
["Skip AI coding skills installation", "hyperframes init my-video --skip-skills"],
];
@@ -32,6 +33,7 @@ import {
import { fetchRemoteTemplate } from "../templates/remote.js";
import { trackInitTemplate } from "../telemetry/events.js";
import { hasFFmpeg } from "../whisper/manager.js";
+import { VERSION } from "../version.js";
interface VideoMeta {
durationSeconds: number;
@@ -53,6 +55,13 @@ const DEFAULT_META: VideoMeta = {
videoCodec: "h264",
};
+// Pin the browser runtime exactly so repeated renders do not drift as Tailwind
+// ships JIT/preflight changes on the CDN.
+const TAILWIND_BROWSER_VERSION = "4.2.4";
+const TAILWIND_BROWSER_SRC = `https://cdn.jsdelivr.net/npm/@tailwindcss/browser@${TAILWIND_BROWSER_VERSION}/dist/index.global.js`;
+const TAILWIND_BROWSER_INTEGRITY =
+ "sha384-v5YF9xS+gLRWdvrQ0u/WRbCkjSIH0NjHIPe8tBL1ZRrmI7PiSH6LLdzs0aAIMCuh";
+
// ---------------------------------------------------------------------------
// ffprobe helper — shells out to ffprobe to avoid engine dependency
// ---------------------------------------------------------------------------
@@ -168,6 +177,111 @@ function getSharedTemplateDir(): string {
return resolveAssetDir(["..", "templates", "_shared"], ["templates", "_shared"]);
}
+function toPackageName(projectName: string): string {
+ const normalized = basename(projectName)
+ .trim()
+ .toLowerCase()
+ .replace(/^[._]+/, "")
+ .replace(/[^a-z0-9._~-]+/g, "-")
+ .replace(/-+/g, "-")
+ .replace(/^[-.]+|[-.]+$/g, "");
+
+ return normalized || "hyperframes-project";
+}
+
+function getHyperframesPackageSpecifier(): string {
+ return VERSION === "0.0.0-dev" ? "hyperframes" : `hyperframes@${VERSION}`;
+}
+
+function hyperframesScript(command: string): string {
+ return `npx --yes ${getHyperframesPackageSpecifier()} ${command}`;
+}
+
+function buildPackageScripts(): Record {
+ return {
+ dev: hyperframesScript("preview"),
+ check:
+ `${hyperframesScript("lint")} && ${hyperframesScript("validate")} && ` +
+ `${hyperframesScript("inspect")}`,
+ render: hyperframesScript("render"),
+ publish: hyperframesScript("publish"),
+ };
+}
+
+function writeDefaultPackageJson(destDir: string, projectName: string): void {
+ const packageJsonPath = resolve(destDir, "package.json");
+ if (existsSync(packageJsonPath)) return;
+
+ writeFileSync(
+ packageJsonPath,
+ `${JSON.stringify(
+ {
+ name: toPackageName(projectName),
+ private: true,
+ type: "module",
+ scripts: buildPackageScripts(),
+ },
+ null,
+ 2,
+ )}\n`,
+ "utf-8",
+ );
+}
+
+function listHtmlFiles(dir: string): string[] {
+ const files: string[] = [];
+ const ignoredDirs = new Set([".git", "dist", "node_modules"]);
+
+ function walk(currentDir: string): void {
+ for (const entry of readdirSync(currentDir, { withFileTypes: true })) {
+ const entryPath = join(currentDir, entry.name);
+ if (entry.isDirectory()) {
+ if (!ignoredDirs.has(entry.name)) walk(entryPath);
+ continue;
+ }
+ if (entry.isFile() && entry.name.endsWith(".html")) {
+ files.push(entryPath);
+ }
+ }
+ }
+
+ walk(dir);
+ return files;
+}
+
+export function injectTailwindBrowserScript(html: string): string {
+ if (html.includes(TAILWIND_BROWSER_SRC)) return html;
+
+ const script = [
+ ``,
+ ``,
+ ].join("\n");
+
+ if (/<\/head>/i.test(html)) {
+ return html.replace(/<\/head>/i, (closingHead) => `\n${script}\n${closingHead}`);
+ }
+
+ return `${script}\n${html}`;
+}
+
+function writeTailwindSupport(destDir: string): void {
+ for (const file of listHtmlFiles(destDir)) {
+ const html = readFileSync(file, "utf-8");
+ writeFileSync(file, injectTailwindBrowserScript(html), "utf-8");
+ }
+}
+
function patchVideoSrc(
dir: string,
videoFilename: string | undefined,
@@ -312,6 +426,7 @@ async function scaffoldProject(
templateId: string,
localVideoName: string | undefined,
durationSeconds?: number,
+ tailwind = false,
): Promise {
mkdirSync(destDir, { recursive: true });
@@ -323,6 +438,7 @@ async function scaffoldProject(
await fetchRemoteTemplate(templateId, destDir);
}
patchVideoSrc(destDir, localVideoName, durationSeconds);
+ if (tailwind) writeTailwindSupport(destDir);
writeFileSync(
resolve(destDir, "meta.json"),
@@ -346,6 +462,8 @@ async function scaffoldProject(
writeProjectConfig(destDir, DEFAULT_PROJECT_CONFIG);
}
+ writeDefaultPackageJson(destDir, name);
+
// Copy shared files (CLAUDE.md, AGENTS.md) for AI agent context
const sharedDir = getSharedTemplateDir();
if (existsSync(sharedDir)) {
@@ -418,6 +536,10 @@ export default defineCommand({
type: "boolean",
description: "Skip AI coding skills installation",
},
+ tailwind: {
+ type: "boolean",
+ description: "Add Tailwind CSS browser-runtime support",
+ },
},
async run({ args }) {
if (args.template !== undefined) {
@@ -435,6 +557,7 @@ export default defineCommand({
const audioFlag = args.audio;
const skipTranscribe = args["skip-transcribe"] === true;
const skipSkills = args["skip-skills"] === true;
+ const tailwind = args.tailwind === true;
const nonInteractive = args["non-interactive"] === true;
const modelFlag = args.model;
const languageFlag = args.language;
@@ -521,6 +644,7 @@ export default defineCommand({
templateId,
localVideoName,
videoDuration,
+ tailwind,
);
} catch (err) {
console.error(
@@ -531,7 +655,7 @@ export default defineCommand({
console.error(c.dim("Use --example blank for offline use."));
process.exit(1);
}
- trackInitTemplate(templateId);
+ trackInitTemplate(templateId, { tailwind });
const transcriptFile = resolve(destDir, "transcript.json");
if (existsSync(transcriptFile)) {
await patchTranscript(destDir, transcriptFile);
@@ -559,10 +683,13 @@ export default defineCommand({
console.log(` ${c.dim("More patterns: hyperframes.heygen.com/guides/prompting")}`);
console.log();
console.log(` ${c.accent("4.")} Preview in the browser:`);
- console.log(` ${c.accent(`cd ${name}`)} && ${c.accent("npx hyperframes preview")}`);
+ console.log(` ${c.accent(`cd ${name}`)} && ${c.accent("npm run dev")}`);
+ console.log();
+ console.log(` ${c.accent("5.")} Check the composition:`);
+ console.log(` ${c.accent(`cd ${name}`)} && ${c.accent("npm run check")}`);
console.log();
- console.log(` ${c.accent("5.")} Render to MP4 when ready:`);
- console.log(` ${c.accent(`cd ${name}`)} && ${c.accent("npx hyperframes render")}`);
+ console.log(` ${c.accent("6.")} Render to MP4 when ready:`);
+ console.log(` ${c.accent(`cd ${name}`)} && ${c.accent("npm run render")}`);
console.log();
console.log(` ${c.dim("Full docs: hyperframes.heygen.com")}`);
return;
@@ -713,7 +840,7 @@ export default defineCommand({
spin.start(`Downloading example ${c.accent(templateId)}...`);
}
try {
- await scaffoldProject(destDir, name, templateId, localVideoName, videoDuration);
+ await scaffoldProject(destDir, name, templateId, localVideoName, videoDuration, tailwind);
if (!isBundled) {
spin.stop(c.success(`Downloaded ${templateId}`));
}
@@ -726,7 +853,7 @@ export default defineCommand({
);
process.exit(1);
}
- trackInitTemplate(templateId);
+ trackInitTemplate(templateId, { tailwind });
// 4b. Patch captions with transcript if available
const transcriptFile = resolve(destDir, "transcript.json");
diff --git a/packages/cli/src/commands/layout.ts b/packages/cli/src/commands/layout.ts
index 92cc0c94b..fa27dd70b 100644
--- a/packages/cli/src/commands/layout.ts
+++ b/packages/cli/src/commands/layout.ts
@@ -1,11 +1,11 @@
import { defineCommand } from "citty";
-import { createServer } from "node:http";
import { existsSync, readFileSync } from "node:fs";
-import { dirname, isAbsolute, join, relative, resolve } from "node:path";
+import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { Example } from "./_examples.js";
import { c } from "../ui/colors.js";
import { resolveProject } from "../utils/project.js";
+import { serveStaticProjectHtml } from "../utils/staticProjectServer.js";
import { withMeta } from "../utils/updateCheck.js";
import {
buildLayoutSampleTimes,
@@ -107,78 +107,10 @@ async function seekTo(page: import("puppeteer-core").Page, time: number): Promis
}
async function bundleProjectHtml(projectDir: string): Promise {
+ // `bundleToSingleHtml` now inlines the runtime IIFE by default, so the
+ // previous post-bundle runtime substitution is no longer needed.
const { bundleToSingleHtml } = await import("@hyperframes/core/compiler");
- let html = await bundleToSingleHtml(projectDir);
-
- const runtimePath = resolve(
- __dirname,
- "..",
- "..",
- "..",
- "core",
- "dist",
- "hyperframe.runtime.iife.js",
- );
- if (existsSync(runtimePath)) {
- const runtimeSource = readFileSync(runtimePath, "utf-8");
- html = html.replace(
- /]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
- () => ``,
- );
- }
-
- return html;
-}
-
-async function serveProject(
- projectDir: string,
- html: string,
-): Promise<{
- url: string;
- close: () => Promise;
-}> {
- const { getMimeType } = await import("@hyperframes/core/studio-api");
- const server = createServer((req, res) => {
- const url = req.url ?? "/";
- if (url === "/" || url === "/index.html") {
- res.writeHead(200, { "Content-Type": "text/html" });
- res.end(html);
- return;
- }
-
- const filePath = resolve(projectDir, decodeURIComponent(url).replace(/^\//, ""));
- const rel = relative(projectDir, filePath);
- if (rel.startsWith("..") || isAbsolute(rel)) {
- res.writeHead(403);
- res.end();
- return;
- }
- if (existsSync(filePath)) {
- res.writeHead(200, { "Content-Type": getMimeType(filePath) });
- res.end(readFileSync(filePath));
- return;
- }
- res.writeHead(404);
- res.end();
- });
-
- const port = await new Promise((resolvePort, rejectPort) => {
- server.on("error", rejectPort);
- server.listen(0, () => {
- const addr = server.address();
- const resolvedPort = typeof addr === "object" && addr ? addr.port : 0;
- if (!resolvedPort) rejectPort(new Error("Failed to bind local layout audit server"));
- else resolvePort(resolvedPort);
- });
- });
-
- return {
- url: `http://127.0.0.1:${port}/`,
- close: () =>
- new Promise((resolveClose) => {
- server.close(() => resolveClose());
- }),
- };
+ return bundleToSingleHtml(projectDir);
}
async function alignViewportToComposition(
@@ -206,7 +138,11 @@ async function runLayoutAudit(
const { ensureBrowser } = await import("../browser/manager.js");
const puppeteer = await import("puppeteer-core");
const html = await bundleProjectHtml(projectDir);
- const server = await serveProject(projectDir, html);
+ const server = await serveStaticProjectHtml(
+ projectDir,
+ html,
+ "Failed to bind local layout audit server",
+ );
let chromeBrowser: import("puppeteer-core").Browser | undefined;
try {
diff --git a/packages/cli/src/commands/remove-background.ts b/packages/cli/src/commands/remove-background.ts
new file mode 100644
index 000000000..1116bef74
--- /dev/null
+++ b/packages/cli/src/commands/remove-background.ts
@@ -0,0 +1,221 @@
+import { defineCommand } from "citty";
+import { resolve } from "node:path";
+import { existsSync } from "node:fs";
+import * as clack from "@clack/prompts";
+import { c } from "../ui/colors.js";
+import { isDevice, DEVICES } from "../background-removal/manager.js";
+import { DEFAULT_QUALITY, QUALITIES, isQuality } from "../background-removal/pipeline.js";
+import type { Example } from "./_examples.js";
+
+export const examples: Example[] = [
+ [
+ "Remove background from a video, output transparent VP9 WebM (default)",
+ "hyperframes remove-background avatar.mp4 -o transparent.webm",
+ ],
+ [
+ "Output ProRes 4444 .mov for editing round-trip",
+ "hyperframes remove-background avatar.mp4 -o transparent.mov",
+ ],
+ [
+ "Remove background from a single image, output transparent PNG",
+ "hyperframes remove-background portrait.jpg -o cutout.png",
+ ],
+ [
+ "Separate the layers — emit both the cutout and an inverse-alpha background plate (subject region transparent)",
+ "hyperframes remove-background avatar.mp4 -o subject.webm --background-output plate.webm",
+ ],
+ [
+ "Force CPU (skip CoreML/CUDA)",
+ "hyperframes remove-background avatar.mp4 -o transparent.webm --device cpu",
+ ],
+ [
+ "Smaller file at the cost of color match (text-behind-subject won't blend as cleanly)",
+ "hyperframes remove-background avatar.mp4 -o transparent.webm --quality fast",
+ ],
+ [
+ "Visually-lossless WebM (master / re-encode source)",
+ "hyperframes remove-background avatar.mp4 -o transparent.webm --quality best",
+ ],
+ ["Show detected providers without rendering", "hyperframes remove-background --info"],
+];
+
+export default defineCommand({
+ meta: {
+ name: "remove-background",
+ description:
+ "Remove background from a video or image using a local AI model — outputs transparent WebM, ProRes 4444, or PNG",
+ },
+ args: {
+ input: {
+ type: "positional",
+ description: "Source video (.mp4/.mov/.webm/.mkv) or image (.jpg/.png/.webp)",
+ required: false,
+ },
+ output: {
+ type: "string",
+ description: "Output path. Format inferred from extension: .webm (default), .mov, .png",
+ alias: "o",
+ },
+ "background-output": {
+ type: "string",
+ description:
+ "Optional second output path for the inverse-alpha background plate (subject region transparent, original surroundings opaque). Hole-cut, not inpainted — composite something underneath to fill the hole. Must be .webm or .mov; not allowed for image inputs.",
+ alias: "b",
+ },
+ device: {
+ type: "string",
+ description: `Execution provider: ${DEVICES.join(", ")}`,
+ default: "auto",
+ },
+ quality: {
+ type: "string",
+ description: `Encoder quality preset for .webm output: ${QUALITIES.join(", ")} (default: ${DEFAULT_QUALITY}). Higher quality = closer color match when overlaying on the source mp4, larger file. Ignored for .mov / .png.`,
+ default: DEFAULT_QUALITY,
+ },
+ info: {
+ type: "boolean",
+ description: "Print detected execution providers and exit (no render)",
+ default: false,
+ },
+ json: {
+ type: "boolean",
+ description: "Output result as JSON",
+ default: false,
+ },
+ },
+ async run({ args }) {
+ if (args.info) {
+ return showInfo(args.json);
+ }
+ if (!args.input) {
+ console.error(
+ c.error(
+ "Input file is required. Run `hyperframes remove-background --info` for providers.",
+ ),
+ );
+ process.exit(1);
+ }
+ if (!args.output) {
+ console.error(c.error("--output (-o) is required. Use a .webm, .mov, or .png path."));
+ process.exit(1);
+ }
+ if (!isDevice(args.device)) {
+ console.error(
+ c.error(`Invalid --device '${String(args.device)}'. Use: ${DEVICES.join(", ")}.`),
+ );
+ process.exit(1);
+ }
+ if (!isQuality(args.quality)) {
+ console.error(
+ c.error(`Invalid --quality '${String(args.quality)}'. Use: ${QUALITIES.join(", ")}.`),
+ );
+ process.exit(1);
+ }
+
+ const inputPath = resolve(args.input);
+ const outputPath = resolve(args.output);
+ const backgroundOutputArg = args["background-output"];
+ const backgroundOutputPath = backgroundOutputArg ? resolve(backgroundOutputArg) : undefined;
+
+ const { render } = await import("../background-removal/pipeline.js");
+
+ const spin = args.json ? null : clack.spinner();
+ spin?.start("Preparing background-removal pipeline...");
+
+ try {
+ const result = await render({
+ inputPath,
+ outputPath,
+ backgroundOutputPath,
+ device: args.device,
+ quality: args.quality,
+ onProgress: (event) => {
+ if (event.kind === "info") {
+ spin?.message(event.message);
+ } else if (event.kind === "metadata") {
+ const dims = `${event.width}×${event.height}`;
+ const frames = event.frameCount ? ` · ${event.frameCount} frames` : "";
+ spin?.message(`Source ${dims} @ ${event.fps.toFixed(0)}fps${frames}`);
+ } else if (event.kind === "frame") {
+ const pct = event.total ? ` (${Math.floor((100 * event.index) / event.total)}%)` : "";
+ spin?.message(
+ `Frame ${event.index}${event.total ? `/${event.total}` : ""}${pct} — ${Math.round(event.avgMsPerFrame)}ms/frame avg`,
+ );
+ }
+ },
+ });
+
+ if (args.json) {
+ console.log(
+ JSON.stringify({
+ ok: true,
+ outputPath: result.outputPath,
+ ...(result.backgroundOutputPath
+ ? { backgroundOutputPath: result.backgroundOutputPath }
+ : {}),
+ framesProcessed: result.framesProcessed,
+ durationSeconds: Number(result.durationSeconds.toFixed(2)),
+ avgMsPerFrame: Number(result.avgMsPerFrame.toFixed(1)),
+ provider: result.provider,
+ format: result.format,
+ }),
+ );
+ } else {
+ const fpsThroughput = result.durationSeconds
+ ? (result.framesProcessed / result.durationSeconds).toFixed(1)
+ : "n/a";
+ const outputs = result.backgroundOutputPath
+ ? `${c.accent(result.outputPath)} + ${c.accent(result.backgroundOutputPath)}`
+ : c.accent(result.outputPath);
+ spin?.stop(
+ c.success(
+ `Removed background from ${c.accent(String(result.framesProcessed))} frames in ${result.durationSeconds.toFixed(1)}s (${fpsThroughput} fps, ${c.accent(result.provider)}) → ${outputs}`,
+ ),
+ );
+ }
+ } catch (err) {
+ const message = err instanceof Error ? err.message : String(err);
+ if (args.json) {
+ console.log(JSON.stringify({ ok: false, error: message }));
+ } else {
+ spin?.stop(c.error(`Background removal failed: ${message}`));
+ }
+ process.exit(1);
+ }
+ },
+});
+
+async function showInfo(json: boolean): Promise {
+ const { selectProviders, listAvailableProviders, DEFAULT_MODEL, MODEL_MEMORY_MB, modelPath } =
+ await import("../background-removal/manager.js");
+
+ const providers = listAvailableProviders();
+ const auto = selectProviders("auto");
+ const cached = existsSync(modelPath());
+
+ if (json) {
+ console.log(
+ JSON.stringify({
+ defaultModel: DEFAULT_MODEL,
+ modelCached: cached,
+ modelPath: modelPath(),
+ peakMemoryMb: MODEL_MEMORY_MB[DEFAULT_MODEL],
+ availableProviders: providers,
+ autoProvider: auto.label,
+ }),
+ );
+ return;
+ }
+
+ console.log(c.bold("hyperframes remove-background — system info"));
+ console.log("");
+ console.log(` ${c.dim("Default model:")} ${c.accent(DEFAULT_MODEL)}`);
+ console.log(` ${c.dim("Peak memory:")} ~${MODEL_MEMORY_MB[DEFAULT_MODEL]} MB`);
+ console.log(
+ ` ${c.dim("Weights cached:")} ${cached ? c.success("yes") : c.dim("no (will download on first run)")}`,
+ );
+ console.log(` ${c.dim("Cache path:")} ${modelPath()}`);
+ console.log("");
+ console.log(` ${c.dim("Available providers:")} ${providers.join(", ")}`);
+ console.log(` ${c.dim("Auto-selected:")} ${c.accent(auto.label)}`);
+}
diff --git a/packages/cli/src/commands/render.test.ts b/packages/cli/src/commands/render.test.ts
new file mode 100644
index 000000000..fbf4c2c12
--- /dev/null
+++ b/packages/cli/src/commands/render.test.ts
@@ -0,0 +1,301 @@
+import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
+
+const producerState = vi.hoisted(() => ({
+ createdJobs: [] as Array>,
+ resolveConfigCalls: [] as Array>,
+}));
+
+vi.mock("../utils/producer.js", () => ({
+ loadProducer: vi.fn(async () => ({
+ resolveConfig: vi.fn((overrides: Record) => {
+ producerState.resolveConfigCalls.push(overrides);
+ return { ...overrides, resolved: true };
+ }),
+ createRenderJob: vi.fn((config: Record) => {
+ producerState.createdJobs.push(config);
+ return { config, progress: 100 };
+ }),
+ executeRenderJob: vi.fn(async () => undefined),
+ })),
+}));
+
+vi.mock("../telemetry/events.js", () => ({
+ trackRenderComplete: vi.fn(),
+ trackRenderError: vi.fn(),
+}));
+
+describe("renderLocal browser GPU config", () => {
+ const savedEnv = new Map();
+
+ function setEnv(key: string, value: string) {
+ savedEnv.set(key, process.env[key]);
+ process.env[key] = value;
+ }
+
+ beforeEach(() => {
+ producerState.createdJobs = [];
+ producerState.resolveConfigCalls = [];
+ savedEnv.clear();
+ });
+
+ afterEach(() => {
+ for (const [key, value] of savedEnv) {
+ if (value === undefined) {
+ delete process.env[key];
+ } else {
+ process.env[key] = value;
+ }
+ }
+ vi.clearAllMocks();
+ vi.useRealTimers();
+ vi.restoreAllMocks();
+ });
+
+ it("passes an explicit software override for --no-browser-gpu even when env requests hardware", async () => {
+ setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
+
+ const { renderLocal } = await import("./render.js");
+ await renderLocal("/tmp/project", "/tmp/out.mp4", {
+ fps: 30,
+ quality: "standard",
+ format: "mp4",
+ gpu: false,
+ browserGpu: false,
+ hdrMode: "auto",
+ quiet: true,
+ });
+
+ expect(producerState.resolveConfigCalls).toContainEqual({ browserGpuMode: "software" });
+ expect(producerState.createdJobs[0]?.producerConfig).toMatchObject({
+ browserGpuMode: "software",
+ resolved: true,
+ });
+ });
+
+ it("passes an explicit hardware override for default local browser GPU", async () => {
+ const { renderLocal } = await import("./render.js");
+ await renderLocal("/tmp/project", "/tmp/out.mp4", {
+ fps: 30,
+ quality: "standard",
+ format: "mp4",
+ gpu: false,
+ browserGpu: true,
+ hdrMode: "auto",
+ quiet: true,
+ });
+
+ expect(producerState.resolveConfigCalls).toContainEqual({ browserGpuMode: "hardware" });
+ expect(producerState.createdJobs[0]?.producerConfig).toMatchObject({
+ browserGpuMode: "hardware",
+ resolved: true,
+ });
+ });
+
+ it("resolves browser GPU from CLI flags, Docker mode, and env fallback", async () => {
+ const { resolveBrowserGpuForCli } = await import("./render.js");
+
+ expect(resolveBrowserGpuForCli(false, undefined, undefined)).toBe(true);
+ expect(resolveBrowserGpuForCli(false, undefined, "hardware")).toBe(true);
+ expect(resolveBrowserGpuForCli(false, undefined, "software")).toBe(false);
+ expect(resolveBrowserGpuForCli(false, true, "software")).toBe(true);
+ expect(resolveBrowserGpuForCli(false, false, "hardware")).toBe(false);
+ expect(resolveBrowserGpuForCli(true, undefined, "hardware")).toBe(false);
+ });
+
+ it("forwards parsed --variables payload to createRenderJob", async () => {
+ const { renderLocal } = await import("./render.js");
+ await renderLocal("/tmp/project", "/tmp/out.mp4", {
+ fps: 30,
+ quality: "standard",
+ format: "mp4",
+ gpu: false,
+ browserGpu: false,
+ hdrMode: "auto",
+ quiet: true,
+ variables: { title: "Hello", count: 3 },
+ });
+
+ expect(producerState.createdJobs[0]?.variables).toEqual({ title: "Hello", count: 3 });
+ });
+
+ it("omits variables from createRenderJob when not provided", async () => {
+ const { renderLocal } = await import("./render.js");
+ await renderLocal("/tmp/project", "/tmp/out.mp4", {
+ fps: 30,
+ quality: "standard",
+ format: "mp4",
+ gpu: false,
+ browserGpu: false,
+ hdrMode: "auto",
+ quiet: true,
+ });
+
+ expect(producerState.createdJobs[0]?.variables).toBeUndefined();
+ });
+
+ it("can force the CLI process to exit after a successful local render", async () => {
+ vi.useFakeTimers();
+ const exit = vi
+ .spyOn(process, "exit")
+ .mockImplementation((code?: string | number | null): never => {
+ throw new Error(`process.exit:${code ?? ""}`);
+ });
+ const { renderLocal } = await import("./render.js");
+
+ await renderLocal("/tmp/project", "/tmp/out.mp4", {
+ fps: 30,
+ quality: "standard",
+ format: "mp4",
+ gpu: false,
+ browserGpu: true,
+ hdrMode: "auto",
+ quiet: true,
+ exitAfterComplete: true,
+ });
+
+ expect(exit).not.toHaveBeenCalled();
+ expect(() => vi.advanceTimersByTime(100)).toThrow("process.exit:0");
+ expect(exit).toHaveBeenCalledWith(0);
+ });
+});
+
+describe("parseVariablesArg", () => {
+ let parseVariablesArg: typeof import("./render.js").parseVariablesArg;
+
+ beforeAll(async () => {
+ ({ parseVariablesArg } = await import("./render.js"));
+ });
+
+ function expectErr(
+ result: import("./render.js").VariablesParseResult,
+ ): T {
+ if (result.ok) throw new Error(`expected error, got ${JSON.stringify(result.value)}`);
+ return result.error as T;
+ }
+
+ it("returns undefined when neither flag is set", () => {
+ expect(parseVariablesArg(undefined, undefined)).toEqual({ ok: true, value: undefined });
+ });
+
+ it("parses inline JSON object", () => {
+ expect(parseVariablesArg('{"title":"Hello","n":3}', undefined)).toEqual({
+ ok: true,
+ value: { title: "Hello", n: 3 },
+ });
+ });
+
+ it("parses file JSON via injected reader", () => {
+ const fakeReader = (path: string) => {
+ if (path === "vars.json") return '{"theme":"dark"}';
+ throw new Error("unexpected path");
+ };
+ expect(parseVariablesArg(undefined, "vars.json", fakeReader)).toEqual({
+ ok: true,
+ value: { theme: "dark" },
+ });
+ });
+
+ it("rejects when both flags are set", () => {
+ const err = expectErr(parseVariablesArg('{"a":1}', "vars.json"));
+ expect(err).toEqual({ kind: "conflict" });
+ });
+
+ it("rejects unparseable JSON with a source-aware kind", () => {
+ expect(expectErr(parseVariablesArg("{not json", undefined))).toMatchObject({
+ kind: "parse-error",
+ source: "inline",
+ });
+ expect(expectErr(parseVariablesArg(undefined, "x", () => "{not json"))).toMatchObject({
+ kind: "parse-error",
+ source: "file",
+ });
+ });
+
+ it("rejects non-object payloads (array, string, null, number)", () => {
+ for (const payload of ["[1,2]", '"hello"', "null", "42"]) {
+ expect(expectErr(parseVariablesArg(payload, undefined))).toEqual({ kind: "shape-error" });
+ }
+ });
+
+ it("surfaces filesystem errors from --variables-file", () => {
+ const err = expectErr<{
+ kind: "read-error";
+ path: string;
+ cause: string;
+ }>(
+ parseVariablesArg(undefined, "missing.json", () => {
+ throw new Error("ENOENT: no such file");
+ }),
+ );
+ expect(err.kind).toBe("read-error");
+ expect(err.path).toBe("missing.json");
+ expect(err.cause).toMatch(/ENOENT/);
+ });
+});
+
+describe("validateVariablesAgainstProject", () => {
+ let validateVariablesAgainstProject: typeof import("./render.js").validateVariablesAgainstProject;
+ let tmpDir: string;
+ let mkdtempSync: typeof import("node:fs").mkdtempSync;
+ let writeFileSync: typeof import("node:fs").writeFileSync;
+ let rmSync: typeof import("node:fs").rmSync;
+ let join: typeof import("node:path").join;
+ let tmpdir: typeof import("node:os").tmpdir;
+
+ beforeAll(async () => {
+ ({ validateVariablesAgainstProject } = await import("./render.js"));
+ ({ mkdtempSync, writeFileSync, rmSync } = await import("node:fs"));
+ ({ join } = await import("node:path"));
+ ({ tmpdir } = await import("node:os"));
+ });
+
+ beforeEach(() => {
+ tmpDir = mkdtempSync(join(tmpdir(), "hf-validate-vars-"));
+ });
+
+ afterEach(() => {
+ rmSync(tmpDir, { recursive: true, force: true });
+ });
+
+ function writeIndex(html: string): string {
+ const path = join(tmpDir, "index.html");
+ writeFileSync(path, html);
+ return path;
+ }
+
+ it("returns [] when the project has no data-composition-variables declarations", () => {
+ const indexPath = writeIndex(`
`);
+ expect(validateVariablesAgainstProject(indexPath, { title: "Hello" })).toEqual([]);
+ });
+
+ it("returns [] when every value matches its declaration", () => {
+ const indexPath = writeIndex(
+ `
`,
+ );
+ expect(validateVariablesAgainstProject(indexPath, { title: "Hello" })).toEqual([]);
+ });
+
+ it("flags undeclared keys", () => {
+ const indexPath = writeIndex(
+ `
`,
+ );
+ expect(validateVariablesAgainstProject(indexPath, { title: "Hello", extra: 1 })).toEqual([
+ { kind: "undeclared", variableId: "extra" },
+ ]);
+ });
+
+ it("flags type mismatches", () => {
+ const indexPath = writeIndex(
+ `
`,
+ );
+ expect(validateVariablesAgainstProject(indexPath, { count: "three" })).toEqual([
+ { kind: "type-mismatch", variableId: "count", expected: "number", actual: "string" },
+ ]);
+ });
+
+ it("returns [] when the index file cannot be read (lint owns that diagnostic)", () => {
+ expect(
+ validateVariablesAgainstProject(join(tmpDir, "missing.html"), { title: "Hello" }),
+ ).toEqual([]);
+ });
+});
diff --git a/packages/cli/src/commands/render.ts b/packages/cli/src/commands/render.ts
index a1e57e74f..c0299e192 100644
--- a/packages/cli/src/commands/render.ts
+++ b/packages/cli/src/commands/render.ts
@@ -9,7 +9,16 @@ export const examples: Example[] = [
["High quality at 60fps", "hyperframes render --fps 60 --quality high --output hd.mp4"],
["Deterministic render via Docker", "hyperframes render --docker --output deterministic.mp4"],
["Parallel rendering with 6 workers", "hyperframes render --workers 6 --output fast.mp4"],
- ["HDR output (H.265 10-bit)", "hyperframes render --hdr --output hdr-output.mp4"],
+ ["Opt out of browser GPU render", "hyperframes render --no-browser-gpu --output cpu.mp4"],
+ ["HDR output (auto-detected)", "hyperframes render --output hdr-output.mp4"],
+ [
+ "Override composition variables (parametrized render)",
+ 'hyperframes render --variables \'{"title":"Q4 Report","theme":"dark"}\' --output q4.mp4',
+ ],
+ [
+ "Variables from a JSON file",
+ "hyperframes render --variables-file ./vars.json --output out.mp4",
+ ],
];
import { cpus, freemem, tmpdir } from "node:os";
import { resolve, dirname, join, basename } from "node:path";
@@ -26,7 +35,14 @@ import { bytesToMb } from "../telemetry/system.js";
import { VERSION } from "../version.js";
import { isDevMode } from "../utils/env.js";
import { buildDockerRunArgs } from "../utils/dockerRunArgs.js";
+import { ensureDOMParser } from "../utils/dom.js";
import type { RenderJob } from "@hyperframes/producer";
+import {
+ extractCompositionMetadata,
+ validateVariables,
+ formatVariableValidationIssue,
+ type VariableValidationIssue,
+} from "@hyperframes/core";
const VALID_FPS = new Set([24, 30, 60]);
const VALID_QUALITY = new Set(["draft", "standard", "high"]);
@@ -35,11 +51,6 @@ const FORMAT_EXT: Record = { mp4: ".mp4", webm: ".webm", mov: ".
const CPU_CORE_COUNT = cpus().length;
-/** 3/4 of CPU cores, capped at 8. Each worker spawns a Chrome process (~256 MB). */
-function defaultWorkerCount(): number {
- return Math.max(1, Math.min(Math.floor((CPU_CORE_COUNT * 3) / 4), 8));
-}
-
export default defineCommand({
meta: {
name: "render",
@@ -87,7 +98,12 @@ export default defineCommand({
},
hdr: {
type: "boolean",
- description: "Enable HDR: probe sources for PQ/HLG, output H.265 10-bit BT.2020",
+ description: "Force HDR output even if no HDR sources are detected",
+ default: false,
+ },
+ sdr: {
+ type: "boolean",
+ description: "Force SDR output even if HDR sources are detected",
default: false,
},
crf: {
@@ -99,6 +115,11 @@ export default defineCommand({
description: "Target video bitrate such as 10M. Mutually exclusive with --crf.",
},
gpu: { type: "boolean", description: "Use GPU encoding", default: false },
+ "browser-gpu": {
+ type: "boolean",
+ description:
+ "Use host GPU acceleration for Chrome/WebGL capture. Enabled by default for local renders; use --no-browser-gpu to opt out.",
+ },
quiet: {
type: "boolean",
description: "Suppress verbose output",
@@ -118,6 +139,22 @@ export default defineCommand({
type: "string",
description: "Max concurrent renders when using the producer server (1-10). Default: 2.",
},
+ variables: {
+ type: "string",
+ description:
+ 'JSON object of variable values, merged over the composition\'s data-composition-variables defaults. Example: --variables \'{"title":"Hello"}\'. Read inside the composition via window.__hyperframes.getVariables().',
+ },
+ "variables-file": {
+ type: "string",
+ description:
+ "Path to a JSON file with variable values (alternative to --variables). The file must contain a single JSON object.",
+ },
+ "strict-variables": {
+ type: "boolean",
+ description:
+ "Fail render if any --variables key is undeclared or has a wrong type vs the composition's data-composition-variables. Without this flag, mismatches are warnings.",
+ default: false,
+ },
},
async run({ args }) {
// ── Resolve project ────────────────────────────────────────────────────
@@ -186,6 +223,8 @@ export default defineCommand({
const useDocker = args.docker ?? false;
const useGpu = args.gpu ?? false;
+ const browserGpuArg = args["browser-gpu"];
+ const useBrowserGpu = resolveBrowserGpuForCli(useDocker, browserGpuArg);
const quiet = args.quiet ?? false;
const strictAll = args["strict-all"] ?? false;
const strictErrors = (args.strict ?? false) || strictAll;
@@ -197,6 +236,15 @@ export default defineCommand({
process.exit(1);
}
+ if (useDocker && browserGpuArg === true) {
+ errorBox(
+ "Browser GPU is local-only",
+ "--browser-gpu uses the host Chrome GPU backend. Docker mode keeps browser rendering deterministic and does not expose a cross-platform Chrome GPU backend.",
+ "Run without --docker, or use --gpu for Docker GPU encoding where your Docker host supports GPU passthrough.",
+ );
+ process.exit(1);
+ }
+
let crf: number | undefined;
if (crfRaw != null) {
const parsed = Number(crfRaw);
@@ -216,12 +264,9 @@ export default defineCommand({
}
// ── Print render plan ─────────────────────────────────────────────────
- const workerCount = workers ?? defaultWorkerCount();
if (!quiet) {
const workerLabel =
- args.workers != null
- ? `${workerCount} workers`
- : `${workerCount} workers (auto — ${CPU_CORE_COUNT} cores detected)`;
+ workers != null ? `${workers} workers` : `auto workers (${CPU_CORE_COUNT} cores detected)`;
console.log("");
console.log(
c.accent("\u25C6") +
@@ -230,6 +275,13 @@ export default defineCommand({
c.dim(" \u2192 " + outputPath),
);
console.log(c.dim(" " + fps + "fps \u00B7 " + quality + " \u00B7 " + workerLabel));
+ if (useGpu || useBrowserGpu) {
+ const gpuModes = [
+ useGpu ? "encoder GPU" : null,
+ useBrowserGpu ? "browser GPU (auto)" : null,
+ ].filter(Boolean);
+ console.log(c.dim(" GPU: " + gpuModes.join(" + ")));
+ }
console.log("");
}
@@ -301,31 +353,73 @@ export default defineCommand({
}
}
+ // ── Validate HDR/SDR mutual exclusion ────────────────────────────────
+ if (args.hdr && args.sdr) {
+ console.error("Error: --hdr and --sdr are mutually exclusive.");
+ process.exit(1);
+ }
+
+ // ── Resolve --variables / --variables-file ──────────────────────────
+ const variables = resolveVariablesArg(args.variables, args["variables-file"]);
+
+ // ── Validate --variables against data-composition-variables ─────────
+ const strictVariables = args["strict-variables"] ?? false;
+ if (variables && Object.keys(variables).length > 0) {
+ const issues = validateVariablesAgainstProject(project.indexPath, variables);
+ if (issues.length > 0) {
+ if (!quiet) {
+ console.log("");
+ console.log(
+ c.warn(
+ `Variable ${issues.length === 1 ? "issue" : "issues"} (${issues.length}) — values may not render as expected:`,
+ ),
+ );
+ for (const issue of issues) {
+ console.log(" " + c.dim(formatVariableValidationIssue(issue)));
+ }
+ console.log("");
+ }
+ if (strictVariables) {
+ console.log(
+ c.error(" Aborting render due to variable issues (--strict-variables mode)."),
+ );
+ console.log("");
+ process.exit(1);
+ }
+ }
+ }
+
// ── Render ────────────────────────────────────────────────────────────
if (useDocker) {
await renderDocker(project.dir, outputPath, {
fps,
quality,
format,
- workers: workerCount,
+ workers,
gpu: useGpu,
- hdr: args.hdr ?? false,
+ browserGpu: useBrowserGpu,
+ hdrMode: args.sdr ? "force-sdr" : args.hdr ? "force-hdr" : "auto",
crf,
videoBitrate,
quiet,
+ variables,
+ exitAfterComplete: true,
});
} else {
await renderLocal(project.dir, outputPath, {
fps,
quality,
format,
- workers: workerCount,
+ workers,
gpu: useGpu,
- hdr: args.hdr ?? false,
+ browserGpu: useBrowserGpu,
+ hdrMode: args.sdr ? "force-sdr" : args.hdr ? "force-hdr" : "auto",
crf,
videoBitrate,
quiet,
browserPath,
+ variables,
+ exitAfterComplete: true,
});
}
},
@@ -335,13 +429,165 @@ interface RenderOptions {
fps: 24 | 30 | 60;
quality: "draft" | "standard" | "high";
format: "mp4" | "webm" | "mov";
- workers: number;
+ workers?: number;
gpu: boolean;
- hdr: boolean;
+ browserGpu: boolean;
+ hdrMode: "auto" | "force-hdr" | "force-sdr";
crf?: number;
videoBitrate?: string;
quiet: boolean;
browserPath?: string;
+ variables?: Record;
+ exitAfterComplete?: boolean;
+}
+
+export type VariablesParseError =
+ | { kind: "conflict" }
+ | { kind: "read-error"; path: string; cause: string }
+ | { kind: "parse-error"; source: "inline" | "file"; cause: string }
+ | { kind: "shape-error" };
+
+export type VariablesParseResult =
+ | { ok: true; value: Record | undefined }
+ | { ok: false; error: VariablesParseError };
+
+/**
+ * Pure parser for `--variables` / `--variables-file` flag pair. Splits out
+ * from `resolveVariablesArg` so validation paths are unit-testable without
+ * triggering `process.exit`. Reports failures via a structured `kind`
+ * discriminant so the side-effecting wrapper owns all UI strings.
+ */
+export function parseVariablesArg(
+ inline: string | undefined,
+ filePath: string | undefined,
+ readFile: (path: string) => string = (p) => readFileSync(resolve(p), "utf8"),
+): VariablesParseResult {
+ if (inline != null && filePath != null) {
+ return { ok: false, error: { kind: "conflict" } };
+ }
+ let raw: string | undefined;
+ let source: "inline" | "file" | undefined;
+ if (inline != null) {
+ raw = inline;
+ source = "inline";
+ } else if (filePath != null) {
+ try {
+ raw = readFile(filePath);
+ source = "file";
+ } catch (error: unknown) {
+ return {
+ ok: false,
+ error: {
+ kind: "read-error",
+ path: filePath,
+ cause: error instanceof Error ? error.message : String(error),
+ },
+ };
+ }
+ }
+ if (raw == null) return { ok: true, value: undefined };
+
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(raw);
+ } catch (error: unknown) {
+ return {
+ ok: false,
+ error: {
+ kind: "parse-error",
+ source: source ?? "inline",
+ cause: error instanceof Error ? error.message : String(error),
+ },
+ };
+ }
+ if (parsed == null || typeof parsed !== "object" || Array.isArray(parsed)) {
+ return { ok: false, error: { kind: "shape-error" } };
+ }
+ return { ok: true, value: parsed as Record };
+}
+
+function variablesErrorMessage(error: VariablesParseError): { title: string; message: string } {
+ switch (error.kind) {
+ case "conflict":
+ return {
+ title: "Conflicting variables flags",
+ message: "Use either --variables or --variables-file, not both.",
+ };
+ case "read-error":
+ return {
+ title: "Could not read --variables-file",
+ message: `${error.path}: ${error.cause}`,
+ };
+ case "parse-error":
+ return {
+ title:
+ error.source === "file"
+ ? "Invalid JSON in --variables-file"
+ : "Invalid JSON in --variables",
+ message: error.cause,
+ };
+ case "shape-error":
+ return {
+ title: "Invalid variables payload",
+ message: 'Variables must be a JSON object (e.g. {"title":"Hello"}).',
+ };
+ }
+}
+
+/**
+ * Resolve `--variables` / `--variables-file` into a plain object, or
+ * `undefined` when neither flag is set. Exits the process with a friendly
+ * error box on any validation failure.
+ */
+export function resolveVariablesArg(
+ inline: string | undefined,
+ filePath: string | undefined,
+): Record | undefined {
+ const result = parseVariablesArg(inline, filePath);
+ if (!result.ok) {
+ const { title, message } = variablesErrorMessage(result.error);
+ errorBox(title, message);
+ process.exit(1);
+ }
+ return result.value;
+}
+
+/**
+ * Validate `--variables` values against the project's top-level
+ * `data-composition-variables` declarations. Returns an empty array when
+ * the index has no declarations or when every key is declared with a
+ * matching type. Errors reading the index are silently treated as "no
+ * declarations" — the lint pass owns malformed-HTML diagnostics, render
+ * shouldn't fail just because the schema is unreadable.
+ */
+export function validateVariablesAgainstProject(
+ indexPath: string,
+ values: Record,
+): VariableValidationIssue[] {
+ let html: string;
+ try {
+ html = readFileSync(indexPath, "utf8");
+ } catch {
+ return [];
+ }
+ // extractCompositionMetadata uses DOMParser, which Node doesn't ship.
+ // Same pattern as `compositions.ts` and other CLI commands that touch
+ // @hyperframes/core's HTML parsers.
+ ensureDOMParser();
+ const meta = extractCompositionMetadata(html);
+ if (meta.variables.length === 0) return [];
+ return validateVariables(values, meta.variables);
+}
+
+export function resolveBrowserGpuForCli(
+ useDocker: boolean,
+ browserGpuArg: boolean | undefined,
+ envMode = process.env.PRODUCER_BROWSER_GPU_MODE,
+): boolean {
+ if (useDocker) return false;
+ if (browserGpuArg !== undefined) return browserGpuArg;
+ if (envMode === "software") return false;
+ return true;
}
const DOCKER_IMAGE_PREFIX = "hyperframes-renderer";
@@ -461,10 +707,12 @@ async function renderDocker(
format: options.format,
workers: options.workers,
gpu: options.gpu,
- hdr: options.hdr,
+ browserGpu: options.browserGpu,
+ hdrMode: options.hdrMode,
crf: options.crf,
videoBitrate: options.videoBitrate,
quiet: options.quiet,
+ variables: options.variables,
},
});
@@ -504,9 +752,10 @@ async function renderDocker(
printRenderComplete(outputPath, elapsed, options.quiet);
await logRenderCost(projectDir, outputPath, elapsed, options, undefined);
+ if (options.exitAfterComplete) scheduleRenderProcessExit();
}
-async function renderLocal(
+export async function renderLocal(
projectDir: string,
outputPath: string,
options: RenderOptions,
@@ -528,9 +777,13 @@ async function renderLocal(
format: options.format,
workers: options.workers,
useGpu: options.gpu,
- hdr: options.hdr,
+ producerConfig: producer.resolveConfig({
+ browserGpuMode: options.browserGpu ? "hardware" : "software",
+ }),
+ hdrMode: options.hdrMode,
crf: options.crf,
videoBitrate: options.videoBitrate,
+ variables: options.variables,
});
const onProgress = options.quiet
@@ -549,6 +802,27 @@ async function renderLocal(
trackRenderMetrics(job, elapsed, options, false);
printRenderComplete(outputPath, elapsed, options.quiet);
await logRenderCost(projectDir, outputPath, elapsed, options, job);
+ if (options.exitAfterComplete) scheduleRenderProcessExit();
+}
+
+type UnrefableTimer = {
+ unref: () => void;
+};
+
+function isUnrefableTimer(
+ timer: ReturnType,
+): timer is ReturnType & UnrefableTimer {
+ return (
+ typeof timer === "object" &&
+ timer !== null &&
+ "unref" in timer &&
+ typeof timer.unref === "function"
+ );
+}
+
+function scheduleRenderProcessExit(): void {
+ const timer = setTimeout(() => process.exit(0), 100);
+ if (isUnrefableTimer(timer)) timer.unref();
}
function getMemorySnapshot() {
@@ -606,7 +880,7 @@ function trackRenderMetrics(
durationMs: elapsedMs,
fps: options.fps,
quality: options.quality,
- workers: options.workers,
+ workers: options.workers ?? perf?.workers,
docker,
gpu: options.gpu,
compositionDurationMs,
@@ -680,7 +954,7 @@ async function logRenderCost(
format: options.format,
workers: options.workers,
gpu: options.gpu,
- hdr: options.hdr,
+ hdrMode: options.hdrMode,
outputPath,
},
);
@@ -693,7 +967,7 @@ async function logRenderCost(
format: options.format,
quality: options.quality,
fps: options.fps,
- hdr: options.hdr,
+ hdrMode: options.hdrMode,
},
});
} catch (err) {
diff --git a/packages/cli/src/commands/snapshot.ts b/packages/cli/src/commands/snapshot.ts
index 5b21d62b9..0fd2a81da 100644
--- a/packages/cli/src/commands/snapshot.ts
+++ b/packages/cli/src/commands/snapshot.ts
@@ -2,15 +2,13 @@ import { spawn } from "node:child_process";
import { defineCommand } from "citty";
import { existsSync, mkdtempSync, readFileSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
-import { resolve, join, dirname, relative, isAbsolute } from "node:path";
-import { fileURLToPath } from "node:url";
+import { resolve, join, relative, isAbsolute } from "node:path";
import { resolveProject } from "../utils/project.js";
+import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js";
+import { serveStaticProjectHtml } from "../utils/staticProjectServer.js";
import { c } from "../ui/colors.js";
import type { Example } from "./_examples.js";
-const __filename = fileURLToPath(import.meta.url);
-const __dirname = dirname(__filename);
-
/** Maximum time a single-frame FFmpeg extract is allowed to run. Mirrors the
* default applied by `@hyperframes/engine`'s `runFfmpeg` so a pathological
* clip (corrupt media, stalled network mount, codec edge case) cannot wedge
@@ -99,63 +97,11 @@ async function captureSnapshots(
const numFrames = opts.frames ?? 5;
- // 1. Bundle
- let html = await bundleToSingleHtml(projectDir);
-
- // Inject local runtime if available
- const runtimePath = resolve(
- __dirname,
- "..",
- "..",
- "..",
- "core",
- "dist",
- "hyperframe.runtime.iife.js",
- );
- if (existsSync(runtimePath)) {
- const runtimeSource = readFileSync(runtimePath, "utf-8");
- html = html.replace(
- /]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
- () => ``,
- );
- }
-
- // 2. Start minimal file server
- const { createServer } = await import("node:http");
- const { getMimeType } = await import("@hyperframes/core/studio-api");
+ // 1. Bundle. `bundleToSingleHtml` now inlines the runtime IIFE by default,
+ // so the previous post-bundle runtime substitution is no longer needed.
+ const html = await bundleToSingleHtml(projectDir);
- const server = createServer((req, res) => {
- const url = req.url ?? "/";
- if (url === "/" || url === "/index.html") {
- res.writeHead(200, { "Content-Type": "text/html" });
- res.end(html);
- return;
- }
- const filePath = resolve(projectDir, decodeURIComponent(url).replace(/^\//, ""));
- const rel = relative(projectDir, filePath);
- if (rel.startsWith("..") || isAbsolute(rel)) {
- res.writeHead(403);
- res.end();
- return;
- }
- if (existsSync(filePath)) {
- res.writeHead(200, { "Content-Type": getMimeType(filePath) });
- res.end(readFileSync(filePath));
- return;
- }
- res.writeHead(404);
- res.end();
- });
-
- const port = await new Promise((resolvePort, rejectPort) => {
- server.on("error", rejectPort); // register before listen to catch sync bind errors
- server.listen(0, () => {
- const addr = server.address();
- const p = typeof addr === "object" && addr ? addr.port : 0;
- if (!p) rejectPort(new Error("Failed to bind local HTTP server"));
- else resolvePort(p);
- });
- });
+ const server = await serveStaticProjectHtml(projectDir, html);
const savedPaths: string[] = [];
@@ -178,9 +124,9 @@ async function captureSnapshots(
try {
const page = await chromeBrowser.newPage();
- await page.setViewport({ width: 1920, height: 1080 });
+ await page.setViewport(resolveCompositionViewportFromHtml(html));
- await page.goto(`http://127.0.0.1:${port}/`, {
+ await page.goto(server.url, {
waitUntil: "domcontentloaded",
timeout: 10000,
});
@@ -424,7 +370,7 @@ async function captureSnapshots(
await chromeBrowser.close();
}
} finally {
- server.close();
+ await server.close();
}
return savedPaths;
diff --git a/packages/cli/src/commands/validate.test.ts b/packages/cli/src/commands/validate.test.ts
new file mode 100644
index 000000000..d80564d8d
--- /dev/null
+++ b/packages/cli/src/commands/validate.test.ts
@@ -0,0 +1,22 @@
+import { describe, expect, it } from "vitest";
+import { shouldIgnoreRequestFailure } from "./validate.js";
+
+describe("shouldIgnoreRequestFailure", () => {
+ it("ignores aborted media preload requests", () => {
+ expect(
+ shouldIgnoreRequestFailure("http://127.0.0.1:3000/assets/sfx.wav", "net::ERR_ABORTED"),
+ ).toBe(true);
+ expect(shouldIgnoreRequestFailure("http://127.0.0.1:3000/video.mp4", "net::ERR_ABORTED")).toBe(
+ true,
+ );
+ });
+
+ it("keeps non-media and non-aborted failures reportable", () => {
+ expect(
+ shouldIgnoreRequestFailure("http://127.0.0.1:3000/assets/map.png", "net::ERR_ABORTED"),
+ ).toBe(false);
+ expect(
+ shouldIgnoreRequestFailure("http://127.0.0.1:3000/assets/sfx.wav", "net::ERR_FAILED"),
+ ).toBe(false);
+ });
+});
diff --git a/packages/cli/src/commands/validate.ts b/packages/cli/src/commands/validate.ts
index 4cfb60780..8c9a2fba0 100644
--- a/packages/cli/src/commands/validate.ts
+++ b/packages/cli/src/commands/validate.ts
@@ -1,8 +1,9 @@
import { defineCommand } from "citty";
import { existsSync, readFileSync } from "node:fs";
-import { resolve, join, dirname } from "node:path";
+import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { resolveProject } from "../utils/project.js";
+import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js";
import { c } from "../ui/colors.js";
import { withMeta } from "../utils/updateCheck.js";
@@ -27,12 +28,18 @@ interface ContrastEntry {
bg: string;
}
-// esbuild's text loader inlines this at build time — no runtime file read.
-// @ts-expect-error — .browser.js files use esbuild text loader, not TS module resolution
-import CONTRAST_AUDIT_SCRIPT from "./contrast-audit.browser.js";
-
const CONTRAST_SAMPLES = 5;
const SEEK_SETTLE_MS = 150;
+const MEDIA_EXTENSIONS = /\.(aac|flac|m4a|mov|mp3|mp4|oga|ogg|wav|webm)$/i;
+
+export function shouldIgnoreRequestFailure(url: string, errorText: string | undefined): boolean {
+ if (errorText !== "net::ERR_ABORTED") return false;
+ try {
+ return MEDIA_EXTENSIONS.test(new URL(url).pathname);
+ } catch {
+ return false;
+ }
+}
async function getCompositionDuration(page: import("puppeteer-core").Page): Promise {
return page.evaluate(() => {
@@ -64,7 +71,7 @@ async function runContrastAudit(page: import("puppeteer-core").Page): Promise]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
- () => ``,
- );
- }
+ // `bundleToSingleHtml` now inlines the runtime IIFE by default, so the
+ // previous post-bundle regex substitution (which matched `src="..."` on the
+ // runtime tag) is no longer needed — there's no `src` attribute to match.
+ const html = await bundleToSingleHtml(projectDir);
const { createServer } = await import("node:http");
const { getMimeType } = await import("@hyperframes/core/studio-api");
@@ -153,7 +159,7 @@ async function validateInBrowser(
});
const page = await chromeBrowser.newPage();
- await page.setViewport({ width: 1920, height: 1080 });
+ await page.setViewport(resolveCompositionViewportFromHtml(html));
page.on("console", (msg) => {
const type = msg.type();
@@ -174,10 +180,12 @@ async function validateInBrowser(
page.on("requestfailed", (req) => {
const url = req.url();
if (url.includes("favicon") || url.startsWith("data:")) return;
+ const failureText = req.failure()?.errorText;
+ if (shouldIgnoreRequestFailure(url, failureText)) return;
const path = decodeURIComponent(new URL(url).pathname).replace(/^\//, "");
errors.push({
level: "error",
- text: `Failed to load ${path}: ${req.failure()?.errorText ?? "net::ERR_FAILED"}`,
+ text: `Failed to load ${path}: ${failureText ?? "net::ERR_FAILED"}`,
url,
});
});
diff --git a/packages/cli/src/docs/compositions.md b/packages/cli/src/docs/compositions.md
index 6a0d42a9d..79ef1c88f 100644
--- a/packages/cli/src/docs/compositions.md
+++ b/packages/cli/src/docs/compositions.md
@@ -26,14 +26,42 @@ Use `npx hyperframes compositions` to see all compositions in a project.
## Variables
-HyperFrames does not automatically bind `data-var-*` attributes into your composition DOM.
+Two attributes with different shapes and different jobs:
+
+- **`data-composition-variables`** on the `` root — a JSON **array of declarations** (`{id, type, label, default}` per entry). Defines the schema: which variables exist, what type they are, and what defaults to use when no override is provided.
+- **`data-variable-values`** on a sub-comp host element — a JSON **object keyed by variable id** (`{"title":"Pro","price":"$29"}`). Carries per-instance overrides for that one mount of the sub-composition.
+
+They aren't redundant — one is "what variables does this composition have?" and the other is "what values should this particular embed use?" Inside any composition script, `window.__hyperframes.getVariables()` returns the merged result. Layering, lowest to highest precedence:
+
+1. Declared defaults from `data-composition-variables`
+2. Per-instance overrides from the host's `data-variable-values` (sub-comp embeds only)
+3. CLI overrides from `npx hyperframes render --variables '{...}'` (top-level renders only)
+
+```html
+
+
+
+
+
+
+
+
```
-Read `data-variable-values` inside the nested composition and apply the values in your own script. Variable metadata for tooling is declared separately via `data-composition-variables` and read with `extractCompositionMetadata()`.
+The runtime layers `data-variable-values` over the sub-comp's declared defaults on a per-instance basis. The same `getVariables()` call works at the top level too — the CLI flag `--variables` provides the override, declared `default`s fall through for missing keys.
diff --git a/packages/cli/src/docs/rendering.md b/packages/cli/src/docs/rendering.md
index 83aa596db..15a4c44f5 100644
--- a/packages/cli/src/docs/rendering.md
+++ b/packages/cli/src/docs/rendering.md
@@ -19,11 +19,14 @@ Requires: Docker installed and running.
- `-w, --workers` — Parallel workers 1-8 (default: auto)
- `--crf` — Override encoder CRF (mutually exclusive with `--video-bitrate`)
- `--video-bitrate` — Target video bitrate such as `10M` (mutually exclusive with `--crf`)
-- `--gpu` — Use GPU encoding (NVENC, VideoToolbox, VAAPI)
+- `--gpu` — Use GPU encoding (NVENC, VideoToolbox, VAAPI, QSV)
+- `--browser-gpu` / `--no-browser-gpu` — Use or opt out of host GPU acceleration for local Chrome/WebGL capture (enabled by default for local renders, disabled in Docker)
- `-o, --output` — Custom output path
## Tips
- Use `draft` quality for fast previews during development
+- Local renders use browser GPU capture automatically; use `--no-browser-gpu` to compare against the software-browser path
+- Use `--gpu` when a local render also benefits from hardware FFmpeg encoding
- Use `npx hyperframes benchmark` to find optimal settings
- 4 workers is usually the sweet spot for most compositions
diff --git a/packages/cli/src/help.ts b/packages/cli/src/help.ts
index 8b6f9afbe..5867851a0 100644
--- a/packages/cli/src/help.ts
+++ b/packages/cli/src/help.ts
@@ -74,6 +74,7 @@ const GROUPS: Group[] = [
"Close the Gemini retention loop on a rendered project (review → patch → re-render until target)",
],
["images", "Manage image assets used by the visual director"],
+ ["remove-background", "Remove background from a video or image to produce transparent media"],
],
},
{
@@ -119,7 +120,7 @@ const STATIC_EXAMPLES: Record = {
// ── Render root help ───────────────────────────────────────────────────────
function renderRootHelp(): string {
- const NAME_COL = 16;
+ const NAME_COL = 19;
const CMD_COL = 46;
const lines: string[] = [];
diff --git a/packages/cli/src/registry/installer.ts b/packages/cli/src/registry/installer.ts
index 5ba6d2cde..4ce4314e1 100644
--- a/packages/cli/src/registry/installer.ts
+++ b/packages/cli/src/registry/installer.ts
@@ -6,6 +6,7 @@
* runtime to reject traversal even if the registry JSON schema was bypassed.
*/
+import { readFileSync, writeFileSync } from "node:fs";
import { resolve, relative, isAbsolute } from "node:path";
import type { FileTarget, RegistryItem } from "@hyperframes/core";
import { fetchItemFile, DEFAULT_REGISTRY_URL } from "./remote.js";
@@ -45,6 +46,22 @@ export function assertSafeTarget(destDir: string, target: string): void {
}
}
+function isInstalledRegistryBlockComposition(item: RegistryItem, file: FileTarget): boolean {
+ return (
+ item.type === "hyperframes:block" &&
+ file.type === "hyperframes:composition" &&
+ file.target.toLowerCase().endsWith(".html")
+ );
+}
+
+function addRegistryItemMarker(source: string, item: RegistryItem): string {
+ if (/^\s*/i.test(source.slice(0, 512))) {
+ return source;
+ }
+
+ return `\n${source}`;
+}
+
/**
* Install a resolved `RegistryItem` into `destDir` by fetching each file in
* parallel and writing it to its validated target path.
@@ -65,6 +82,10 @@ export async function installItem(
item.files.map(async (file: FileTarget) => {
const destPath = resolve(destDir, file.target);
await fetchItemFile(item, file, destPath, baseUrl);
+ if (isInstalledRegistryBlockComposition(item, file)) {
+ const source = readFileSync(destPath, "utf-8");
+ writeFileSync(destPath, addRegistryItemMarker(source, item), "utf-8");
+ }
return destPath;
}),
);
diff --git a/packages/cli/src/registry/remote.ts b/packages/cli/src/registry/remote.ts
index 49b254a4c..16a5f5fbd 100644
--- a/packages/cli/src/registry/remote.ts
+++ b/packages/cli/src/registry/remote.ts
@@ -58,9 +58,14 @@ function readCache(path: string): T | undefined {
}
function writeCache(path: string, data: T): void {
- mkdirSync(dirname(path), { recursive: true });
- const entry: CacheEntry = { fetchedAt: Date.now(), data };
- writeFileSync(path, JSON.stringify(entry), "utf-8");
+ try {
+ mkdirSync(dirname(path), { recursive: true });
+ const entry: CacheEntry = { fetchedAt: Date.now(), data };
+ writeFileSync(path, JSON.stringify(entry), "utf-8");
+ } catch {
+ // Cache writes are opportunistic. A read-only home directory or sandboxed
+ // environment should not make the registry appear unreachable.
+ }
}
// ── Fetchers ────────────────────────────────────────────────────────────────
diff --git a/packages/cli/src/server/fileWatcher.test.ts b/packages/cli/src/server/fileWatcher.test.ts
new file mode 100644
index 000000000..62412fcb9
--- /dev/null
+++ b/packages/cli/src/server/fileWatcher.test.ts
@@ -0,0 +1,19 @@
+import { describe, expect, it } from "vitest";
+
+import { shouldWatchProjectFile } from "./fileWatcher.js";
+
+describe("shouldWatchProjectFile", () => {
+ it("watches files that can affect the project signature", () => {
+ expect(shouldWatchProjectFile("index.html")).toBe(true);
+ expect(shouldWatchProjectFile("src/scene.tsx")).toBe(true);
+ expect(shouldWatchProjectFile("assets/hero.png")).toBe(true);
+ expect(shouldWatchProjectFile("Dockerfile")).toBe(true);
+ });
+
+ it("skips generated and dependency directories excluded from signatures", () => {
+ expect(shouldWatchProjectFile("node_modules/pkg/index.js")).toBe(false);
+ expect(shouldWatchProjectFile("renders/output.mp4")).toBe(false);
+ expect(shouldWatchProjectFile("dist/index.html")).toBe(false);
+ expect(shouldWatchProjectFile(".hyperframes/cache.json")).toBe(false);
+ });
+});
diff --git a/packages/cli/src/server/fileWatcher.ts b/packages/cli/src/server/fileWatcher.ts
index ffe0620a9..050f009ea 100644
--- a/packages/cli/src/server/fileWatcher.ts
+++ b/packages/cli/src/server/fileWatcher.ts
@@ -8,9 +8,27 @@ export interface ProjectWatcher {
close(): void;
}
-const WATCHED_EXTENSIONS = new Set([".html", ".css", ".js", ".json"]);
+const WATCHER_EXCLUDED_DIRS = new Set([
+ ".cache",
+ ".git",
+ ".hyperframes",
+ ".next",
+ ".vite",
+ "build",
+ "coverage",
+ "dist",
+ "node_modules",
+ "outputs",
+ "renders",
+]);
const DEBOUNCE_MS = 300;
+export function shouldWatchProjectFile(filename: string): boolean {
+ if (!filename) return false;
+ const parts = filename.split(/[\\/]+/);
+ return !parts.some((part) => WATCHER_EXCLUDED_DIRS.has(part));
+}
+
export function createProjectWatcher(projectDir: string): ProjectWatcher {
const listeners = new Set();
let debounceTimer: ReturnType | null = null;
@@ -19,13 +37,13 @@ export function createProjectWatcher(projectDir: string): ProjectWatcher {
try {
watcher = watch(projectDir, { recursive: true }, (_event, filename) => {
if (!filename) return;
- const ext = "." + filename.split(".").pop()?.toLowerCase();
- if (!WATCHED_EXTENSIONS.has(ext)) return;
+ const relativePath = filename.toString();
+ if (!shouldWatchProjectFile(relativePath)) return;
if (debounceTimer) clearTimeout(debounceTimer);
debounceTimer = setTimeout(() => {
for (const fn of listeners) {
- fn(filename);
+ fn(relativePath);
}
}, DEBOUNCE_MS);
});
diff --git a/packages/cli/src/server/studioServer.ts b/packages/cli/src/server/studioServer.ts
index adc99d606..f61f6a62a 100644
--- a/packages/cli/src/server/studioServer.ts
+++ b/packages/cli/src/server/studioServer.ts
@@ -5,7 +5,7 @@
* providing a CLI-specific adapter for single-project, in-process rendering.
*/
-import { Hono } from "hono";
+import { Hono, type Context } from "hono";
import { streamSSE } from "hono/streaming";
import { existsSync, readFileSync, writeFileSync, statSync } from "node:fs";
import { resolve, join, basename } from "node:path";
@@ -14,11 +14,14 @@ import { loadRuntimeSource } from "./runtimeSource.js";
import { VERSION as version } from "../version.js";
import {
createStudioApi,
+ createProjectSignature,
getMimeType,
type StudioApiAdapter,
type ResolvedProject,
type RenderJobState,
} from "@hyperframes/core/studio-api";
+import { getElementScreenshotClip } from "@hyperframes/core/studio-api/screenshot-clip";
+import type { ScreenshotClip } from "@hyperframes/core/studio-api/screenshot-clip";
// ── Path resolution ─────────────────────────────────────────────────────────
@@ -142,6 +145,10 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
// ── CLI adapter for the shared studio API ──────────────────────────────
const project: ResolvedProject = { id: projectId, dir: projectDir, title: projectId };
+ let cachedProjectSignature: string | null = null;
+ watcher.addListener(() => {
+ cachedProjectSignature = null;
+ });
const adapter: StudioApiAdapter = {
listProjects: () => [project],
@@ -151,8 +158,11 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
async bundle(dir: string): Promise {
try {
const { bundleToSingleHtml } = await import("@hyperframes/core/compiler");
- let html = await bundleToSingleHtml(dir);
- // Fix empty runtime src from bundler — point to the local runtime endpoint
+ // Studio dev server: ask the bundler for an empty `src=""` placeholder so
+ // we can point it at our hot-reloadable local runtime endpoint. Inlining
+ // ~150 KB of runtime body on every preview render would defeat browser
+ // caching across composition edits.
+ let html = await bundleToSingleHtml(dir, { runtime: "placeholder" });
html = html.replace(
'data-hyperframes-preview-runtime="1" src=""',
'data-hyperframes-preview-runtime="1" src="/api/runtime.js"',
@@ -164,6 +174,12 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
}
},
+ getProjectSignature(dir: string): string {
+ if (resolve(dir) !== resolve(projectDir)) return createProjectSignature(dir);
+ cachedProjectSignature ??= createProjectSignature(projectDir);
+ return cachedProjectSignature;
+ },
+
async lint(html: string, opts?: { filePath?: string }) {
const { lintHyperframeHtml } = await import("@hyperframes/core/lint");
return lintHyperframeHtml(html, opts);
@@ -258,31 +274,22 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
}, opts.seekTime);
// Let the seek render settle.
await new Promise((r) => setTimeout(r, 200));
- let clip: { x: number; y: number; width: number; height: number } | undefined;
+ let clip: ScreenshotClip | undefined;
if (opts.selector) {
- clip = await page.evaluate((selector: string) => {
- const el = document.querySelector(selector);
- if (!(el instanceof HTMLElement)) return undefined;
- const rect = el.getBoundingClientRect();
- if (rect.width < 4 || rect.height < 4) return undefined;
- const pad = 8;
- const x = Math.max(0, rect.left - pad);
- const y = Math.max(0, rect.top - pad);
- const maxWidth = window.innerWidth - x;
- const maxHeight = window.innerHeight - y;
- return {
- x,
- y,
- width: Math.max(1, Math.min(rect.width + pad * 2, maxWidth)),
- height: Math.max(1, Math.min(rect.height + pad * 2, maxHeight)),
- };
- }, opts.selector);
+ clip = await page.evaluate(getElementScreenshotClip, opts.selector);
}
- const screenshot = (await page.screenshot({
- type: "jpeg",
- quality: 80,
- ...(clip ? { clip } : {}),
- })) as Buffer;
+ const screenshot = (await page.screenshot(
+ opts.format === "png"
+ ? {
+ type: "png",
+ ...(clip ? { clip } : {}),
+ }
+ : {
+ type: "jpeg",
+ quality: 80,
+ ...(clip ? { clip } : {}),
+ },
+ )) as Buffer;
return screenshot;
} catch {
return null;
@@ -353,23 +360,16 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
});
// Studio SPA static files
- app.get("/assets/*", (c) => {
+ const serveStudioStaticFile = (c: Context) => {
const filePath = resolve(studioDir, c.req.path.slice(1));
if (!existsSync(filePath) || !statSync(filePath).isFile()) return c.text("not found", 404);
const content = readFileSync(filePath);
return new Response(content, {
headers: { "Content-Type": getMimeType(filePath), "Cache-Control": "no-store" },
});
- });
-
- app.get("/icons/*", (c) => {
- const filePath = resolve(studioDir, c.req.path.slice(1));
- if (!existsSync(filePath) || !statSync(filePath).isFile()) return c.text("not found", 404);
- const content = readFileSync(filePath);
- return new Response(content, {
- headers: { "Content-Type": getMimeType(filePath), "Cache-Control": "no-store" },
- });
- });
+ };
+ app.get("/assets/*", serveStudioStaticFile);
+ app.get("/icons/*", serveStudioStaticFile);
// SPA fallback
app.get("*", (c) => {
diff --git a/packages/cli/src/telemetry/events.ts b/packages/cli/src/telemetry/events.ts
index c89016a0b..b26d81a1c 100644
--- a/packages/cli/src/telemetry/events.ts
+++ b/packages/cli/src/telemetry/events.ts
@@ -8,7 +8,7 @@ export function trackRenderComplete(props: {
durationMs: number;
fps: number;
quality: string;
- workers: number;
+ workers?: number;
docker: boolean;
gpu: boolean;
// Composition metadata
@@ -105,8 +105,8 @@ export function trackRenderError(props: {
});
}
-export function trackInitTemplate(templateId: string): void {
- trackEvent("init_template", { template: templateId });
+export function trackInitTemplate(templateId: string, props?: { tailwind?: boolean }): void {
+ trackEvent("init_template", { template: templateId, tailwind: props?.tailwind });
}
export function trackBrowserInstall(): void {
diff --git a/packages/cli/src/templates/_shared/AGENTS.md b/packages/cli/src/templates/_shared/AGENTS.md
index 506ecf394..4f5e90cea 100644
--- a/packages/cli/src/templates/_shared/AGENTS.md
+++ b/packages/cli/src/templates/_shared/AGENTS.md
@@ -8,15 +8,15 @@ This project uses AI agent skills for framework-specific patterns. Install them
npx skills add heygen-com/hyperframes
```
-Skills encode patterns like `window.__timelines` registration, `data-*` attribute semantics, and shader-compatible CSS rules that are not in generic web docs. Using them produces correct compositions from the start.
+Skills encode patterns like `window.__timelines` registration, `data-*` attribute semantics, Tailwind v4 browser-runtime styling for `--tailwind` projects, and shader-compatible CSS rules that are not in generic web docs. Using them produces correct compositions from the start.
## Commands
```bash
-npx hyperframes preview # preview in browser (studio editor)
-npx hyperframes render # render to MP4
-npx hyperframes lint # validate compositions (errors + warnings)
-npx hyperframes lint --json # machine-readable output for CI
+npm run dev # preview in browser (studio editor)
+npm run check # lint + validate + inspect
+npm run render # render to MP4
+npm run publish # publish and get a shareable link
npx hyperframes docs # reference docs in terminal
```
@@ -30,10 +30,10 @@ npx hyperframes docs # reference docs in terminal
## Linting — Always Run After Changes
-After creating or editing any `.html` composition, run the linter before considering the task complete:
+After creating or editing any `.html` composition, run the full check before considering the task complete:
```bash
-npx hyperframes lint
+npm run check
```
Fix all errors before presenting the result.
diff --git a/packages/cli/src/templates/_shared/CLAUDE.md b/packages/cli/src/templates/_shared/CLAUDE.md
index 1d1a7930d..fea7ce58f 100644
--- a/packages/cli/src/templates/_shared/CLAUDE.md
+++ b/packages/cli/src/templates/_shared/CLAUDE.md
@@ -7,10 +7,17 @@
| Skill | Command | When to use |
| -------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------- |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights |
-| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts |
+| **hyperframes-cli** | `/hyperframes-cli` | Dev-loop CLI: init, lint, inspect, preview, render, doctor |
+| **hyperframes-media** | `/hyperframes-media` | Asset preprocessing: tts (Kokoro), transcribe (Whisper), remove-background (u2net) |
| **hyperframes-registry** | `/hyperframes-registry` | Installing blocks and components via `hyperframes add` |
| **website-to-hyperframes** | `/website-to-hyperframes` | Capturing a URL and turning it into a video — full website-to-video pipeline |
+| **tailwind** | `/tailwind` | Tailwind v4 browser-runtime styles for projects created with `hyperframes init --tailwind` |
| **gsap** | `/gsap` | GSAP animations for HyperFrames — tweens, timelines, easing, performance |
+| **animejs** | `/animejs` | Anime.js animations registered on `window.__hfAnime` |
+| **css-animations** | `/css-animations` | CSS keyframes that HyperFrames can pause and seek |
+| **lottie** | `/lottie` | `lottie-web` and dotLottie players registered on `window.__hfLottie` |
+| **three** | `/three` | Three.js scenes rendered from HyperFrames `hf-seek` events |
+| **waapi** | `/waapi` | Web Animations API motion driven through `document.getAnimations()` |
> **Skills not available?** Ask the user to run `npx hyperframes skills` and restart their
> agent session, or install manually: `npx skills add heygen-com/hyperframes`.
@@ -18,9 +25,10 @@
## Commands
```bash
-npx hyperframes preview # preview in browser (studio editor)
-npx hyperframes render # render to MP4
-npx hyperframes lint # validate compositions (errors + warnings)
+npm run dev # preview in browser (studio editor)
+npm run check # lint + validate + inspect
+npm run render # render to MP4
+npm run publish # publish and get a shareable link
npx hyperframes lint --verbose # include info-level findings
npx hyperframes lint --json # machine-readable output for CI
npx hyperframes docs # reference docs in terminal
@@ -51,13 +59,13 @@ https://hyperframes.heygen.com/llms.txt
## Linting — ALWAYS RUN AFTER CHANGES
-After creating or editing any `.html` composition, **always** run the linter before considering the task complete:
+After creating or editing any `.html` composition, **always** run the full check before considering the task complete:
```bash
-npx hyperframes lint
+npm run check
```
-Fix all errors before presenting the result. Warnings are informational and usually safe to ignore.
+Fix all errors before presenting the result. Inspect warnings should be reviewed before rendering.
## Key Rules
diff --git a/packages/cli/src/utils/compositionViewport.test.ts b/packages/cli/src/utils/compositionViewport.test.ts
new file mode 100644
index 000000000..46566f721
--- /dev/null
+++ b/packages/cli/src/utils/compositionViewport.test.ts
@@ -0,0 +1,20 @@
+import { describe, expect, it } from "vitest";
+import { resolveCompositionViewportFromHtml } from "./compositionViewport.js";
+
+describe("resolveCompositionViewportFromHtml", () => {
+ it("uses root composition dimensions", () => {
+ const html = `
+
+
+`;
+
+ expect(resolveCompositionViewportFromHtml(html)).toEqual({ width: 1080, height: 1920 });
+ });
+
+ it("falls back to landscape defaults when dimensions are missing", () => {
+ expect(resolveCompositionViewportFromHtml("")).toEqual({
+ width: 1920,
+ height: 1080,
+ });
+ });
+});
diff --git a/packages/cli/src/utils/compositionViewport.ts b/packages/cli/src/utils/compositionViewport.ts
new file mode 100644
index 000000000..c7a840af9
--- /dev/null
+++ b/packages/cli/src/utils/compositionViewport.ts
@@ -0,0 +1,26 @@
+import { ensureDOMParser } from "./dom.js";
+
+const DEFAULT_VIEWPORT = { width: 1920, height: 1080 } as const;
+const MAX_VIEWPORT_DIMENSION = 4096;
+
+function parseViewportDimension(value: string | null): number | null {
+ if (!value) return null;
+ const parsed = Number.parseInt(value, 10);
+ if (!Number.isFinite(parsed) || parsed <= 0) return null;
+ return Math.min(parsed, MAX_VIEWPORT_DIMENSION);
+}
+
+export function resolveCompositionViewportFromHtml(html: string): {
+ width: number;
+ height: number;
+} {
+ ensureDOMParser();
+ const doc = new DOMParser().parseFromString(html, "text/html");
+ const root = doc.querySelector("[data-composition-id][data-width][data-height]");
+ const width = parseViewportDimension(root?.getAttribute("data-width") ?? null);
+ const height = parseViewportDimension(root?.getAttribute("data-height") ?? null);
+ return {
+ width: width ?? DEFAULT_VIEWPORT.width,
+ height: height ?? DEFAULT_VIEWPORT.height,
+ };
+}
diff --git a/packages/cli/src/utils/dockerRunArgs.test.ts b/packages/cli/src/utils/dockerRunArgs.test.ts
index 287afba65..5877985f1 100644
--- a/packages/cli/src/utils/dockerRunArgs.test.ts
+++ b/packages/cli/src/utils/dockerRunArgs.test.ts
@@ -5,9 +5,9 @@ const BASE: DockerRenderOptions = {
fps: 30,
quality: "standard",
format: "mp4",
- workers: 4,
gpu: false,
- hdr: false,
+ browserGpu: false,
+ hdrMode: "auto",
crf: undefined,
videoBitrate: undefined,
quiet: false,
@@ -43,17 +43,28 @@ describe("buildDockerRunArgs", () => {
"standard",
"--format",
"mp4",
- "--workers",
- "4",
+ "--no-browser-gpu",
]
`);
});
+ it("omits --workers when auto sizing should happen inside the container", () => {
+ const args = buildDockerRunArgs({ ...FIXED_INPUT, options: BASE });
+ expect(args).not.toContain("--workers");
+ });
+
it("matches snapshot when every renderer flag is enabled", () => {
expect(
buildDockerRunArgs({
...FIXED_INPUT,
- options: { ...BASE, gpu: true, hdr: true, crf: 18, videoBitrate: undefined, quiet: true },
+ options: {
+ ...BASE,
+ gpu: true,
+ hdrMode: "force-hdr",
+ crf: 18,
+ videoBitrate: undefined,
+ quiet: true,
+ },
}),
).toMatchInlineSnapshot(`
[
@@ -78,12 +89,11 @@ describe("buildDockerRunArgs", () => {
"standard",
"--format",
"mp4",
- "--workers",
- "4",
"--crf",
"18",
"--quiet",
"--gpu",
+ "--no-browser-gpu",
"--hdr",
]
`);
@@ -92,17 +102,28 @@ describe("buildDockerRunArgs", () => {
// Regression for the original PR feedback: --hdr was silently dropped from
// the docker arg array. Keep this assertion explicit (in addition to the
// snapshot above) so the failure message points directly at the flag.
- it("forwards --hdr to the container when hdr is enabled", () => {
+ it("forwards --hdr to the container when hdrMode is force-hdr", () => {
const args = buildDockerRunArgs({
...FIXED_INPUT,
- options: { ...BASE, hdr: true },
+ options: { ...BASE, hdrMode: "force-hdr" },
});
expect(args).toContain("--hdr");
+ expect(args).not.toContain("--sdr");
});
- it("omits --hdr when hdr is disabled", () => {
+ it("forwards --sdr to the container when hdrMode is force-sdr", () => {
+ const args = buildDockerRunArgs({
+ ...FIXED_INPUT,
+ options: { ...BASE, hdrMode: "force-sdr" },
+ });
+ expect(args).toContain("--sdr");
+ expect(args).not.toContain("--hdr");
+ });
+
+ it("omits --hdr and --sdr when hdrMode is auto", () => {
const args = buildDockerRunArgs({ ...FIXED_INPUT, options: BASE });
expect(args).not.toContain("--hdr");
+ expect(args).not.toContain("--sdr");
});
it("requests host GPU passthrough only when gpu is enabled", () => {
@@ -121,6 +142,11 @@ describe("buildDockerRunArgs", () => {
expect(on).toContain("--gpu");
});
+ it("forces software browser capture inside Docker", () => {
+ const args = buildDockerRunArgs({ ...FIXED_INPUT, options: BASE });
+ expect(args).toContain("--no-browser-gpu");
+ });
+
it("forwards every renderer-shaped option (regression tripwire for silent drops)", () => {
const args = buildDockerRunArgs({
...FIXED_INPUT,
@@ -130,7 +156,8 @@ describe("buildDockerRunArgs", () => {
format: "webm",
workers: 8,
gpu: true,
- hdr: true,
+ browserGpu: false,
+ hdrMode: "force-hdr",
crf: 16,
videoBitrate: undefined,
quiet: true,
@@ -147,6 +174,7 @@ describe("buildDockerRunArgs", () => {
expect(args).toContain("16");
expect(args).toContain("--quiet");
expect(args).toContain("--gpu");
+ expect(args).toContain("--no-browser-gpu");
expect(args).toContain("--hdr");
});
@@ -159,4 +187,27 @@ describe("buildDockerRunArgs", () => {
expect(args).toContain("10M");
expect(args).not.toContain("--crf");
});
+
+ it("forwards --variables JSON to the container when set", () => {
+ const args = buildDockerRunArgs({
+ ...FIXED_INPUT,
+ options: { ...BASE, variables: { title: "Hello", n: 3 } },
+ });
+ const idx = args.indexOf("--variables");
+ expect(idx).toBeGreaterThan(-1);
+ expect(args[idx + 1]).toBe('{"title":"Hello","n":3}');
+ });
+
+ it("omits --variables when none provided", () => {
+ const args = buildDockerRunArgs({ ...FIXED_INPUT, options: BASE });
+ expect(args).not.toContain("--variables");
+ });
+
+ it("omits --variables when payload is empty", () => {
+ const args = buildDockerRunArgs({
+ ...FIXED_INPUT,
+ options: { ...BASE, variables: {} },
+ });
+ expect(args).not.toContain("--variables");
+ });
});
diff --git a/packages/cli/src/utils/dockerRunArgs.ts b/packages/cli/src/utils/dockerRunArgs.ts
index b56b5aef5..5e6303b52 100644
--- a/packages/cli/src/utils/dockerRunArgs.ts
+++ b/packages/cli/src/utils/dockerRunArgs.ts
@@ -22,12 +22,14 @@ export interface DockerRenderOptions {
fps: 24 | 30 | 60;
quality: "draft" | "standard" | "high";
format: "mp4" | "webm" | "mov";
- workers: number;
+ workers?: number;
gpu: boolean;
- hdr: boolean;
+ browserGpu: boolean;
+ hdrMode: "auto" | "force-hdr" | "force-sdr";
crf?: number;
videoBitrate?: string;
quiet: boolean;
+ variables?: Record;
}
export function buildDockerRunArgs(input: DockerRunArgsInput): string[] {
@@ -54,12 +56,16 @@ export function buildDockerRunArgs(input: DockerRunArgsInput): string[] {
options.quality,
"--format",
options.format,
- "--workers",
- String(options.workers),
+ ...(options.workers != null ? ["--workers", String(options.workers)] : []),
...(options.crf != null ? ["--crf", String(options.crf)] : []),
...(options.videoBitrate ? ["--video-bitrate", options.videoBitrate] : []),
...(options.quiet ? ["--quiet"] : []),
...(options.gpu ? ["--gpu"] : []),
- ...(options.hdr ? ["--hdr"] : []),
+ ...(options.browserGpu ? [] : ["--no-browser-gpu"]),
+ ...(options.hdrMode === "force-hdr" ? ["--hdr"] : []),
+ ...(options.hdrMode === "force-sdr" ? ["--sdr"] : []),
+ ...(options.variables && Object.keys(options.variables).length > 0
+ ? ["--variables", JSON.stringify(options.variables)]
+ : []),
];
}
diff --git a/packages/cli/src/utils/lintProject.test.ts b/packages/cli/src/utils/lintProject.test.ts
index bc0364838..3c72febc4 100644
--- a/packages/cli/src/utils/lintProject.test.ts
+++ b/packages/cli/src/utils/lintProject.test.ts
@@ -99,6 +99,28 @@ describe("lintProject", () => {
expect(subFindings.some((f) => f.code === "media_missing_id")).toBe(true);
});
+ it("lints linked CSS next to sub-compositions", () => {
+ const project = makeProject(validHtml(), {
+ "scene.html": `
+
+`,
+ });
+ writeFileSync(
+ join(project.dir, "compositions", "scene.css"),
+ '[data-composition-id="scene"] .title { opacity: 0; }',
+ );
+
+ const { results } = lintProject(project);
+ const subResult = results.find((result) => result.file === "compositions/scene.html");
+ const finding = subResult?.result.findings.find(
+ (item) => item.code === "composition_self_attribute_selector",
+ );
+
+ expect(finding).toBeDefined();
+ expect(finding?.selector).toBe('[data-composition-id="scene"] .title');
+ });
+
it("aggregates errors across index.html and sub-compositions", () => {
const project = makeProject(htmlWithMissingMediaId(), {
"overlay.html": htmlWithMissingMediaId(),
diff --git a/packages/cli/src/utils/lintProject.ts b/packages/cli/src/utils/lintProject.ts
index c119a141f..45eef143c 100644
--- a/packages/cli/src/utils/lintProject.ts
+++ b/packages/cli/src/utils/lintProject.ts
@@ -1,5 +1,5 @@
import { existsSync, readFileSync, readdirSync } from "node:fs";
-import { join, resolve, extname } from "node:path";
+import { dirname, join, resolve, extname } from "node:path";
import { lintHyperframeHtml, type HyperframeLintResult } from "@hyperframes/core/lint";
import type { HyperframeLintFinding } from "@hyperframes/core/lint";
import { rewriteAssetPath } from "@hyperframes/core";
@@ -27,6 +27,32 @@ export interface ProjectLintResult {
const AUDIO_EXTENSIONS = new Set([".mp3", ".wav", ".aac", ".ogg", ".m4a", ".flac", ".opus"]);
+function isLocalStylesheetHref(href: string): boolean {
+ return !!href && !/^(https?:|data:|blob:|\/\/)/i.test(href);
+}
+
+function collectExternalStyles(
+ projectDir: string,
+ html: string,
+ compSrcPath?: string,
+): Array<{ href: string; content: string }> {
+ const styles: Array<{ href: string; content: string }> = [];
+ const linkRe = /]*>/gi;
+ let match: RegExpExecArray | null;
+ while ((match = linkRe.exec(html)) !== null) {
+ const tag = match[0];
+ const rel = tag.match(/\brel\s*=\s*["']([^"']+)["']/i)?.[1] ?? "";
+ if (!rel.split(/\s+/).some((part) => part.toLowerCase() === "stylesheet")) continue;
+ const href = tag.match(/\bhref\s*=\s*["']([^"']+)["']/i)?.[1] ?? "";
+ if (!isLocalStylesheetHref(href)) continue;
+ const rootRelative = compSrcPath ? join(dirname(compSrcPath), href) : href;
+ const resolved = resolve(projectDir, rootRelative);
+ if (!existsSync(resolved)) continue;
+ styles.push({ href, content: readFileSync(resolved, "utf-8") });
+ }
+ return styles;
+}
+
/**
* Lint the root index.html and all sub-compositions in the compositions/ directory.
* Returns aggregated results across all files.
@@ -39,7 +65,10 @@ export function lintProject(project: ProjectDir): ProjectLintResult {
// Lint root composition
const rootHtml = readFileSync(project.indexPath, "utf-8");
- const rootResult = lintHyperframeHtml(rootHtml, { filePath: project.indexPath });
+ const rootResult = lintHyperframeHtml(rootHtml, {
+ filePath: project.indexPath,
+ externalStyles: collectExternalStyles(project.dir, rootHtml),
+ });
results.push({ file: "index.html", result: rootResult });
totalErrors += rootResult.errorCount;
totalWarnings += rootResult.warningCount;
@@ -53,8 +82,13 @@ export function lintProject(project: ProjectDir): ProjectLintResult {
for (const file of files) {
const filePath = join(compositionsDir, file);
const html = readFileSync(filePath, "utf-8");
- allHtmlSources.push({ html, compSrcPath: `compositions/${file}` });
- const result = lintHyperframeHtml(html, { filePath, isSubComposition: true });
+ const compSrcPath = `compositions/${file}`;
+ allHtmlSources.push({ html, compSrcPath });
+ const result = lintHyperframeHtml(html, {
+ filePath,
+ isSubComposition: true,
+ externalStyles: collectExternalStyles(project.dir, html, compSrcPath),
+ });
results.push({ file: `compositions/${file}`, result });
totalErrors += result.errorCount;
totalWarnings += result.warningCount;
diff --git a/packages/cli/src/utils/publishProject.test.ts b/packages/cli/src/utils/publishProject.test.ts
index b5a3b8117..4764e0409 100644
--- a/packages/cli/src/utils/publishProject.test.ts
+++ b/packages/cli/src/utils/publishProject.test.ts
@@ -7,6 +7,7 @@ import {
createPublishArchive,
getPublishApiBaseUrl,
publishProjectArchive,
+ uploadTimeoutMs,
} from "./publishProject.js";
function makeProjectDir(): string {
@@ -35,6 +36,22 @@ describe("createPublishArchive", () => {
});
});
+describe("uploadTimeoutMs", () => {
+ it("returns the minimum timeout for small files", () => {
+ expect(uploadTimeoutMs(0)).toBe(120_000);
+ expect(uploadTimeoutMs(50 * 1024 * 1024)).toBe(120_000);
+ });
+
+ it("scales above the floor for large files", () => {
+ expect(uploadTimeoutMs(64 * 1024 * 1024)).toBeGreaterThan(120_000);
+ expect(uploadTimeoutMs(500 * 1024 * 1024)).toBeGreaterThan(900_000);
+ });
+
+ it("returns an integer", () => {
+ expect(Number.isInteger(uploadTimeoutMs(123_456))).toBe(true);
+ });
+});
+
describe("publishProjectArchive", () => {
beforeEach(() => {
vi.stubEnv("HYPERFRAMES_PUBLISHED_PROJECTS_API_URL", "");
diff --git a/packages/cli/src/utils/publishProject.ts b/packages/cli/src/utils/publishProject.ts
index 3b4d62855..9a317b619 100644
--- a/packages/cli/src/utils/publishProject.ts
+++ b/packages/cli/src/utils/publishProject.ts
@@ -5,7 +5,11 @@ import AdmZip from "adm-zip";
const IGNORED_DIRS = new Set([".git", "node_modules", "dist", ".next", "coverage"]);
const IGNORED_FILES = new Set([".DS_Store", "Thumbs.db"]);
const PUBLISH_CONTENT_TYPE = "application/zip";
-const PUBLISH_REQUEST_TIMEOUT_MS = 30_000;
+const PUBLISH_METADATA_TIMEOUT_MS = 30_000;
+const PUBLISH_UPLOAD_MIN_TIMEOUT_MS = 120_000;
+// Conservative floor — most connections are faster, but this prevents
+// premature aborts on slow/unstable networks (hotel wifi, tethering).
+const PUBLISH_UPLOAD_BYTES_PER_SECOND = 500_000;
export interface PublishArchiveResult {
buffer: Buffer;
@@ -25,6 +29,7 @@ interface StagedUploadResponse {
uploadKey: string;
contentType: string;
uploadHeaders: Record;
+ expiresInSeconds: number;
}
type JsonRecord = Record;
@@ -73,11 +78,14 @@ function parseStagedUploadResponse(
const uploadKey = stringField(data, "upload_key");
const contentType = stringField(data, "content_type") || PUBLISH_CONTENT_TYPE;
if (!uploadUrl || !uploadKey) return null;
+ const rawExpires = data["expires_in_seconds"];
+ const expiresInSeconds = typeof rawExpires === "number" && rawExpires > 0 ? rawExpires : 1800;
return {
uploadUrl,
uploadKey,
contentType,
uploadHeaders: getUploadHeaders(data, uploadUrl, contentType, archiveByteLength),
+ expiresInSeconds,
};
}
@@ -142,6 +150,13 @@ async function readErrorMessage(response: Response, fallback: string): Promise Promise;
+}
+
+export async function serveStaticProjectHtml(
+ projectDir: string,
+ html: string,
+ bindErrorMessage = "Failed to bind local HTTP server",
+): Promise {
+ const server = createServer((req, res) => {
+ const url = req.url ?? "/";
+ if (url === "/" || url === "/index.html") {
+ res.writeHead(200, { "Content-Type": "text/html" });
+ res.end(html);
+ return;
+ }
+
+ const filePath = resolve(projectDir, decodeURIComponent(url).replace(/^\//, ""));
+ const rel = relative(projectDir, filePath);
+ if (rel.startsWith("..") || isAbsolute(rel)) {
+ res.writeHead(403);
+ res.end();
+ return;
+ }
+ if (existsSync(filePath)) {
+ res.writeHead(200, { "Content-Type": getMimeType(filePath) });
+ res.end(readFileSync(filePath));
+ return;
+ }
+ res.writeHead(404);
+ res.end();
+ });
+
+ const port = await new Promise((resolvePort, rejectPort) => {
+ server.on("error", rejectPort);
+ server.listen(0, () => {
+ const addr = server.address();
+ const resolvedPort = typeof addr === "object" && addr ? addr.port : 0;
+ if (!resolvedPort) rejectPort(new Error(bindErrorMessage));
+ else resolvePort(resolvedPort);
+ });
+ });
+
+ return {
+ url: `http://127.0.0.1:${port}/`,
+ port,
+ close: () =>
+ new Promise((resolveClose) => {
+ server.close(() => resolveClose());
+ }),
+ };
+}
diff --git a/packages/cli/src/whisper/manager.ts b/packages/cli/src/whisper/manager.ts
index 4a1237a2a..4866c6c60 100644
--- a/packages/cli/src/whisper/manager.ts
+++ b/packages/cli/src/whisper/manager.ts
@@ -205,8 +205,16 @@ export async function ensureModel(
}
export function hasFFmpeg(): boolean {
+ return hasBinary("ffmpeg");
+}
+
+export function hasFFprobe(): boolean {
+ return hasBinary("ffprobe");
+}
+
+function hasBinary(name: string): boolean {
try {
- execFileSync("ffmpeg", ["-version"], { stdio: "ignore", timeout: 5000 });
+ execFileSync(name, ["-version"], { stdio: "ignore", timeout: 5000 });
return true;
} catch {
return false;
diff --git a/packages/core/package.json b/packages/core/package.json
index 4c98851d0..bd88700f4 100644
--- a/packages/core/package.json
+++ b/packages/core/package.json
@@ -1,6 +1,6 @@
{
"name": "@hyperframes/core",
- "version": "0.4.27",
+ "version": "0.4.45",
"description": "",
"repository": {
"type": "git",
@@ -30,10 +30,18 @@
"types": "./src/compiler/index.ts"
},
"./runtime": "./dist/hyperframe.runtime.iife.js",
+ "./runtime/lottie-readiness": {
+ "import": "./src/runtime/adapters/lottieReadiness.ts",
+ "types": "./src/runtime/adapters/lottieReadiness.ts"
+ },
"./studio-api": {
"import": "./src/studio-api/index.ts",
"types": "./src/studio-api/index.ts"
},
+ "./studio-api/screenshot-clip": {
+ "import": "./src/studio-api/helpers/screenshotClip.ts",
+ "types": "./src/studio-api/helpers/screenshotClip.ts"
+ },
"./text": {
"import": "./src/text/index.ts",
"types": "./src/text/index.ts"
@@ -77,10 +85,18 @@
"types": "./dist/compiler/index.d.ts"
},
"./runtime": "./dist/hyperframe.runtime.iife.js",
+ "./runtime/lottie-readiness": {
+ "import": "./dist/runtime/adapters/lottieReadiness.js",
+ "types": "./dist/runtime/adapters/lottieReadiness.d.ts"
+ },
"./studio-api": {
"import": "./dist/studio-api/index.js",
"types": "./dist/studio-api/index.d.ts"
},
+ "./studio-api/screenshot-clip": {
+ "import": "./dist/studio-api/helpers/screenshotClip.js",
+ "types": "./dist/studio-api/helpers/screenshotClip.d.ts"
+ },
"./text": {
"import": "./dist/text/index.js",
"types": "./dist/text/index.d.ts"
@@ -135,6 +151,7 @@
},
"dependencies": {
"@chenglou/pretext": "^0.0.5",
+ "postcss": "^8.5.8",
"sharp": "^0.34.5"
},
"devDependencies": {
diff --git a/packages/core/schemas/registry-item.json b/packages/core/schemas/registry-item.json
index ba72da93c..a2051a5bf 100644
--- a/packages/core/schemas/registry-item.json
+++ b/packages/core/schemas/registry-item.json
@@ -35,6 +35,14 @@
"type": "string",
"minLength": 1
},
+ "authorUrl": {
+ "type": "string",
+ "format": "uri"
+ },
+ "sourcePrompt": {
+ "type": "string",
+ "minLength": 1
+ },
"license": {
"type": "string",
"minLength": 1,
diff --git a/packages/core/src/compiler/compositionScoping.test.ts b/packages/core/src/compiler/compositionScoping.test.ts
new file mode 100644
index 000000000..4c1a0189b
--- /dev/null
+++ b/packages/core/src/compiler/compositionScoping.test.ts
@@ -0,0 +1,304 @@
+import { describe, expect, it, vi } from "vitest";
+import { parseHTML } from "linkedom";
+import { scopeCssToComposition, wrapScopedCompositionScript } from "./compositionScoping";
+
+describe("composition scoping", () => {
+ it("scopes regular selectors while preserving global at-rules", () => {
+ const scoped = scopeCssToComposition(
+ `
+@import url("https://example.com/font.css");
+.title, .card:hover { opacity: 0; }
+@media (min-width: 800px) {
+ .title { transform: translateY(30px); }
+}
+@keyframes rise {
+ from { opacity: 0; }
+ to { opacity: 1; }
+}
+[data-composition-id="scene"] .already { color: red; }
+body { margin: 0; }
+`,
+ "scene",
+ );
+
+ expect(scoped).toContain('@import url("https://example.com/font.css");');
+ expect(scoped).toContain(
+ '[data-composition-id="scene"] .title, [data-composition-id="scene"] .card:hover',
+ );
+ expect(scoped).toContain('[data-composition-id="scene"] .title { transform');
+ expect(scoped).toContain("@keyframes rise");
+ expect(scoped).toContain("from { opacity: 0; }");
+ expect(scoped).toContain('[data-composition-id="scene"] .already { color: red; }');
+ expect(scoped).toContain("body { margin: 0; }");
+ });
+
+ it("wraps classic scripts without render-loop requestAnimationFrame waits", () => {
+ const wrapped = wrapScopedCompositionScript("window.__ran = true;", "scene");
+
+ expect(wrapped).toContain('var __hfCompId = "scene";');
+ expect(wrapped).toContain("new Proxy(window.document");
+ expect(wrapped).toContain("new Proxy(__hfBaseGsap");
+ expect(wrapped).not.toContain("requestAnimationFrame");
+ });
+
+ it("normalizes root timing attributes when scoping selectors", () => {
+ const scoped = scopeCssToComposition(
+ '[data-composition-id="scene"][data-start="0"] .title { opacity: 0; }',
+ "scene",
+ );
+
+ expect(scoped).toContain('[data-composition-id="scene"] .title { opacity: 0; }');
+ expect(scoped).not.toContain('[data-start="0"]');
+ });
+
+ it("exposes a scoped __hyperframes.getVariables that reads __hfVariablesByComp[compId]", () => {
+ const { document } = parseHTML(`
`);
+ const fakeWindow: Record = {
+ document,
+ __timelines: {},
+ __hfVariablesByComp: {
+ "card-1": { title: "Pro", price: "$29" },
+ "card-2": { title: "Enterprise", price: "Custom" },
+ },
+ __hyperframes: {
+ getVariables: () => ({ title: "TOP-LEVEL-LEAK" }),
+ fitTextFontSize: () => undefined,
+ },
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `window.__captured = __hyperframes.getVariables();`,
+ "card-1",
+ );
+
+ new Function("window", wrapped)(fakeWindow);
+
+ expect(fakeWindow.__captured).toEqual({ title: "Pro", price: "$29" });
+ });
+
+ it("scoped getVariables returns {} when __hfVariablesByComp has no entry for the comp", () => {
+ const { document } = parseHTML(`
`);
+ const fakeWindow: Record = {
+ document,
+ __timelines: {},
+ __hyperframes: {
+ getVariables: () => ({ title: "TOP-LEVEL-LEAK" }),
+ fitTextFontSize: () => undefined,
+ },
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `window.__captured = __hyperframes.getVariables();`,
+ "missing",
+ );
+
+ new Function("window", wrapped)(fakeWindow);
+
+ expect(fakeWindow.__captured).toEqual({});
+ });
+
+ it("scoped getVariables returns a fresh object — mutations don't leak into the shared table", () => {
+ const { document } = parseHTML(`
`);
+ const variablesByComp: Record> = {
+ "card-1": { title: "Pro" },
+ };
+ const fakeWindow: Record = {
+ document,
+ __timelines: {},
+ __hfVariablesByComp: variablesByComp,
+ __hyperframes: {
+ getVariables: () => ({}),
+ fitTextFontSize: () => undefined,
+ },
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `var v = __hyperframes.getVariables(); v.title = "MUTATED"; v.added = "extra";`,
+ "card-1",
+ );
+
+ new Function("window", wrapped)(fakeWindow);
+
+ expect(variablesByComp["card-1"]).toEqual({ title: "Pro" });
+ });
+
+ it("executes document and GSAP selectors inside the composition root", () => {
+ const { document } = parseHTML(`
+
Scene
+
Other
+ `);
+ const gsapTargets: string[][] = [];
+ const fakeWindow = {
+ document,
+ __selectedTitle: "",
+ __selectedRootTitle: "",
+ __timelines: {},
+ gsap: {
+ timeline: () => ({
+ to(targets: Element[]) {
+ gsapTargets.push(Array.from(targets).map((target) => target.textContent || ""));
+ return this;
+ },
+ }),
+ },
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `
+const tl = gsap.timeline({ paused: true });
+tl.to('.title', { opacity: 1 });
+tl.to('[data-composition-id="scene"][data-start="0"] .title', { opacity: 1 });
+window.__selectedTitle = document.querySelector('.title')?.textContent || '';
+window.__selectedRootTitle = document.querySelector('[data-composition-id="scene"][data-start="0"] .title')?.textContent || '';
+window.__timelines.scene = tl;
+`,
+ "scene",
+ );
+
+ new Function("window", "gsap", wrapped)(fakeWindow, fakeWindow.gsap);
+
+ expect(fakeWindow.__selectedTitle).toBe("Scene");
+ expect(fakeWindow.__selectedRootTitle).toBe("Scene");
+ expect(gsapTargets).toEqual([["Scene"], ["Scene"]]);
+ });
+
+ it("reads scoped proxy accessors with the original target receiver", () => {
+ const root = {
+ contains(node: unknown) {
+ return node === root;
+ },
+ };
+ const body = { tagName: "BODY" };
+ const fakeDocument = {
+ querySelector(selector: string) {
+ return selector === '[data-composition-id="scene"]' ? root : null;
+ },
+ querySelectorAll() {
+ return [];
+ },
+ getElementById() {
+ return null;
+ },
+ get body() {
+ if (this !== fakeDocument) {
+ throw new TypeError("Illegal invocation");
+ }
+ return body;
+ },
+ };
+ const location = { href: "https://example.test/scene" };
+ const fakeUtils = {
+ get marker() {
+ if (this !== fakeUtils) {
+ throw new TypeError("Illegal invocation");
+ }
+ return "utils-ok";
+ },
+ };
+ const fakeGsap = {
+ utils: fakeUtils,
+ get version() {
+ if (this !== fakeGsap) {
+ throw new TypeError("Illegal invocation");
+ }
+ return "gsap-ok";
+ },
+ };
+ const fakeWindow = {
+ document: fakeDocument,
+ __bodyTag: "",
+ __href: "",
+ __windowSet: "",
+ __gsapVersion: "",
+ __utilsMarker: "",
+ __timelines: {},
+ gsap: fakeGsap,
+ get location() {
+ if (this !== fakeWindow) {
+ throw new TypeError("Illegal invocation");
+ }
+ return location;
+ },
+ set customValue(value: string) {
+ if (this !== fakeWindow) {
+ throw new TypeError("Illegal invocation");
+ }
+ this.__windowSet = value;
+ },
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `
+window.__bodyTag = document.body.tagName;
+window.__href = window.location.href;
+window.customValue = "window-set-ok";
+window.__gsapVersion = gsap.version;
+window.__utilsMarker = gsap.utils.marker;
+`,
+ "scene",
+ );
+ const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
+
+ try {
+ new Function("window", "gsap", wrapped)(fakeWindow, fakeWindow.gsap);
+ } finally {
+ errorSpy.mockRestore();
+ }
+
+ expect(fakeWindow.__bodyTag).toBe("BODY");
+ expect(fakeWindow.__href).toBe("https://example.test/scene");
+ expect(fakeWindow.__windowSet).toBe("window-set-ok");
+ expect(fakeWindow.__gsapVersion).toBe("gsap-ok");
+ expect(fakeWindow.__utilsMarker).toBe("utils-ok");
+ expect(errorSpy).not.toHaveBeenCalled();
+ });
+
+ it("reads remapped timeline registry accessors with the original target receiver", () => {
+ let timeline = "initial";
+ const timelineRegistry = {
+ get host() {
+ if (this !== timelineRegistry) {
+ throw new TypeError("Illegal invocation");
+ }
+ return timeline;
+ },
+ set host(value: string) {
+ if (this !== timelineRegistry) {
+ throw new TypeError("Illegal invocation");
+ }
+ timeline = value;
+ },
+ };
+ const fakeWindow = {
+ document: {
+ querySelector() {
+ return null;
+ },
+ querySelectorAll() {
+ return [];
+ },
+ },
+ __timelines: timelineRegistry,
+ __beforeTimeline: "",
+ __afterTimeline: "",
+ gsap: {},
+ };
+ const wrapped = wrapScopedCompositionScript(
+ `
+window.__beforeTimeline = window.__timelines.scene;
+window.__timelines.scene = "updated";
+window.__afterTimeline = window.__timelines.scene;
+`,
+ "scene",
+ "[HyperFrames] composition script error:",
+ undefined,
+ "host",
+ );
+ const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
+
+ try {
+ new Function("window", "gsap", wrapped)(fakeWindow, fakeWindow.gsap);
+ } finally {
+ errorSpy.mockRestore();
+ }
+
+ expect(fakeWindow.__beforeTimeline).toBe("initial");
+ expect(fakeWindow.__afterTimeline).toBe("updated");
+ expect(errorSpy).not.toHaveBeenCalled();
+ });
+});
diff --git a/packages/core/src/compiler/compositionScoping.ts b/packages/core/src/compiler/compositionScoping.ts
new file mode 100644
index 000000000..bc7153011
--- /dev/null
+++ b/packages/core/src/compiler/compositionScoping.ts
@@ -0,0 +1,290 @@
+import postcss, { type AtRule, type Node, type Rule } from "postcss";
+
+function escapeRegExp(value: string): string {
+ return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
+
+function escapeCssAttributeValue(value: string): string {
+ return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
+}
+
+function scopeSelector(selector: string, scope: string, compositionId: string): string {
+ const selectorWithoutRootTiming = normalizeCompositionRootSelector(
+ selector,
+ scope,
+ compositionId,
+ );
+ const trimmed = selectorWithoutRootTiming.trim();
+ if (!trimmed) return selector;
+ if (/^(html|body|:root|\*)$/i.test(trimmed)) return selector;
+ const compositionIdPattern = new RegExp(
+ `\\[\\s*data-composition-id\\s*=\\s*(["'])${escapeRegExp(compositionId)}\\1\\s*\\]`,
+ "g",
+ );
+ if (compositionIdPattern.test(trimmed)) {
+ return selectorWithoutRootTiming.replace(compositionIdPattern, scope);
+ }
+ const leading = selectorWithoutRootTiming.match(/^\s*/)?.[0] ?? "";
+ const trailing = selectorWithoutRootTiming.match(/\s*$/)?.[0] ?? "";
+ return `${leading}${scope} ${trimmed}${trailing}`;
+}
+
+function normalizeCompositionRootSelector(
+ selector: string,
+ scope: string,
+ compositionId: string,
+): string {
+ const quotedCompId = escapeRegExp(compositionId);
+ const compAttr = String.raw`\[\s*data-composition-id\s*=\s*(?:"${quotedCompId}"|'${quotedCompId}')\s*\]`;
+ const timingAttr = String.raw`\s*\[\s*data-(?:start|duration)\s*=\s*(?:"[^"]*"|'[^']*')\s*\]`;
+ return selector
+ .replace(new RegExp(`${compAttr}(?:${timingAttr})+`, "g"), scope)
+ .replace(new RegExp(`(?:${timingAttr})+${compAttr}`, "g"), scope);
+}
+
+const GLOBAL_AT_RULES = new Set(["keyframes", "-webkit-keyframes", "font-face"]);
+
+function isAtRuleNode(node: Node["parent"]): node is AtRule {
+ return node?.type === "atrule";
+}
+
+function isInsideGlobalAtRule(rule: Rule): boolean {
+ let current: Node["parent"] = rule.parent;
+ while (current) {
+ if (isAtRuleNode(current) && GLOBAL_AT_RULES.has(current.name.toLowerCase())) {
+ return true;
+ }
+ current = current.parent;
+ }
+ return false;
+}
+
+export function scopeCssToComposition(
+ css: string,
+ compositionId: string,
+ scopeSelectorOverride?: string,
+): string {
+ const trimmedCompositionId = compositionId.trim();
+ if (!css || !trimmedCompositionId) return css;
+ const scope =
+ scopeSelectorOverride ||
+ `[data-composition-id="${escapeCssAttributeValue(trimmedCompositionId)}"]`;
+ const root = postcss.parse(css);
+
+ root.walkRules((rule) => {
+ if (isInsideGlobalAtRule(rule)) return;
+ rule.selectors = rule.selectors.map((selector) =>
+ scopeSelector(selector, scope, trimmedCompositionId),
+ );
+ });
+
+ return root.toResult({ map: false }).css;
+}
+
+export function wrapScopedCompositionScript(
+ source: string,
+ compositionId: string,
+ errorLabel = "[HyperFrames] composition script error:",
+ scopeSelectorOverride?: string,
+ timelineCompositionId = compositionId,
+): string {
+ const compositionIdLiteral = JSON.stringify(compositionId);
+ const timelineCompositionIdLiteral = JSON.stringify(timelineCompositionId);
+ const errorLabelLiteral = JSON.stringify(errorLabel);
+ const escapedCompositionId = escapeRegExp(compositionId);
+ const scopeSelectorLiteral = JSON.stringify(scopeSelectorOverride ?? null);
+ const rootSelectorPatternLiteral = JSON.stringify(
+ String.raw`\[\s*data-composition-id\s*=\s*(?:"${escapedCompositionId}"|'${escapedCompositionId}')\s*\]`,
+ );
+ const timingSelectorPatternLiteral = JSON.stringify(
+ String.raw`\s*\[\s*data-(?:start|duration)\s*=\s*(?:"[^"]*"|'[^']*')\s*\]`,
+ );
+ return `(function(){
+ var __hfCompId = ${compositionIdLiteral};
+ var __hfTimelineCompId = ${timelineCompositionIdLiteral};
+ var __hfErrorLabel = ${errorLabelLiteral};
+ var __hfEscapeAttr = function(value) {
+ return (value + "").replace(/\\\\/g, "\\\\\\\\").replace(/"/g, "\\\\\\"");
+ };
+ var __hfRootSelector = ${scopeSelectorLiteral} || (__hfCompId
+ ? '[data-composition-id="' + __hfEscapeAttr(__hfCompId) + '"]'
+ : "");
+ var __hfRoot = null;
+ var __hfRootSelectorPattern = ${rootSelectorPatternLiteral};
+ var __hfTimingSelectorPattern = ${timingSelectorPatternLiteral};
+ var __hfNormalizeSelector = function(selector) {
+ if (!__hfCompId || typeof selector !== "string") return selector;
+ return selector
+ .replace(new RegExp(__hfRootSelectorPattern + '(?:' + __hfTimingSelectorPattern + ')+', 'g'), __hfRootSelector)
+ .replace(new RegExp('(?:' + __hfTimingSelectorPattern + ')+' + __hfRootSelectorPattern, 'g'), __hfRootSelector);
+ };
+ var __hfFindRoot = function() {
+ if (!__hfRoot && __hfRootSelector) {
+ __hfRoot = window.document.querySelector(__hfRootSelector);
+ }
+ return __hfRoot;
+ };
+ var __hfContains = function(node) {
+ var root = __hfFindRoot();
+ return !root || node === root || root.contains(node);
+ };
+ var __hfQueryAll = function(selector) {
+ var root = __hfFindRoot();
+ if (!root || typeof selector !== "string") {
+ return window.document.querySelectorAll(selector);
+ }
+ return Array.prototype.filter.call(window.document.querySelectorAll(__hfNormalizeSelector(selector)), function(node) {
+ return __hfContains(node);
+ });
+ };
+ var __hfQueryOne = function(selector) {
+ var matches = __hfQueryAll(selector);
+ return matches[0] || null;
+ };
+ var __hfScopedDocument = typeof Proxy === "function"
+ ? new Proxy(window.document, {
+ get: function(target, prop, receiver) {
+ if (prop === "querySelector") return __hfQueryOne;
+ if (prop === "querySelectorAll") return __hfQueryAll;
+ if (prop === "getElementById") {
+ return function(id) {
+ var found = target.getElementById(id);
+ return found && __hfContains(found) ? found : null;
+ };
+ }
+ var value = Reflect.get(target, prop, target);
+ return typeof value === "function" ? value.bind(target) : value;
+ },
+ })
+ : window.document;
+ var __hfTimelineRegistryProxy = null;
+ var __hfGetTimelineRegistry = function() {
+ window.__timelines = window.__timelines || {};
+ if (!__hfCompId || __hfCompId === __hfTimelineCompId || typeof Proxy !== "function") {
+ return window.__timelines;
+ }
+ if (!__hfTimelineRegistryProxy) {
+ __hfTimelineRegistryProxy = new Proxy(window.__timelines, {
+ get: function(target, prop, receiver) {
+ return Reflect.get(target, prop === __hfCompId ? __hfTimelineCompId : prop, target);
+ },
+ set: function(target, prop, value, receiver) {
+ return Reflect.set(target, prop === __hfCompId ? __hfTimelineCompId : prop, value, target);
+ },
+ });
+ }
+ return __hfTimelineRegistryProxy;
+ };
+ var __hfScopedWindow = typeof Proxy === "function"
+ ? new Proxy(window, {
+ get: function(target, prop, receiver) {
+ if (prop === "__timelines") return __hfGetTimelineRegistry();
+ var value = Reflect.get(target, prop, target);
+ return typeof value === "function" ? value.bind(target) : value;
+ },
+ set: function(target, prop, value, receiver) {
+ if (prop === "__timelines") {
+ target.__timelines = value || {};
+ __hfTimelineRegistryProxy = null;
+ return true;
+ }
+ return Reflect.set(target, prop, value, target);
+ },
+ })
+ : window;
+ var __hfResolveGsapTarget = function(target) {
+ if (typeof target !== "string") return target;
+ return __hfQueryAll(target);
+ };
+ var __hfScopeTimeline = function(timeline) {
+ if (!timeline || timeline.__hfScopedCompositionRoot === __hfFindRoot()) return timeline;
+ ["to", "from", "fromTo", "set"].forEach(function(method) {
+ var original = timeline[method];
+ if (typeof original !== "function") return;
+ timeline[method] = function(target) {
+ var args = Array.prototype.slice.call(arguments);
+ args[0] = __hfResolveGsapTarget(target);
+ return original.apply(timeline, args);
+ };
+ });
+ try {
+ Object.defineProperty(timeline, "__hfScopedCompositionRoot", {
+ value: __hfFindRoot(),
+ configurable: true,
+ });
+ } catch {
+ // Best-effort: timelines coming from user code may have a frozen target
+ // or a non-extensible defineProperty path. Swallow — the scoped root
+ // is an enrichment, not a correctness invariant for playback.
+ }
+ return timeline;
+ };
+ var __hfBaseGsap = typeof gsap === "undefined" ? window.gsap : gsap;
+ var __hfScopedGsap = !__hfBaseGsap || typeof Proxy !== "function"
+ ? __hfBaseGsap
+ : new Proxy(__hfBaseGsap, {
+ get: function(target, prop, receiver) {
+ if (prop === "timeline") {
+ return function() {
+ return __hfScopeTimeline(target.timeline.apply(target, arguments));
+ };
+ }
+ if (prop === "to" || prop === "from" || prop === "fromTo" || prop === "set") {
+ return function(firstArg) {
+ var args = Array.prototype.slice.call(arguments);
+ args[0] = __hfResolveGsapTarget(firstArg);
+ return target[prop].apply(target, args);
+ };
+ }
+ if (prop === "utils" && target.utils && typeof Proxy === "function") {
+ return new Proxy(target.utils, {
+ get: function(utilsTarget, utilsProp, utilsReceiver) {
+ if (utilsProp === "toArray") {
+ return function(firstArg) {
+ var args = Array.prototype.slice.call(arguments);
+ args[0] = __hfResolveGsapTarget(firstArg);
+ return utilsTarget.toArray.apply(utilsTarget, args);
+ };
+ }
+ if (utilsProp === "selector") {
+ return function(base) {
+ var baseEl = typeof base === "string" ? __hfQueryOne(base) : base;
+ var root = baseEl || __hfFindRoot();
+ return function(selector) {
+ if (!root || typeof selector !== "string") return [];
+ return Array.prototype.slice.call(root.querySelectorAll(selector));
+ };
+ };
+ }
+ var value = Reflect.get(utilsTarget, utilsProp, utilsTarget);
+ return typeof value === "function" ? value.bind(utilsTarget) : value;
+ },
+ });
+ }
+ var value = Reflect.get(target, prop, target);
+ return typeof value === "function" ? value.bind(target) : value;
+ },
+ });
+ var __hfBaseHyperframes = window.__hyperframes;
+ var __hfScopedHyperframes = !__hfBaseHyperframes
+ ? __hfBaseHyperframes
+ : Object.assign({}, __hfBaseHyperframes, {
+ getVariables: function() {
+ var byComp = window.__hfVariablesByComp;
+ var scoped = byComp && __hfCompId ? byComp[__hfCompId] : null;
+ return scoped ? Object.assign({}, scoped) : {};
+ },
+ });
+ var __hfRun = function() {
+ try {
+ (function(document, gsap, window, __hyperframes) {
+${source}
+ }).call(window, __hfScopedDocument, __hfScopedGsap, __hfScopedWindow, __hfScopedHyperframes);
+ } catch (_err) {
+ console.error(__hfErrorLabel, __hfCompId, _err);
+ }
+ };
+ __hfFindRoot();
+ __hfRun();
+})();`;
+}
diff --git a/packages/core/src/compiler/htmlBundler.test.ts b/packages/core/src/compiler/htmlBundler.test.ts
index 118cd9a45..665ff9811 100644
--- a/packages/core/src/compiler/htmlBundler.test.ts
+++ b/packages/core/src/compiler/htmlBundler.test.ts
@@ -2,6 +2,7 @@
import { mkdtempSync, writeFileSync, mkdirSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
+import { parseHTML } from "linkedom";
import { describe, it, expect } from "vitest";
import { bundleToSingleHtml } from "./htmlBundler";
@@ -16,6 +17,137 @@ function makeTempProject(files: Record): string {
}
describe("bundleToSingleHtml", () => {
+ it("does not merge author scripts into the runtime bootstrap placeholder", async () => {
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+
+`,
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+ const runtimeBlock = bundled.match(
+ /]*data-hyperframes-preview-runtime[^>]*>[\s\S]*?<\/script>/i,
+ )?.[0];
+
+ expect(runtimeBlock).toBeDefined();
+ // The runtime block must contain the inlined HF runtime IIFE — bundled
+ // output is self-contained, so the bundle's runtime body is loaded inline,
+ // not referenced via src.
+ expect(runtimeBlock).toMatch(/data-hyperframes-preview-runtime="1">/);
+ expect(runtimeBlock).not.toMatch(/src=""/);
+ // The author's specific composition script must NOT be merged INTO the
+ // runtime tag — it stays as its own when no runtime URL was configured. An
+ // empty src resolves to the page URL itself, which Chrome flags as an
+ // infinite-fetch hazard. Verify that bundleToSingleHtml inlines the
+ // runtime body so the bundle is genuinely self-contained.
+ const dir = makeTempProject({
+ "index.html": `
+
+
+`,
+ });
+
+ const previousUrl = process.env.HYPERFRAME_RUNTIME_URL;
+ delete process.env.HYPERFRAME_RUNTIME_URL;
+ let bundled: string;
+ try {
+ bundled = await bundleToSingleHtml(dir);
+ } finally {
+ if (previousUrl !== undefined) process.env.HYPERFRAME_RUNTIME_URL = previousUrl;
+ }
+
+ const runtimeBlock = bundled.match(
+ /]*data-hyperframes-preview-runtime[^>]*>[\s\S]*?<\/script>/i,
+ )?.[0];
+ expect(runtimeBlock).toBeDefined();
+ // Must NOT have an empty src attribute (would self-fetch).
+ expect(runtimeBlock).not.toMatch(/src=""/);
+ // Must have a non-trivial inlined body (the runtime IIFE is ~150KB).
+ const innerLength = (runtimeBlock!.match(/>([\s\S]*?)<\/script>/)?.[1] ?? "").length;
+ expect(innerLength).toBeGreaterThan(1000);
+ });
+
+ it("preserves chunk integrity when a chunk ends with a line comment (ASI hazard guard)", async () => {
+ // Regression guard for the joinJsChunks helper. If a chunk ends with `// ...`
+ // and we naively appended `;` on the same line, the appended semicolon would
+ // be eaten by the comment, leaving the next chunk's first statement attached
+ // to the previous chunk's last expression. Verify the helper appends `\n;`
+ // instead so the comment terminates and the semicolon stands alone.
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+
+
+`,
+ // Chunk A ends with a // line comment — without the \n separator before
+ // the appended ;, that ; would be eaten by the comment.
+ "local-a.js": "window.__a = 1 // trailing line comment",
+ "local-b.js": "window.__b = 2",
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+ // Run every inline script body through esbuild; if the line comment ate
+ // the separator, parse would fail with an unexpected-token error somewhere
+ // around the chunk boundary. Use a real HTML parser (CodeQL flags regex-
+ // based script extraction as bad-tag-filter).
+ const { transformSync } = await import("esbuild");
+ const { document } = parseHTML(bundled);
+ for (const script of document.querySelectorAll("script")) {
+ const body = script.textContent;
+ if (!body || !body.trim()) continue;
+ expect(() => transformSync(body, { loader: "js", minify: false })).not.toThrow();
+ }
+ });
+
+ it("does not produce stray bare-semicolon lines between concatenated JS chunks", async () => {
+ // Regression guard: hf#XXX. Earlier the bundler joined script chunks with
+ // `\n;\n`, which produces a lone `;` on its own line between chunks. Valid
+ // JS but reads as a code smell. Each chunk should end in `;` and chunks
+ // should join with `\n`.
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+
+
+`,
+ "local-a.js": "window.__a = 1",
+ "local-b.js": "window.__b = 2",
+ "compositions/child.html": `
+
+
+
+ `,
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+ // No line is JUST a bare semicolon (with optional surrounding whitespace).
+ expect(bundled).not.toMatch(/\n\s*;\s*\n/);
+ });
+
it("hoists external CDN scripts from sub-compositions into the bundle", async () => {
const dir = makeTempProject({
"index.html": `
@@ -58,8 +190,14 @@ describe("bundleToSingleHtml", () => {
// GSAP CDN from main doc should still be present
expect(bundled).toContain("cdn.jsdelivr.net/npm/gsap");
- // data-composition-src should be stripped (composition was inlined)
- expect(bundled).not.toContain("data-composition-src");
+ // data-composition-src should be stripped from the host element (composition
+ // was inlined). The literal string may still appear inside the inlined
+ // runtime IIFE that knows how to look up that attribute — so check the DOM,
+ // not the raw text.
+ const { document: doc } = parseHTML(bundled);
+ const hostEl = doc.getElementById("rockets-host");
+ expect(hostEl).toBeTruthy();
+ expect(hostEl?.hasAttribute("data-composition-src")).toBe(false);
});
it("does not duplicate CDN scripts already present in the main document", async () => {
@@ -136,7 +274,7 @@ describe("bundleToSingleHtml", () => {
expect(bundled).toContain(".logo");
// Scripts from template should be included
- expect(bundled).toContain('window.__timelines["logo-reveal"]');
+ expect(bundled).toContain('__timelines["logo-reveal"]');
});
it("does not inline template when host already has content", async () => {
@@ -196,6 +334,149 @@ describe("bundleToSingleHtml", () => {
expect(bundled).toContain("Sized content");
});
+ it("flattens the sub-composition root onto the host when inlining external compositions", async () => {
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+`,
+ "compositions/scene.html": `
+
+
+
Scene
+
+
+ `,
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+
+ const { document } = parseHTML(bundled);
+ const host = document.querySelector("#scene-host");
+
+ expect(host?.getAttribute("data-composition-id")).toBe("scene");
+ expect(host?.getAttribute("data-start")).toBe("intro");
+ expect(host?.getAttribute("data-width")).toBe("1920");
+ expect(host?.querySelector(".title")?.textContent).toBe("Scene");
+ expect(
+ Array.from(host?.children ?? []).some(
+ (child) => child.getAttribute("data-composition-id") === "scene",
+ ),
+ ).toBe(false);
+ expect(bundled).toContain('[data-composition-id="scene"] .title');
+ expect(bundled).toContain("__hfNormalizeSelector");
+ });
+
+ it("scopes external sub-composition styles and classic scripts", async () => {
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+
+
+`,
+ "compositions/scene.html": `
+
+
+
Scene
+
+
+ `,
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+
+ expect(bundled).toContain('[data-composition-id="scene"] .title');
+ expect(bundled).toContain('[data-composition-id="scene"] .title { color: red; }');
+ expect(bundled).toContain("new Proxy(window.document");
+ expect(bundled).toContain("new Proxy(__hfBaseGsap");
+ expect(bundled).toContain('tl.to(".title"');
+ });
+
+ it("isolates sibling instances of the same external sub-composition", async () => {
+ const dir = makeTempProject({
+ "index.html": `
+
+
+
+
+
+`,
+ "compositions/scene.html": `
+
+
+
Scene
+
+
+ `,
+ });
+
+ const bundled = await bundleToSingleHtml(dir);
+
+ const { document } = parseHTML(bundled);
+ const sceneA = document.querySelector("#scene-a");
+ const sceneB = document.querySelector("#scene-b");
+ const sceneAId = sceneA?.getAttribute("data-composition-id") ?? "";
+ const sceneBId = sceneB?.getAttribute("data-composition-id") ?? "";
+
+ expect(sceneAId).not.toBe("scene");
+ expect(sceneBId).not.toBe("scene");
+ expect(sceneAId).not.toBe(sceneBId);
+ expect(sceneA?.getAttribute("data-hf-original-composition-id")).toBe("scene");
+ expect(sceneB?.getAttribute("data-hf-original-composition-id")).toBe("scene");
+ expect(bundled).toContain(`[data-composition-id="${sceneAId}"] .title`);
+ expect(bundled).toContain(`[data-composition-id="${sceneBId}"] .title`);
+ expect(bundled).toContain('var __hfTimelineCompId = "scene__hf1"');
+ expect(bundled).toContain('var __hfTimelineCompId = "scene__hf2"');
+ expect(bundled).not.toContain('[data-composition-id="scene"] .title { opacity: 0; }');
+ });
+
it("rewrites CSS url(...) asset paths from sub-compositions when styles are hoisted", async () => {
const dir = makeTempProject({
"index.html": `
diff --git a/packages/core/src/compiler/htmlBundler.ts b/packages/core/src/compiler/htmlBundler.ts
index c59058478..def7c8463 100644
--- a/packages/core/src/compiler/htmlBundler.ts
+++ b/packages/core/src/compiler/htmlBundler.ts
@@ -1,23 +1,16 @@
import { readFileSync, existsSync } from "fs";
import { join, resolve, isAbsolute, sep } from "path";
-import { parseHTML } from "linkedom";
import { transformSync } from "esbuild";
import { compileHtml, type MediaDurationProber } from "./htmlCompiler";
+import {
+ RUNTIME_BOOTSTRAP_ATTR,
+ parseHTMLContent,
+ stripEmbeddedRuntimeScripts,
+} from "./htmlDocument";
import { rewriteAssetPaths, rewriteCssAssetUrls } from "./rewriteSubCompPaths";
+import { scopeCssToComposition, wrapScopedCompositionScript } from "./compositionScoping";
import { validateHyperframeHtmlContract } from "./staticGuard";
-
-/**
- * Parse an HTML string into a document. Fragments (without a full document
- * structure) are wrapped in `…`
- * so that linkedom places the content inside `document.body`.
- */
-function parseHTMLContent(html: string): Document {
- const trimmed = html.trimStart().toLowerCase();
- if (trimmed.startsWith("${html}`).document;
-}
+import { getHyperframeRuntimeScript } from "../generated/runtime-inline";
/** Resolve a relative path within projectDir, rejecting traversal outside it. */
function safePath(projectDir: string, relativePath: string): string | null {
@@ -27,54 +20,43 @@ function safePath(projectDir: string, relativePath: string): string | null {
return resolved;
}
-const RUNTIME_BOOTSTRAP_ATTR = "data-hyperframes-preview-runtime";
const DEFAULT_RUNTIME_SCRIPT_URL = "";
-function stripEmbeddedRuntimeScripts(html: string): string {
- if (!html) return html;
- const scriptRe = /]*>[\s\S]*?<\/script>/gi;
- const runtimeSrcMarkers = [
- "hyperframe.runtime.iife.js",
- "hyperframe-runtime.modular-runtime.inline.js",
- RUNTIME_BOOTSTRAP_ATTR,
- ];
- const runtimeInlineMarkers = [
- "__hyperframeRuntimeBootstrapped",
- "__hyperframeRuntime",
- "__hyperframeRuntimeTeardown",
- "window.__player =",
- "window.__playerReady",
- "window.__renderReady",
- ];
-
- const shouldStrip = (block: string): boolean => {
- const lowered = block.toLowerCase();
- for (const marker of runtimeSrcMarkers) {
- if (lowered.includes(marker.toLowerCase())) return true;
- }
- for (const marker of runtimeInlineMarkers) {
- if (block.includes(marker)) return true;
- }
- return false;
- };
-
- return html.replace(scriptRe, (block) => (shouldStrip(block) ? "" : block));
-}
-
function getRuntimeScriptUrl(): string {
const configured = (process.env.HYPERFRAME_RUNTIME_URL || "").trim();
return configured || DEFAULT_RUNTIME_SCRIPT_URL;
}
-function injectInterceptor(html: string): string {
+function injectInterceptor(html: string, runtimeMode: "inline" | "placeholder" = "inline"): string {
const sanitized = stripEmbeddedRuntimeScripts(html);
if (sanitized.includes(RUNTIME_BOOTSTRAP_ATTR)) return sanitized;
- const runtimeScriptUrl = getRuntimeScriptUrl().replace(/"/g, """);
- const tag = ``;
+ // Three modes for the runtime `;
+ } else if (runtimeMode === "placeholder") {
+ tag = ``;
+ } else {
+ const inlinedRuntime = getHyperframeRuntimeScript();
+ tag = ``;
+ }
if (sanitized.includes("")) {
return sanitized.replace("", `${tag}\n`);
}
+ const htmlOpenMatch = sanitized.match(/]*>/i);
+ if (htmlOpenMatch?.index != null) {
+ const insertPos = htmlOpenMatch.index + htmlOpenMatch[0].length;
+ return `${sanitized.slice(0, insertPos)}${tag}${sanitized.slice(insertPos)}`;
+ }
const doctypeIdx = sanitized.toLowerCase().indexOf("= 0) {
const insertPos = sanitized.indexOf(">", doctypeIdx) + 1;
@@ -192,6 +174,14 @@ function rewriteCssUrlsWithInlinedAssets(cssText: string, projectDir: string): s
);
}
+function cssAttributeSelector(attr: string, value: string): string {
+ return `[${attr}="${value.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"]`;
+}
+
+function uniqueCompositionId(baseId: string, index: number): string {
+ return `${baseId}__hf${index}`;
+}
+
function enforceCompositionPixelSizing(document: Document): void {
const compositionEls = [
...document.querySelectorAll("[data-composition-id][data-width][data-height]"),
@@ -290,17 +280,12 @@ function coalesceHeadStylesAndBodyScripts(document: Document): void {
}
const bodyInlineScripts = [...document.querySelectorAll("body script")].filter((el) => {
- const src = (el.getAttribute("src") || "").trim();
- if (src) return false;
+ if (el.hasAttribute(RUNTIME_BOOTSTRAP_ATTR) || el.hasAttribute("src")) return false;
const type = (el.getAttribute("type") || "").trim().toLowerCase();
return !type || type === "text/javascript" || type === "application/javascript";
});
if (bodyInlineScripts.length > 0) {
- const mergedJs = bodyInlineScripts
- .map((el) => (el.textContent || "").trim())
- .filter(Boolean)
- .join("\n;\n")
- .trim();
+ const mergedJs = joinJsChunks(bodyInlineScripts.map((el) => el.textContent || ""));
for (const el of bodyInlineScripts) el.remove();
if (mergedJs) {
const stripped = stripJsCommentsParserSafe(mergedJs);
@@ -311,6 +296,31 @@ function coalesceHeadStylesAndBodyScripts(document: Document): void {
}
}
+/**
+ * Concatenate JS chunks safely. Goals:
+ * - Each chunk's last statement is terminated, so joining can't introduce ASI
+ * surprises (e.g. `a()` followed by `(b)()` — the second chunk would parse
+ * as a call on the first's return value).
+ * - In the common case (chunk already ends with `;` — typical of esbuild
+ * output and IIFE-wrapped composition scripts ending in `})();`), the join
+ * produces clean output: chunks separated by `\n` with no stray bare
+ * semicolon lines.
+ * - Defensive against trailing line comments. If a chunk ends with `// ...`
+ * and we appended `;` on the same line, the appended semicolon would be
+ * swallowed by the comment, leaving the next chunk's first statement
+ * attached to the previous chunk's last expression — exactly the ASI
+ * hazard this helper exists to prevent. So when a chunk doesn't already
+ * end in `;`, we append `\n;` instead — the newline closes any line
+ * comment, and the standalone `;` becomes the statement separator.
+ */
+function joinJsChunks(chunks: string[]): string {
+ return chunks
+ .map((chunk) => chunk.trim())
+ .filter((chunk) => chunk.length > 0)
+ .map((chunk) => (chunk.endsWith(";") ? chunk : chunk + "\n;"))
+ .join("\n");
+}
+
function stripJsCommentsParserSafe(source: string): string {
if (!source) return source;
try {
@@ -324,6 +334,22 @@ function stripJsCommentsParserSafe(source: string): string {
export interface BundleOptions {
/** Optional media duration prober (e.g., ffprobe). If omitted, media durations are not resolved. */
probeMediaDuration?: MediaDurationProber;
+ /**
+ * How to handle the HyperFrames runtime ` so the caller can
+ * substitute it with a real URL via string replace. Used by the dev studio
+ * server and vite preview to point at a local runtime endpoint, which keeps
+ * the runtime cacheable across hot-reloads instead of re-inlining ~150 KB
+ * on every change.
+ *
+ * The `HYPERFRAME_RUNTIME_URL` env var, when set, takes precedence over both
+ * modes and emits `
+
+
+
+
+`;
+
+ const stripped = stripEmbeddedRuntimeScripts(html);
+
+ expect(stripped).not.toContain("hyperframe.runtime.iife.js");
+ expect(stripped).not.toContain("hyperframes-runtime.modular.inline.js");
+ expect(stripped).not.toContain("hyperframe-runtime.modular-runtime.inline.js");
+ expect(stripped).not.toContain("data-hyperframes-preview-runtime");
+ expect(stripped).not.toContain("window.__playerReady");
+ expect(stripped).toContain("window.authored = true");
+ });
+
+ it("does not treat non-script tags as scripts when stripping runtimes", () => {
+ const html = "window.__playerReady = true; ";
+
+ expect(stripEmbeddedRuntimeScripts(html)).toBe(html);
+ });
+
+ it("injects head and body scripts without replacement-token interpolation", () => {
+ const html = "";
+ const injected = injectScriptsIntoHtml(html, ["window.x = '$&';"], ["window.y = '$&';"]);
+
+ expect(injected).toContain("\n");
+ expect(injected).toContain("\n 
+ 