Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ When adding a new CLI command:

## Skills

Composition authoring (not repo development) is guided by skills installed via `npx skills add heygen-com/hyperframes`. See `skills/` for source. Invoke `/hyperframes`, `/hyperframes-cli`, or `/gsap` when authoring compositions. When a user provides a website URL and wants a video, invoke `/website-to-hyperframes` — it runs the full 7-step capture-to-video pipeline.
Composition authoring (not repo development) is guided by skills installed via `npx skills add heygen-com/hyperframes`. See `skills/` for source. Invoke `/hyperframes`, `/hyperframes-cli`, `/hyperframes-registry`, or `/gsap` when authoring compositions. When a user provides a website URL and wants a video, invoke `/website-to-hyperframes` — it runs the full 7-step capture-to-video pipeline.
24 changes: 24 additions & 0 deletions CREDITS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Credits

## Prior art

HyperFrames was inspired by prior work in the browser-based video rendering space.
In particular, we want to acknowledge:

- **[Remotion](https://www.remotion.dev)** pioneered the approach of using a
headless browser + FFmpeg `image2pipe` pipeline to turn web primitives into
deterministic video in the JavaScript ecosystem. Several of HyperFrames'
architectural ideas — ordered async barriers for parallel frame capture,
multi-host port availability probing for dev servers, and the broader shape
of a "render HTML to video" CLI — were informed by studying how Remotion
approaches these problems.

All code in this repository is independently implemented and distributed
under the [Apache 2.0 License](LICENSE). HyperFrames is not affiliated with
Remotion.

## Thanks

Thanks also to the authors and maintainers of the open-source projects
HyperFrames builds on, including Puppeteer, FFmpeg, GSAP, Hono, and the
broader Node.js ecosystem.
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,13 @@ HyperFrames ships [skills](https://github.com/vercel-labs/skills) that teach AI
npx skills add heygen-com/hyperframes
```

| Skill | What it teaches |
| ---------------------- | -------------------------------------------------------------------------------------------- |
| `hyperframes` | HTML composition authoring, captions, TTS, audio-reactive animation, transitions |
| `hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts, doctor |
| `hyperframes-registry` | Block and component installation via `hyperframes add` |
| `gsap` | GSAP animation API, timelines, easing, ScrollTrigger, plugins, React/Vue/Svelte, performance |
| Skill | What it teaches |
| ------------------------ | -------------------------------------------------------------------------------------------- |
| `hyperframes` | HTML composition authoring, captions, TTS, audio-reactive animation, transitions |
| `hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts, doctor |
| `hyperframes-registry` | Block and component installation via `hyperframes add` |
| `website-to-hyperframes` | Capture a URL and turn it into a video — full website-to-video pipeline |
| `gsap` | GSAP animation API, timelines, easing, ScrollTrigger, plugins, React/Vue/Svelte, performance |

## Contributing

Expand Down
12 changes: 7 additions & 5 deletions docs/guides/prompting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@ npx skills add heygen-com/hyperframes

In Claude Code, restart the session after installing. Skills register as **slash commands**:

| Slash command | What it loads |
| ------------------- | ---------------------------------------------------------------------------- |
| `/hyperframes` | Composition authoring — HTML structure, timing, captions, TTS, transitions |
| `/hyperframes-cli` | CLI commands — `init`, `lint`, `preview`, `render`, `transcribe`, `tts` |
| `/gsap` | GSAP animation API — timelines, easing, ScrollTrigger, plugins |
| Slash command | What it loads |
| -------------------------- | -------------------------------------------------------------------------- |
| `/hyperframes` | Composition authoring — HTML structure, timing, captions, TTS, transitions |
| `/hyperframes-cli` | CLI commands — `init`, `lint`, `preview`, `render`, `transcribe`, `tts` |
| `/hyperframes-registry` | Block and component installation via `hyperframes add` |
| `/website-to-hyperframes` | Capture a URL and turn it into a video — full website-to-video pipeline |
| `/gsap` | GSAP animation API — timelines, easing, ScrollTrigger, plugins |

<Tip>
Always prefix Hyperframes prompts with `/hyperframes` (or invoke the skill another way for non-Claude agents). This loads the skill context explicitly so the agent gets composition rules right the first time, instead of relying on whatever it remembers about web video.
Expand Down
44 changes: 28 additions & 16 deletions packages/cli/src/server/portUtils.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
/**
* Port utilities for the HyperFrames preview server.
*
* Implements Remotion-style port handling:
* The multi-host availability probe and instance-reuse port selection are
* inspired by Remotion's approach to dev-server port management.
*
* - Multi-host availability testing (catches port-forwarding ghosts)
* - HTTP probe for detecting existing HyperFrames instances
* - PID detection for actionable conflict logging
Expand Down Expand Up @@ -30,23 +32,32 @@ const PROBE_MAX_BYTES = 4096;

/**
* Test whether a port is free on a specific host.
* Returns false (unavailable) only for EADDRINUSE. Other errors (e.g.,
* EADDRNOTAVAIL when IPv6 is disabled) are treated as "this host doesn't
* apply" and return true.
*
* Attempts an ephemeral bind-and-release with `net.createServer()`. Only
* `EADDRINUSE` means "genuinely occupied" — other errnos (EADDRNOTAVAIL when
* IPv6 is disabled, EACCES for privileged ports, EAFNOSUPPORT for missing
* address families) mean "this host doesn't apply to our probe", and we treat
* the port as free for this host rather than poisoning the whole scan.
*/
function isPortAvailableOnHost(port: number, host: string): Promise<boolean> {
return new Promise<boolean>((resolve) => {
const server = net.createServer();
server.unref();
server.on("error", (err: NodeJS.ErrnoException) => {
resolve(err.code !== "EADDRINUSE");
});
server.listen({ port, host }, () => {
server.close(() => {
resolve(true);
});
async function isPortAvailableOnHost(port: number, host: string): Promise<boolean> {
const probe = net.createServer();
probe.unref();

const bindError = await new Promise<NodeJS.ErrnoException | null>((settle) => {
const handleError = (err: NodeJS.ErrnoException): void => settle(err);
probe.once("error", handleError);
probe.listen({ port, host }, () => {
probe.removeListener("error", handleError);
settle(null);
});
});

if (bindError !== null) {
return bindError.code !== "EADDRINUSE";
}

await new Promise<void>((done) => probe.close(() => done()));
return true;
}

export const PORT_PROBE_HOSTS = ["127.0.0.1", "0.0.0.0", "::1", "::"] as const;
Expand Down Expand Up @@ -300,7 +311,8 @@ export type FindPortResult =
| { type: "already-running"; port: number };

/**
* Smart port selection with instance reuse (Remotion-style).
* Smart port selection with instance reuse (inspired by Remotion's dev-server
* port handling).
*
* For each port in the scan range:
* 1. Test availability on multiple hosts (catches port-forwarding ghosts)
Expand Down
12 changes: 7 additions & 5 deletions packages/cli/src/templates/_shared/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@

**Always invoke the relevant skill before writing or modifying compositions.** Skills encode framework-specific patterns (e.g., `window.__timelines` registration, `data-*` attribute semantics, shader-compatible CSS rules) that are NOT in generic web docs. Skipping them produces broken compositions.

| Skill | Command | When to use |
| ------------------- | ------------------ | ------------------------------------------------------------------------------------------------- |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights |
| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts |
| **gsap** | `/gsap` | GSAP animations for HyperFrames — tweens, timelines, easing, performance |
| Skill | Command | When to use |
| -------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------- |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights |
| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts |
| **hyperframes-registry** | `/hyperframes-registry` | Installing blocks and components via `hyperframes add` |
| **website-to-hyperframes** | `/website-to-hyperframes` | Capturing a URL and turning it into a video — full website-to-video pipeline |
| **gsap** | `/gsap` | GSAP animations for HyperFrames — tweens, timelines, easing, performance |

> **Skills not available?** Ask the user to run `npx hyperframes skills` and restart their
> agent session, or install manually: `npx skills add heygen-com/hyperframes`.
Expand Down
9 changes: 6 additions & 3 deletions packages/engine/src/services/browserManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,8 +244,10 @@ export function buildChromeArgs(
options: BuildChromeArgsOptions,
config?: Partial<Pick<EngineConfig, "disableGpu" | "chromePath">>,
): string[] {
// Chrome flags tuned for headless rendering performance.
// Based on Remotion's open-browser.ts flags with additions for our use case.
// Chrome flags tuned for headless rendering performance. The set below is a
// fairly standard "headless-for-capture" configuration — similar profiles
// appear in Puppeteer's defaults, Playwright, Remotion, and Chrome's own
// headless-shell guidance.
const chromeArgs = [
"--no-sandbox",
"--disable-setuid-sandbox",
Expand All @@ -257,7 +259,8 @@ export function buildChromeArgs(
"--font-render-hinting=none",
"--force-color-profile=srgb",
`--window-size=${options.width},${options.height}`,
// Remotion perf flags — prevent Chrome from throttling background tabs/timers
// Prevent Chrome from throttling background tabs/timers — critical when the
// page is offscreen during headless capture
"--disable-background-timer-throttling",
"--disable-backgrounding-occluded-windows",
"--disable-renderer-backgrounding",
Expand Down
70 changes: 69 additions & 1 deletion packages/engine/src/services/streamingEncoder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@

import { describe, expect, it } from "vitest";

import { buildStreamingArgs, type StreamingEncoderOptions } from "./streamingEncoder.js";
import {
buildStreamingArgs,
createFrameReorderBuffer,
type StreamingEncoderOptions,
} from "./streamingEncoder.js";
import { DEFAULT_HDR10_MASTERING } from "../utils/hdr.js";

const baseHdrPq: StreamingEncoderOptions = {
Expand Down Expand Up @@ -158,3 +162,67 @@ describe("buildStreamingArgs", () => {
});
});
});

describe("createFrameReorderBuffer", () => {
it("fast-paths waitForFrame(cursor) without queueing", async () => {
const buf = createFrameReorderBuffer(0, 3);
await buf.waitForFrame(0);
});

it("gates out-of-order writers into cursor order", async () => {
const buf = createFrameReorderBuffer(0, 4);
const writeOrder: number[] = [];

const writer = async (frame: number) => {
await buf.waitForFrame(frame);
writeOrder.push(frame);
buf.advanceTo(frame + 1);
};

const p3 = writer(3);
const p1 = writer(1);
const p2 = writer(2);
const p0 = writer(0);

await Promise.all([p0, p1, p2, p3]);
expect(writeOrder).toEqual([0, 1, 2, 3]);
});

it("supports multiple waiters registered for the same frame", async () => {
const buf = createFrameReorderBuffer(0, 2);
const resolved: string[] = [];

const a = buf.waitForFrame(1).then(() => resolved.push("a"));
const b = buf.waitForFrame(1).then(() => resolved.push("b"));

buf.advanceTo(0);
await Promise.resolve();
expect(resolved).toEqual([]);

buf.advanceTo(1);
await Promise.all([a, b]);
expect(resolved.sort()).toEqual(["a", "b"]);
});

it("waitForAllDone resolves when cursor reaches endFrame", async () => {
const buf = createFrameReorderBuffer(0, 3);
let done = false;
const allDone = buf.waitForAllDone().then(() => {
done = true;
});

buf.advanceTo(1);
await Promise.resolve();
expect(done).toBe(false);

buf.advanceTo(3);
await allDone;
expect(done).toBe(true);
});

it("waitForAllDone fast-paths when cursor already past endFrame", async () => {
const buf = createFrameReorderBuffer(0, 3);
buf.advanceTo(5);
await buf.waitForAllDone();
});
});
79 changes: 51 additions & 28 deletions packages/engine/src/services/streamingEncoder.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
/**
* Streaming Encoder Service
*
* Pipes frame screenshot buffers directly to FFmpeg's stdin instead of writing
* them to disk and reading them back in a separate encode stage. Follows the
* Remotion pattern of image2pipe → FFmpeg.
* Pipes frame screenshot buffers directly to FFmpeg's stdin via `-f image2pipe`
* instead of writing them to disk and reading them back in a separate encode
* stage. Inspired by Remotion's approach to browser-based video rendering.
*
* Two building blocks:
* 1. Frame reorder buffer – ensures out-of-order parallel workers feed
Expand All @@ -25,8 +25,16 @@ import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
export type { EncoderOptions } from "./chunkEncoder.types.js";

// ---------------------------------------------------------------------------
// 1. Frame reorder buffer (based on Remotion's ensure-frames-in-order.ts)
// 1. Frame reorder buffer — ordered async barrier
// ---------------------------------------------------------------------------
//
// Parallel workers produce frames out of order; FFmpeg's stdin expects them in
// strict sequential order. Each worker calls `waitForFrame(n)` to block until
// its turn, writes, then calls `advanceTo(n + 1)` to release the next waiter.
//
// `pending` holds an array per frame index (not a single resolver) so that
// `waitForAllDone` can coexist with the writer still waiting on the final
// frame without one clobbering the other.

export interface FrameReorderBuffer {
waitForFrame: (frame: number) => Promise<void>;
Expand All @@ -35,34 +43,49 @@ export interface FrameReorderBuffer {
}

export function createFrameReorderBuffer(startFrame: number, endFrame: number): FrameReorderBuffer {
let nextFrame = startFrame;
let waiters: Array<{ frame: number; resolve: () => void }> = [];

const resolveWaiters = () => {
for (const waiter of waiters.slice()) {
if (waiter.frame === nextFrame) {
waiter.resolve();
waiters = waiters.filter((w) => w !== waiter);
}
let cursor = startFrame;
const pending = new Map<number, Array<() => void>>();

const enqueueAt = (frame: number, resolve: () => void): void => {
const list = pending.get(frame);
if (list === undefined) {
pending.set(frame, [resolve]);
} else {
list.push(resolve);
}
};

return {
waitForFrame: (frame: number) =>
new Promise<void>((resolve) => {
waiters.push({ frame, resolve });
resolveWaiters();
}),
advanceTo: (frame: number) => {
nextFrame = frame;
resolveWaiters();
},
waitForAllDone: () =>
new Promise<void>((resolve) => {
waiters.push({ frame: endFrame, resolve });
resolveWaiters();
}),
const flushAt = (frame: number): void => {
const list = pending.get(frame);
if (list === undefined) return;
pending.delete(frame);
for (const resolve of list) resolve();
};

const waitForFrame = (frame: number): Promise<void> =>
new Promise<void>((resolve) => {
if (frame === cursor) {
resolve();
return;
}
enqueueAt(frame, resolve);
});

const advanceTo = (frame: number): void => {
cursor = frame;
flushAt(frame);
};

const waitForAllDone = (): Promise<void> =>
new Promise<void>((resolve) => {
if (cursor >= endFrame) {
resolve();
return;
}
enqueueAt(endFrame, resolve);
});

return { waitForFrame, advanceTo, waitForAllDone };
}

// ---------------------------------------------------------------------------
Expand Down
Loading