Skip to content

fix(lint): flag media clips whose window has no out-point #2972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xiayewang-heygen wants to merge 1 commit into from
Closed

fix(lint): flag media clips whose window has no out-point #2972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions packages/lint/src/rules/media.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,65 @@ describe("media rules", () => {
expect(finding?.elementId).toBe("demo-video");
});

it("reports error for a clip cut with data-media-start but no out-point", async () => {
const html = `
<html><body>
<div id="root" data-composition-id="c1" data-width="1920" data-height="1080" data-start="0" data-duration="7">
<video id="five9-video" src="training.mp4" data-start="0" data-media-start="2280" muted playsinline></video>
</div>
<script>window.__timelines = window.__timelines || {}; window.__timelines["c1"] = gsap.timeline({ paused: true });</script>
</body></html>`;
const result = await lintHyperframeHtml(html);
const finding = result.findings.find((f) => f.code === "media_unbounded_media_window");
expect(finding).toBeDefined();
expect(finding?.severity).toBe("error");
expect(finding?.elementId).toBe("five9-video");
expect(finding?.message).toContain("2280");
});

it("only warns for a full-source clip with no in-point and no out-point", async () => {
const html = `
<html><body>
<div id="root" data-composition-id="c1" data-width="1920" data-height="1080" data-start="0" data-duration="7">
<video id="aroll" src="aroll.mp4" data-start="0" muted playsinline></video>
</div>
<script>window.__timelines = window.__timelines || {}; window.__timelines["c1"] = gsap.timeline({ paused: true });</script>
</body></html>`;
const result = await lintHyperframeHtml(html);
const matches = result.findings.filter((f) => f.code === "media_unbounded_media_window");
expect(matches).toHaveLength(1);
expect(matches[0]?.severity).toBe("warning");
expect(matches[0]?.elementId).toBe("aroll");
});

it("accepts a cut clip that declares an out-point", async () => {
const html = `
<html><body>
<div id="root" data-composition-id="c1" data-width="1920" data-height="1080" data-start="0" data-duration="7">
<video id="v-dur" src="training.mp4" data-start="0" data-media-start="2280" data-duration="7" muted playsinline></video>
<video id="v-end" src="training.mp4" data-start="0" data-media-start="1089" data-end="7" muted playsinline></video>
</div>
<script>window.__timelines = window.__timelines || {}; window.__timelines["c1"] = gsap.timeline({ paused: true });</script>
</body></html>`;
const result = await lintHyperframeHtml(html);
expect(result.findings.find((f) => f.code === "media_unbounded_media_window")).toBeUndefined();
});

it("leaves untimed and sourceless media to their own rules", async () => {
const html = `
<html><body>
<div id="root" data-composition-id="c1" data-width="1920" data-height="1080" data-start="0" data-duration="7">
<video id="untimed" src="clip.mp4" muted playsinline></video>
<audio id="sourceless" data-start="0"></audio>
</div>
<script>window.__timelines = window.__timelines || {}; window.__timelines["c1"] = gsap.timeline({ paused: true });</script>
</body></html>`;
const result = await lintHyperframeHtml(html);
expect(result.findings.find((f) => f.code === "media_unbounded_media_window")).toBeUndefined();
expect(result.findings.find((f) => f.code === "media_missing_data_start")).toBeDefined();
expect(result.findings.find((f) => f.code === "media_missing_src")).toBeDefined();
});

it("allows audible video clips to omit muted when data-has-audio is true", async () => {
const html = `
<html><body>
Expand Down
38 changes: 38 additions & 0 deletions packages/lint/src/rules/media.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,44 @@ export const mediaRules: Array<(ctx: LintContext) => HyperframeLintFinding[]> =
return findings;
},

// media_unbounded_media_window — a clip with no out-point extracts to the end
// of its source. Distributed plans pre-extract one image per frame of each
// clip's media window, so that window — not the composition duration — is the
// planDir budget.
({ tags }) => {
const findings: HyperframeLintFinding[] = [];
for (const tag of tags) {
if (tag.name !== "video" && tag.name !== "audio") continue;
// Untimed / sourceless media belongs to media_missing_data_start and
// media_missing_src / media_variable_src_no_fallback.
if (!readAttr(tag.raw, "data-start")) continue;
if (!readAttr(tag.raw, "src") && !readAttr(tag.raw, "data-var-src")) continue;
if (readAttr(tag.raw, "data-end") || readAttr(tag.raw, "data-duration")) continue;

const elementId = readAttr(tag.raw, "id") || undefined;
const label = `<${tag.name}${elementId ? ` id="${elementId}"` : ""}>`;
const mediaStart = Number(readAttr(tag.raw, "data-media-start"));
const hasInPoint = Number.isFinite(mediaStart) && mediaStart > 0;

findings.push({
code: "media_unbounded_media_window",
// An in-point proves a slice was intended, so a missing out-point is a
// contradiction, not a style choice. Without one, "play the whole clip"
// is legitimate (full-length A-roll) — warn about the plan cost only.
severity: hasInPoint ? "error" : "warning",
message: hasInPoint
? `${label} sets data-media-start="${mediaStart}" but has neither data-end nor data-duration, so its media window runs from that in-point to the END of the source. Distributed renders pre-extract every frame of that window, so a few seconds cut from a long source can produce a multi-GiB plan and fail the render.`
: `${label} has neither data-end nor data-duration, so its media window is the entire source. Distributed renders pre-extract every frame of that window, so a source much longer than the clip's on-screen time inflates the plan and can exceed the plan-size limit.`,
elementId,
fixHint: hasInPoint
? `Add data-duration="<seconds the clip should play>" (or data-end) alongside data-media-start="${mediaStart}". An in-point without an out-point is almost always unintended.`
: `Add data-duration="<seconds the clip should play>" if the clip should stop before its source ends. If it is intentionally the full source, this is safe — but an explicit data-duration keeps plan size predictable.`,
snippet: truncateSnippet(tag.raw),
});
}
return findings;
},

// media_crossorigin_breaks_preview — `crossorigin` on <video>/<audio> forces a
// CORS-checked fetch. The server-side renderer downloads media directly (no CORS),
// so it always works there; but Studio preview runs in the browser, where a media
Expand Down
2 changes: 1 addition & 1 deletion skills-manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"files": 11
},
"hyperframes-core": {
"hash": "e9daaccaed05b8b4",
"hash": "5347ea558a44e683",
"files": 19
},
"hyperframes-creative": {
Expand Down
20 changes: 11 additions & 9 deletions skills/hyperframes-core/references/data-attributes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,17 @@ Timed child elements are clips. **`class="clip"` is required on visible timed el

**Visual clips (`class="clip"`) must be DIRECT children of the composition root.** A clip nested inside a wrapper `<div>` is not registered as a clip, so its `data-start`/`data-duration` are ignored and it stays visible the whole composition. To wrap/transform a clip, put the wrapper _inside_ the clip, or animate the clip element itself; do not wrap the clip. (This is a clip-_visibility_ rule. `<video>`/`<audio>` are exempt: the framework drives their playback via a flat DOM query, so they seek/decode at any depth, including inside a sub-comp `<template>` — see `variables-and-media.md`.)

| Attribute | Required | Meaning |
| ------------------ | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Yes | Stable DOM ID for linting, timeline targets, and debugging. |
| `data-start` | Yes | Start time in seconds, or a supported clip-time reference. |
| `data-duration` | Required for `div`, `img`, and sub-compositions | Duration in seconds. Video/audio can default to media duration when known. |
| `data-track-index` | Yes | Timeline track. Clips on the same track must not overlap. |
| `data-media-start` | No | Offset into the media source, in seconds. |
| `data-volume` | No | Static audio volume, `0` to `1`, default `1`. For fades, animate `volume` on the timeline instead (see `variables-and-media.md`). |
| `data-has-audio` | No (`<video>` only) | `"true"` to declare the video carries an audio track when auto-detection would miss it. |
| Attribute | Required | Meaning |
| ------------------ | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Yes | Stable DOM ID for linting, timeline targets, and debugging. |
| `data-start` | Yes | Start time in seconds, or a supported clip-time reference. |
| `data-duration` | Required for `div`, `img`, and sub-compositions; required for `video`/`audio` that set `data-media-start` | Duration in seconds. Video/audio may omit it to play their whole source — but see the plan-size note below before you do. |
| `data-track-index` | Yes | Timeline track. Clips on the same track must not overlap. |
| `data-media-start` | No (but pair it with `data-duration`) | Offset into the media source, in seconds. An in-point with no out-point extracts to the end of the source — see below. |
| `data-volume` | No | Static audio volume, `0` to `1`, default `1`. For fades, animate `volume` on the timeline instead (see `variables-and-media.md`). |
| `data-has-audio` | No (`<video>` only) | `"true"` to declare the video carries an audio track when auto-detection would miss it. |

**A media clip's window is its extraction budget, not just its playback range.** Distributed renders pre-extract one image per frame of each `<video>`'s media window at source resolution. That window is `[data-media-start, data-media-start + data-duration)` — and with no `data-duration`/`data-end` it runs to the **end of the source file**, however long that is. Cutting 7 seconds out of a 55-minute upload with `data-media-start="2280"` and no out-point therefore extracts ~29,600 frames (5+ GiB) instead of ~210, which fails the render with `PLAN_TOO_LARGE`. Always pair an in-point with an out-point; `hyperframes lint` errors on `media_unbounded_media_window` when you don't.

**Visibility window is inclusive of both ends.** A clip shows while `start ≤ t ≤ start + duration` — it still renders at exactly `t = start + duration`, so the final frame holds the animation's resolved end state (the runtime does not hide it one frame early). A reveal/entrance that lands on `data-duration` is therefore visible on the last frame; you do not need to finish it _before_ `data-duration` just to guarantee the end state renders. (Climax-dwell guidance in `/hyperframes-animation` is about pacing, not this boundary.)

Expand Down
2 changes: 2 additions & 0 deletions skills/hyperframes-core/references/tracks-and-clips.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ When adding a new clip to an existing composition:

`data-media-start` (on `<video>`/`<audio>`) is an offset _into the source media_. Use it to skip the first few seconds of a media file without trimming the file itself.

**Always give a clip with `data-media-start` a matching `data-duration`.** The in-point only says where the media window opens; without an out-point the window runs to the end of the source, and distributed renders pre-extract every frame of it. Cutting seconds out of a long recording without `data-duration` is what produces multi-GiB plans and `PLAN_TOO_LARGE` failures — `hyperframes lint` errors on `media_unbounded_media_window` to catch it.

## Relative Timing

`data-start` accepts a clip ID instead of a number, meaning "start when that clip ends". Add `+ N` / `- N` to offset; negative produces overlap (useful for crossfades).
Expand Down
Loading