Skip to content

Releases: heristop/leclap

ffmpeg-video-composer 3.2.0

Choose a tag to compare

@heristop heristop released this 23 Oct 15:36
54d9351

Added

  • Layer blend modes: inputs[].options.blend (normal, multiply, screen, overlay, add, darken,
    lighten, difference) mixes an image, animation or html layer into the picture inside its own alpha,
    with CSS mix-blend-mode formulas in RGB: transparent areas leave the picture untouched and half-transparent
    edges mix by their alpha. Lowered to blend over a transparent copy of the frame.
  • Motion blur: motionBlur (true or { shutter 0-360, samples 2-16 }) on layers with position, scale or
    rotate tracks and on travelling kinetic blocks. Only the moving frames are supersampled on a sub-frame
    shutter clock and averaged with tmix; resting frames and the background stay sharp. Bounded per element
    (motion_blur_capped).
  • The on-device engine adds blend and tmix (LGPL); a build without them composites normally or draws
    sharp, with a blend_unavailable / motion_blur_unavailable compile warning (validation reports them only
    when getMotionWarnings is given the target build's capabilities). Validation also advises
    motion_blur_static and motion_blur_ignored.
  • Sample blend-motion-blur and motion-review fixtures layer-blend and motion-blur.
  • Six more packaged samples: html-motion (animated html cards), vector-shapes, line-draw, fx-burst,
    fx-liquid and fx-glitch-iris-sheen. layer-keyframes and kinetic-type blur their fast moves with
    motionBlur, and effects-tour adds shaped-iris and bands beats plus blend-mode, mask, shape and motion-blur
    beats. 59 samples in all.
  • Masks and track mattes: an image, animation or html layer takes options.mask, and a video or project_video
    section takes options.mask. A mask is a shape (rect with radius, ellipse, or a path d of
    M/L/H/V/C/Q/Z, 4000 characters at most) drawn once at compile time into a grey image, or a layer, another
    layer of the section read as a track matte (mode alpha or luma; the matte layer is not drawn). invert
    shows outside the mask, feather softens its edge (px), and animate moves a shape with the layer tracks
    (x/y offsets, scale, scaleX, scaleY, rotate about origin). A section mask shows background (a
    colour token) outside; layers and text draw on top. The element is drawn on a copy of the frame as before,
    the copy takes the mask as its alpha and goes back over the frame, so a layer keeps its own transparency
    (split, format, lutyuv, overlay, alphamerge, colorchannelmixer, gblur, all on device). Without
    alphamerge a shape falls back to its resting box and a track matte draws both layers (mask_unavailable).
    New validation codes: mask_layer_unknown, mask_layer_self, mask_layer_shared, mask_layer_chained,
    mask_not_layer, mask_with_clip, mask_with_composite (a mask next to a non-normal blend or a
    motionBlur), mask_path_invalid, mask_too_large.
  • Clip boxes: a kinetic block takes clip ("block", its own resting box; "line", one box per line; or
    { x, y, w, h } in px) and an image, animation or html layer takes options.clip (a box). The element only
    shows inside the box, so a rise reveals out of an invisible slot above an underline, on any background,
    footage included, without a background-coloured panel. The frame is split, the element is drawn on one copy
    exactly as before, and the copy is cropped to the box and laid back over the frame (split, crop,
    overlay, all on device). Elements without a clip compile to the same filters as before.
  • bands, a designed transition: count parallel colour bands (2–8, default 4) at angle degrees (default 30) sweep in along their length, staggered (stagger, default 0.06 s), cover the frame at the midpoint,
    then carry on and uncover the next scene. colors default to the theme's brand, accent, accent2 and fg.
    Each band is a rectangle turned once by rotate and held by loop; per frame only overlay positions move.
  • Iris shapes: a designed iris takes shape (circle by default, unchanged, or star, heart,
    rounded-square). A shape opens from origin on the ease through a reveal field drawn once at compile
    time (per pixel, the scale the shape must reach to cover it), compared with the eased level on a half-size
    xfade mask like the circle's.
  • Zoom through a window: a designed zoom-through takes through, a letter's counter
    ({ text, glyph?, font?, size?, x?, y? }, laid out exactly as the kinetic block with the same values draws
    it: the O of ZOOM) or a shape ({ shape, size, x?, y? }). The window is a compile-time matte, read from the
    font's TrueType outlines; the outgoing scene and the matte run through the same zoom chain about the
    window's centre, so the letter and its hole stay locked together, and the next scene shows through the
    hole until it covers the frame. A window that cannot be drawn falls back to the plain zoom-through with a
    warning. Shaped irises and windows add their matte image as an input of the transition assembly.
  • Reference structure: analyzeStructure reads a reference clip's format (shots with their length, the
    transition into each (cut, dissolve → fade, dip → fadeblack), light or dark ground, where the main
    element sits on a 3×3 grid and motion) plus its rhythm, from the frames the style analyzer already
    samples. structurePromptRules turns it into binding generation rules (24 shots listed at most) and
    structureDrift holds a template to it. analyzeStructureFile decodes a clip on Node. Format only: never
    the reference's subjects, logos or text.
  • Brand kits: fetchBrandKit(url) reads a website's name, colours (theme-color, brand-named custom properties, the most used declared colours), fonts (heading, body, code) and logo over HTTPS from public addresses only, with size, time and redirect caps; saveBrandKit writes the kit and keeps it when the logo fails (logoError); brandTheme(kit) turns it into a global.theme (roles with WCAG contrast, the nearest bundled fonts plus the exact families as FontRef suggestions) and brandPromptRules into binding generation rules.
  • Animated HTML cards: elements tagged data-motion="name" move on the input's motion map
    ({ name: { x, y, scale, rotate, opacity, stagger } }, keyframes as in options.animate), so a card's tag,
    lines and chips can come in one after another. The card is drawn at the output fps over the animated span
    only, held frames drawn once, joined into one animated PNG and composited with the overlay path (no new
    filter; the apng demuxer is on every backend), holding its final frame. Bounded (8 s, 240 frames, 500
    megapixels: html_motion_too_long), html_motion_unmatched for a name no element carries, and a still in
    its final state (html_motion_static) when the frames can't be produced. Sample:
    examples/motion-design/html-motion.json.
  • A shape graphic: a filled vector shape for a sticker star, a heart, an arrow pointing at a detail, a blob
    behind a headline or an icon. The outline is a preset (star, polygon, heart, arrow, blob with a
    seed, rect with rounded corners) laid in a box (x, y, width, height, px or fractions of a
    target), or an SVG d of M/L/H/V/C/Q/Z, mapped into the box by a viewBox for icons. Star, polygon,
    heart and blob keep their proportions in portrait and square. It takes a color or a linear / radial
    gradient, fillRule (nonzero, evenodd), and a stroke that is either part of the shape or drawn on
    (stroke.draw, the path graphic's reveal, then the fill fades in). It moves with the layer keyframe tracks
    (animate: x, y, scale, scaleX, scaleY, rotate, opacity, springs included; origin is the
    pivot), or fades in over duration. The shape is rasterised once at compile time (shape: images, the same
    bytes on every platform) at its on-screen size, twice that for a growing scale track, never past 4 MP
    (the render log reports shape_downsampled; advisories are logged at compile time and validate does not rasterise). It renders on device with the filters keyframed image layers use. Advisories:
    shape_too_large, shape_unavailable, shape_draw_unavailable, shape_target, shape_empty,
    shape_color.
  • Validation: invalid_keyframes for a shape's tracks (keys out of order, opacity outside 0..1, scale outside
    0..10, relative values on scale or opacity).
  • Keyframe tracks on image, animation and html layer inputs: inputs[].options.animate takes x, y,
    scale, scaleX, scaleY, rotate and opacity tracks, with the same { t, v, ease? } keys, easings,
    springs, tokens and time references as the drawtext tracks. Cards, badges and icons can now pop in with an
    overshoot, squash and stretch, flip (scaleX 1 → 0 → 1 across two layers), swing like a needle or drift in
    parallax, instead of only entering with options.motion. options.origin (center by default, or an edge
    or corner) is the point that stays put while the layer scales or turns. x/y take pixels or offsets from
    options.position; rotate takes degrees, absolute or relative to options.rotation; opacity multiplies
    options.opacity. A track overrides the same property of options.motion. Scale and rotate tracks need the
    layer box: an html layer has it, an image or animation needs options.scale as "W:H" pixels. Every filter
    runs on the layer's own frame and is in the on-device build (fps, scale, rotate, colorchannelmixer,
    overlay); a property without a track adds nothing, and templates without animate compile to the same
    commands as before.
    • Validation: invalid_keyframes for layer tracks out of order, an opacity outside 0..1 or a scale outside
      0..10, relative values on scale or opacity, a scale or rotate track on a layer whose size is unknown, and
      tracks on an input that is not a layer; an unknown track name is a...
Read more

@leclap/mcp 0.7.0

Choose a tag to compare

@heristop heristop released this 23 Oct 15:36
54d9351

Added

  • get_template_schema and get_motion_catalog teach layer blend modes (options.blend) and motion blur
    (motionBlur on layers and kinetic blocks); validate_template advises motion_blur_static
    and motion_blur_ignored. A build without blend or tmix falls back at render time (a normal composite,
    or sharp) with a blend_unavailable / motion_blur_unavailable warning in the compile log; validate_template
    does not know the target build, so it does not report those two.
  • get_template_schema and get_motion_catalog teach masks and track mattes (options.mask,
    compositing.masks).
  • extract_structure: a reference clip's format (shots, rhythm, transitions, grounds, layout) as binding
    rules, from at most 480 sampled frames.
  • get_template_schema and get_motion_catalog teach animated HTML cards: data-motion names, the input's
    motion map with its stagger, the budget (html_motion_too_long, html_motion_unmatched) and a
    staggeredList recipe.
  • get_template_schema and get_motion_catalog teach the engine's shape graphic: presets or an SVG d with
    viewBox, colour or gradient fill, stroke, layer tracks or a drawn-on outline, and its shape_* advisories.
  • --allow-raw-filters / LECLAP_MCP_ALLOW_RAW_FILTERS (off by default) trusts a template's raw filters for
    trusted local use; get_capabilities reports which mode the server runs in, and the template guide says so.
  • extract_brand (behind --allow-network / LECLAP_MCP_ALLOW_NETWORK): a website's brand kit as a theme and binding rules, saved under the media dir.
  • get_template_schema's guide describes layer keyframe tracks (options.animate with x/y/scale/scaleX/
    scaleY/rotate/opacity and options.origin on image, animation and html inputs), the path graphic and the
    drip and bands transitions, clip boxes, shaped iris and zoom-through windows. get_motion_catalog lists the burst and goo fx and the art-direction rules for the
    new effects; validate_template and compose_video accept everything ffmpeg-video-composer 3.2.0 adds.

Changed

  • Raw filters in agent templates are allow-listed by the engine: a filter outside the list is a
    filter_not_allowed validation error instead of a render. A drawtext fontfile is a bare font name; fonts
    from the media dir go through global.fonts.
  • Requires ffmpeg-video-composer ^3.2.0.

Fixed

  • validate_template, compose_video and render_preview refuse a template whose string fields (a
    transition colour, for instance) carry FFmpeg filter syntax that opens files or URLs, not only the filter
    value and values. Ordinary copy with semicolons or commas is unaffected.
  • The template guard also sees through backslash-escaped filter names and refuses the command, socket and
    model-loading filters, and file options chained into a filter.
  • The template guard also refuses the LUT, SOFA, mask, OCR, speech and neural-network filters that load files,
    and more file-capable filter options.
  • The template guard also refuses a protocol hidden behind an escaped colon ('file\:/…', 'http\://…'), and
    the server warns on stderr at start-up when --allow-raw-filters is on.

@leclap/cli 0.6.0

Choose a tag to compare

@heristop heristop released this 23 Oct 15:36
54d9351

Added

  • leclap init writes agent guidance: an AGENTS.md pointer (imported by a one-line CLAUDE.md) and .agents/skills/authoring-leclap-videos/SKILL.md,
    with or without the .mcp.json option. It covers the validate/render loop (through the project's package
    manager, since leclap is a devDependency), the sugar tier before raw filters[], the re-encode cost model and
    the traps that never surface as errors, and points at llms.txt instead of restating the schema.
  • leclap reference <clip> prints a reference clip's format as binding rules (--json, --out);
    --check template.json exits 1 on drift. leclap compare --reference <clip> adds the reference's frame
    as the first tile.
  • leclap render --allow-raw-filters and leclap verify --rerender --allow-raw-filters trust a template's
    raw filters (trusted local templates only). Without it, raw filters are allow-listed by the engine and a
    template that uses a filter outside the list fails with filter_not_allowed.
  • leclap brand <url> writes a brand kit, a theme and binding rules.
  • leclap samples lists 59 samples: layer-keyframes animates cards and shapes with layer keyframe tracks
    (pop-in, squash and stretch, flip, needle, parallax), blend-motion-blur mixes layers with blend modes and
    blurs their motion, masks cuts layers and a video section with shape masks and track mattes, and
    html-motion, vector-shapes, line-draw, fx-burst, fx-liquid and fx-glitch-iris-sheen join the list.
  • leclap render and leclap validate take everything ffmpeg-video-composer 3.2.0 adds: layer keyframe
    tracks (options.animate, options.origin), the path and shape graphics, animated html cards
    (data-motion with the input's motion map), layer blend modes (options.blend) and motionBlur, masks and
    track mattes (options.mask), the burst and goo fx, the drip and bands transitions, clip boxes
    (clip, options.clip), shaped iris, zoom-through windows, glitch tear, iris origin/ease/rim and
    sheen profile: "gloss".

Changed

  • Requires ffmpeg-video-composer ^3.2.0.

Fixed

  • leclap verify --rerender validates the manifest's template before rendering it again, instead of
    skipping validation.

ffmpeg-video-composer 3.1.0

Choose a tag to compare

@heristop heristop released this 12 Oct 21:01
1926b0b

Added

  • Template fonts for HTML layers. global.fonts declares the faces a template brings,
    { family, src, weight?, style? }, one entry per weight or style; an HTML layer selects one with
    font-family in its css, and a declared family wins over a bundled family of the same name. src is a
    TrueType (.ttf), OpenType (.otf) or WOFF (.woff, unpacked to TrueType by the engine's own pure-JS
    inflate, no new dependency) file, or a base64
    data: URI, at most 8 MB. WOFF2 is refused with the command that converts it (Satori and HarfBuzz read
    neither WOFF2 nor its Brotli tables). Drawn text (drawtext, kinetic, captions) keeps the font registry.
  • ProjectConfig.fontDirs (Node): read-only directories a src path resolves in, before the assets dir. A
    relative src is looked up in each in turn, an absolute one must lie inside one of them, and symlinks are
    resolved before the check. Template fonts are never fetched over the network, and the temp and build dirs
    other staged media may come from are not searched.
  • A layer drawn with a template font is named by the hash of the font's bytes: the same font file gives the
    same frames, and a changed file under the same name is drawn again rather than reused.
  • Validation: font_woff2_unsupported and font_format for a src that is WOFF2 or no font, font_too_large
    for an oversized data URI, and the font_unused advisory for a declared family no HTML layer selects.
    templateFontErrors(descriptor, { assetsDir, fontDirs }) (Node entry) opens the files the way a render
    reads them and reports font_not_found, font_unreadable, font_format, font_woff2_unsupported and
    font_too_large. loadTemplateFonts and templateFontSpecs are exported from the Node and browser
    entries; renderHtmlLayerPreview takes the loaded faces as templateFonts.

Changed

  • CSS @font-face is still dropped, and its html_unsupported_css finding now points at global.fonts.
    html_font_unknown says the family is neither declared in global.fonts nor in the registry.
  • The packaged evidence samples (house-evidence, pr-evidence, before-after) take LeClap's evidence look:
    ink cards, BEFORE in rose and AFTER in lavender, Oswald headlines in the brand gradient and engine motion
    (kinetic scramble and cascade, drawn rules, a sheen). house-evidence now records one sidebyside
    clip, BEFORE and AFTER composed side by side, instead of six sections; pr-evidence and before-after
    keep their sections and gain a ticket variable.

Fixed

  • Speed ramps no longer judder at constant speed. The ramp's setpts now rounds to whole ticks instead of
    truncating, so a source frame that lands on a half tick (a 120 fps clip at 1x) stops falling either side of
    the frame-rate conform with float noise, which cadenced 5, 3 source frames instead of 4, 4.
  • Files read on the phone (React Native) keep their last bytes: the Expo filesystem adapter decoded base64 with
    its own helper, which turned the final one or two bytes of any file whose size is not a multiple of 3 into
    0xFF. An image inside an HTML layer (a JPEG losing its end marker) drew blank on the phone, and fonts or
    other staged files could be cut the same way. The adapter now uses the engine's shared base64 helpers.
  • A muted project_video section (muteSection: true) with layers (HTML, image or animation inputs, @video
    maps, chroma key) no longer fails with "Stream specifier ':v' … matches no streams". The muted section
    puts its silent audio input first, so the recorded clip is input 1, but the overlay graph still read the
    video from input 0 and numbered the layers from 1.
  • The phone's HTML layer page (dist/html-rasteriser.html) loads on older WebViews again: it called
    Promise.withResolvers (Chrome 119, Safari 17.4), so every HTML layer failed on an Android System WebView
    before 119 or iOS before 17.4. The page now supports Android System WebView 87+ and iOS 16.4+: its build
    targets them and it polyfills Array.prototype.at for Satori where missing. Rendered bytes are unchanged.
  • The before/after wipe (and split panes) no longer stutters. A still or clip pane arrived at the image
    demuxer's 25 fps and was never conformed to the section's frame rate, so the wipe's crop, timed on that
    leg, drifted from the overlay at the output rate: the revealed picture crept a few pixels a frame, then
    snapped back every 6 frames at 30 fps. Media and clip panes now run at the section frame rate.
  • Boxed caption plates draw their rounded corners row by row instead of in three coarse steps, so the corners
    no longer look pixelated when the frame is scaled up. Each plate line stays within about 25 drawbox
    filters: an anti-aliased version cost 77 to 193 per line and made long boxed transcripts render many times
    slower, so it was not kept.

@leclap/mcp 0.6.0

Choose a tag to compare

@heristop heristop released this 12 Oct 21:01
1926b0b

Added

  • global.fonts support. --fonts-dir / LECLAP_MCP_FONTS_DIR adds a read-only directory template fonts
    resolve in, before the media dir; nothing outside the two is read (realpath-checked) and fonts are never
    fetched. validate_template and compose_video refuse a declared font that is missing, outside those
    dirs, not a TTF/OTF/WOFF font, WOFF2 or over 8 MB, naming the entry and the fix.
  • get_template_schema describes global.fonts in its schema and guide.

Changed

  • Requires ffmpeg-video-composer ^3.1.0.

@leclap/cli 0.5.0

Choose a tag to compare

@heristop heristop released this 12 Oct 21:01
1926b0b

Added

  • global.fonts support: leclap render and leclap validate resolve a template font's src against
    the template's own directory first, then each --fonts <dir> (repeatable), then the assets dir, and read
    nothing outside them. leclap validate opens the declared files and fails on one that is missing,
    outside those dirs, not a TTF/OTF/WOFF font, WOFF2 or over 8 MB.

Changed

  • Requires ffmpeg-video-composer ^3.1.0.
  • leclap samples: the three evidence samples (house-evidence, pr-evidence, before-after) take LeClap's
    evidence look; house-evidence now takes one sidebyside clip, and the other two a ticket variable.

ffmpeg-video-composer 3.0.0

Choose a tag to compare

A motion, effects, footage, audio and agent-tooling release: HTML layers, composed sound effects, typed
template fields, auto-captions, per-format compositions and a determinism contract. Validation is stricter
and renders are no longer byte-identical to 2.5.0, hence the major version.

Upgrading from v2? See the migration guide.

Added

  • HTML layers, on Node, in the browser and on the phone. An inputs[] entry of type: "html" takes html,
    css, width and height and lays out a card, badge, price tag or stat row in a flexbox subset of CSS
    (tag, .class and descendant selectors). The markup is sanitised (no scripts, iframes, forms, SVG, event
    handlers or links), {{ name }} placeholders are filled HTML-escaped from typed fields, variables and form
    values, and $color.* / $font.* tokens resolve in css and html. Fonts come from the registry
    (variable fonts are pinned to static weights with HarfBuzz; layers of one section may share a font); images
    must be template assets or PNG/JPEG data URIs. Satori, resvg and HarfBuzz draw the layer at 2× into a transparent PNG named by content hash
    (html:<hash>, cached per process), which composites as a still image, so position, scale, start and
    motion work unchanged. One pipeline (core/html/satori-raster.ts) runs on every host, so a layer's PNG is
    byte-identical everywhere; only where its WebAssembly comes from differs:

    • Node reads it from node_modules.
    • The browser engine (compileBrowser) takes it from the host through BrowserCompileOptions.loadHtmlWasm
      (the web app serves it under /html-engine/<version>/); the engine never fetches it from a third party.
      It loads on the first HTML layer, in lazy chunks. renderHtmlLayerPreview(request, { loadHtmlWasm })
      draws one layer with the advisories its render reports, for a live preview.
    • Hermes has no WebAssembly, so the React Native entry takes a rasteriser from the host
      (registerHtmlRasteriser). The build ships dist/html-rasteriser.html, one self-contained page (5.2 MB,
      the pipeline and its WebAssembly inlined) that a hidden WebView runs, spoken to with
      createRasterSession / readRasterReply (JSON messages, PNG and fonts as base64, each font sent once per
      page).

    A host with no rasteriser registered (a browser compile without loadHtmlWasm, a React Native host that
    registered none) refuses the template with html_unavailable; validators that only check templates (the
    MCP server, the CLI) accept HTML layers. Validation also fails on html_too_large (a side over 1920 px).
    Advisories html_unsupported_css, html_unsupported_markup, html_font_unknown, html_missing_field and
    html_overflow (measured in the Node geometry checks); motionCatalog().html lists the subset and four
    recipes. Samples: html-card (a listing reel), html-testimonial (a Reels quote card), html-speaker (a
    conference lower third) and html-stats (a square stat dashboard), in examples/motion-design/.

  • Composed sound effects. An sfx cue takes a sound instead of an id: layers of tone, noise,
    strike and silence shaped by envelopes, filter chains, glides, drive, pan and sequences, with
    whole-sound saturate/crush/room/echo, bounded to 4 s and 8 layers. A pure TypeScript synth renders it,
    seeded by global.seed and the cue path, to build/sfx/<hash>.wav, mixed like a library file: the same
    sound and seed give the same file on a given platform, while Node, browsers and Hermes can differ in the
    last bit (each JavaScript engine computes Math.sin and Math.exp its own way). A note shorter than its
    attack plus release (a tick, a fast roll) stays audible, and jittered, accelerating rolls keep every hit.
    Every library sound is also a recipe (SOUND_PRESETS); sound.preset varies one by pitch, length,
    brightness and room, while id cues keep playing the shipped files. A sound's notes add up to at most
    32 s of audio, so its render cost stays bounded. Advisories sound_silent, sound_clipped,
    sound_harsh, sound_muddy, sound_long, sound_repeated and sound_overlap;
    motionCatalog().audio.compose; renderSound, analyzeChannels, soundSpec and SoundSchema exports.

  • Typed template fields: global.fields declares a template's inputs (map or list of { name, type, default?, required?, maxLength?, min?, max?, options?, label?, description? }, types text, color, url, media, number,
    enum and time). Values come from ProjectConfig.fields, then default; each is coerced by its type and
    filled into {{ name }} after partial expansion — a whole-string placeholder takes the typed value, so
    "duration": "{{ HOLD }}" lands as a number — then the slot's own schema judges it. A render fails before
    encoding on a missing required or ill-typed value; validateTemplate(t, { fields }) checks strictly,
    without fields it probes. New advisories (getFieldWarnings, also in getMotionWarnings):
    field_undefined, field_unused, field_type_mismatch, field_missing_required. Node entry exports
    resolveFields, assertFieldsResolved, coerceFieldValue (pluggable coercers and encode),
    declaredFields, fieldAdvisories and resolveTemplate. Templates without global.fields are unchanged.
    Colours follow FFmpeg's grammar (#rgb(a) and rgb()/rgba() normalised to #rrggbb(aa), names from
    FFmpeg's list, @alpha in 0–1); urls take http(s), data, media:// or relative paths; a field value with
    filtergraph separators is refused in a raw filter value. Validation without values returns the authored
    descriptor; the resolved descriptor drops global.fields, so resolution is idempotent. A form field bound
    to a non-text declared field may omit maxLength.

  • Auto-captions: subtitles.transcribe: { from?, language?, model? } asks for a section's speech as word
    timings. Transcribe once, then pin: a Node resolve pass (whisper.cpp — the whisper-cli binary with DTW word
    alignment, else an FFmpeg built with --enable-whisper) replaces the request with subtitles.words and records
    engine, model, language, clip digest and date in meta.resolved.transcripts; renders only read pinned words.
    Word times are mapped from clip to section seconds through clip, speed, speed ramps, freeze holds and kept
    take windows. The Node compile resolves an unpinned request itself; the browser and on-device engines report
    transcribe_unavailable. New advisories transcript_low_confidence and transcript_stale (the clip changed
    since the pin), new error invalid_transcribe_source, and optional confidence on words. Node exports
    transcribeMediaFile, transcribeTemplate, ensureWhisperModel (models downloaded once on explicit opt-in to
    ~/.cache/leclap/whisper, SHA-256-verified, never bundled) and the platform-neutral mapTranscriptWords,
    pinTranscript, transcriptSrt (also on the React Native entry). Pins also record a fingerprint of the
    section's edits, and transcript_edit_changed reports a pin whose clip, speed, trim, ramp or freeze changed
    since. CompileReporter.signal (Node) cancels a build, transcription included.

  • Template links (Node entry): createBuilderLink, encodeTemplatePayload / decodeTemplatePayload and
    mediaToRebind carry a template, compressed, in a builder URL's #t= fragment (never sent to a server),
    listing the media only the source machine can read (local paths, file:, media://, unsupported schemes).
    A base URL other than https://leclap.dev adds a warning, since that page can read the fragment.

  • applyJsonPatch(doc, operations, { maxOps }) and parsePointer(pointer) (Node entry): RFC 6902 JSON
    Patch (add, remove, replace, move, copy, test) over RFC 6901 pointers (- append, ~0/~1
    escapes). Atomic (runs on a copy, the input is never mutated); rejects __proto__/prototype/constructor
    segments and keys and anything deeper than 64 levels; failures throw a JsonPatchError with a code, the
    failing operation's index and its path.

  • templateRevision(template): stable SHA-256 of a template's canonical JSON (object key order ignored),
    synchronous and platform-neutral; matches a node:crypto digest of the same JSON byte for byte.

  • findingLine, invalidTemplateText and summarizeErrors (Node entry): the plain-text renderings of
    validation findings @leclap/mcp uses, now shared with other agent surfaces.

  • Effects tour (examples/motion-design/effects-tour.json): a six-minute tour of every motion effect in ten chapters (chapter 5 now shows every fx primitive), first in the Effects & editing showcase (51 samples).

  • samples: new effects category with 16 native samples (effects tour, FX pack, word captions, formats, kinetic fills, split layouts, right-to-left type, emoji type, beat grid, theme/roles/safe zones, footage editing, sound design, and four HTML layer samples: listing reel, testimonial, speaker card, stats dashboard).

  • Determinism contract. global.seed roots every procedural effect. A deterministic encoder profile
    (bit-exact muxing, pinned libx264 threads) is applied to every command through one adapter tap; it is on
    by default and ProjectConfig.deterministic: false opts out. A render manifest is delivered through
    CompileReporter.onManifest. Raw filters that read the wall clock or random() raise the advisory
    nondeterministic_expression (their renders are not reproducible).

  • Motion system: physical springs, cubic-bezier, the named curve set,
    steps() and point curves, all lowered to piecewise polynomials in t within 0.1%. global.motion
    tokens (springs, curves, durations, energy) and built-ins mirror the app's motion curves. animate
    keyframe tracks (x, y, opacity, scale) on positioned drawtext. The energy dial scales every
    travel distance. Each section chain is confor...

Read more

@leclap/mcp 0.5.0

Choose a tag to compare

@heristop heristop released this 13 Oct 00:14

Removed

  • Breaking: four 0.4.0 tools are gone, merged into others or dropped, because every tool's name and
    description is sent to the agent on every turn and overlapping tools dilute its choice. No aliases;
    call the replacement:

    Removed Call instead
    ping the MCP protocol ping request
    patch_template { edits } edit_template { template, expectedRevision, effectProps: edits }
    list_samples { …filters } get_samples { …filters }
    get_sample { id } get_samples { id }

    Fourteen tools are always registered (seventeen with the Remotion opt-in): analyze_music,
    analyze_sound, compose_video, edit_template, extract_style, get_capabilities,
    get_motion_catalog, get_samples, get_template_schema, open_in_builder, probe_media,
    render_frames, transcribe_media and validate_template, plus get_effect_schema,
    render_preview and render_remotion_clip with Remotion.

Changed

  • get_samples replaces list_samples and get_sample: without id it lists (it accepts category: "effects"; 51 packaged samples), with id it returns the sample and its template, plus a
    partialCatalog summary.
  • validate_template takes include: ["resolved" | "timeline"], plus fields and format for them:
    resolved is { descriptor, values }, the descriptor compose_video would render for the given
    fields (a refused field value is still an error), and timeline is the resolved section timeline.
    Without include, validation is unchanged.
  • Template revisions (expectedRevision / revision) and the validation finding text now come from the
    engine (templateRevision, invalidTemplateText), so other surfaces compute the same revision for the
    same JSON; values are unchanged.
  • Built on ffmpeg-video-composer 3: validate_template, edit_template and compose_video reject
    templates with unknown keys (unknown_key) and text a bundled font cannot draw (font_missing_glyphs),
    which 0.4.0 accepted.

Added

  • HTML layers (inputs[].type: "html"): compose_video and render_frames draw a card, badge or price
    tag laid out in HTML and CSS and composite it like an image; its images must be template assets under the
    media dir. validate_template reports html_unsupported_css, html_unsupported_markup,
    html_font_unknown, html_missing_field and a measured html_overflow, and fails on html_too_large.
    get_template_schema documents the input and get_motion_catalog lists the CSS subset and four layout
    recipes under html.
  • analyze_sound: renders an sfx[].sound (composed layers, or a library preset varied by pitch, length,
    brightness and room) with the engine synth and returns its length, peak and RMS dBFS, raw
    pre-normalisation peak, spectral centroid, energy above 8 kHz / under 250 Hz, attack time and the sound
    advisories it raises, plus a spectrogram and a waveform PNG. An optional cue path
    (sections.intro.sfx[0]) seeds the render exactly like the mix; a silent sound reports −120 dB and
    sound_silent.
  • Typed template fields: compose_video and render_frames take fields (declared global.fields) as
    strings, numbers or booleans and refuse a missing required or ill-typed value before rendering;
    validate_template lists the declared contract as fields, reports field_undefined, field_unused,
    field_type_mismatch and field_missing_required, and with include: ["resolved"] returns the
    descriptor the given values produce or every value it would refuse.
  • open_in_builder: returns a https://leclap.dev/studio/builder#t=v1.… link that opens the template in
    the web builder for a person to edit, with mediaToRebind (local paths and uploads the browser cannot
    read) and warnings. The template travels compressed in the URL fragment, which browsers never send to a
    server; baseUrl targets a locale prefix or a local dev server, with a warning off https://leclap.dev.
  • transcribe_media: transcribe a local audio/video file's speech with whisper.cpp on the host → words, SRT,
    language, mean confidence and the "pin, then review" advice. compose_video resolves subtitles.transcribe
    before rendering. The model is never downloaded by the tool: the operator opts in once
    (leclap transcribe --download-model or LECLAP_WHISPER_DOWNLOAD=1).
  • edit_template: an RFC 6902 JSON Patch over inline template JSON under expectedRevision (stale →
    revision_conflict), all-or-nothing and validated after applying; returns the template, its new
    revision and changedPaths. It also takes effectProps ([{ section, props }], the former
    patch_template edits), applied after the operations in the same batch and checked by the effect
    backend, and then returns changedSections; operations is optional when effectProps is given. The
    web builder's WebMCP tools share the name, operations and revision, along with get_template_schema,
    get_motion_catalog, validate_template and render_frames.
  • compose_video runs the engine's output QC and returns it in structuredContent.qc with a one-line
    verdict.
  • get_motion_catalog returns the engine's motion catalog: kinetic typography presets with
    defaults, exits, stagger orders, the easing grammar, built-in tokens, art-direction rules and a starter.
    get_template_schema and the compose-video prompt point agents at it.
  • compose_video renders with the deterministic encoder profile.
  • validate_template returns every finding with its hint (structuredContent.errors with hint,
    suggestion, kind) and advisory motionWarnings; failing section assertions are validation errors.
  • get_motion_catalog also returns genre doctrine, scene blueprints, per-preset verbs and guidance,
    built-in themes, delivery platforms and the time-reference grammar; the registry manifest lists it.
  • get_template_schema describes global.platform; the compose-video prompt adds motion pacing rules.
  • New always-registered tools: extract_style (theme and style guide from a reference under the media
    dir), analyze_music (beat grid and cues), get_capabilities (local FFmpeg capability report) and
    render_frames (PNG frames, contact sheets, safe zones, variant and look grids).
  • get_motion_catalog accepts { query, kind? } and returns ranked matches; a query that matches nothing
    is appended to the catalog gap log (--catalog-gap-log / LECLAP_MCP_CATALOG_GAP_LOG, confined to the
    output dir).
  • compose_video and render_frames accept format; the template is resolved to that format before
    validation, the sandbox guard and the render. validate_template takes it for include.
  • validate_template adds featureWarnings from the local capability probe and reports
    partial_compressed.
  • probe_media reports hdr, colorPrimaries, colorTransfer, bitDepth, vfr and rotation.
  • The get_template_schema guide covers formats, footage editing, subtitles, voice/automation/sfx, music
    timing, motion roles and section purpose, trails, the new graphics, whips, lower-third styles, fills,
    layouts, right-to-left text and emoji.
  • get_motion_catalog lists the type: "fx" primitives (parameters, defaults, design intent, reduced
    motion) under fx, and the library APNGs under samples with the primitives that replace them; kind: "fx" searches them. get_template_schema and the compose-video prompt steer agents to compose motion
    from these primitives, tuned to the brief, and validate_template returns the sameness lint
    (fx_untuned, effect_repeated, library_animation_sample, effect_off_theme, decor_overload).

@leclap/cli 0.4.0

Choose a tag to compare

@heristop heristop released this 13 Oct 00:14

Added

  • Typed template fields: leclap render --set name=value (repeatable) fills a declared global.fields
    entry, coerced to its type (--field still works; --set wins); leclap resolve <template> [--set …]
    prints the descriptor a render would see and exits 1 listing every value it would refuse.
  • leclap studio <template> [--open] [--base <url>] [--json] prints a link that opens the template in the
    web builder. The template travels compressed in the URL fragment, which browsers never send to a server;
    media only this machine can read are listed to re-bind. --open starts the platform's browser opener;
    flags may come before the template.
  • leclap transcribe <template|media>: pins every subtitles.transcribe request of a template into
    subtitles.words with whisper.cpp, locally (--section, --language, --model tiny|base|small, --force,
    --out, --video section=path, --json), or prints a media file's words / SRT (--json, --srt).
    --download-model fetches the model once (checksum-verified); leclap render --download-model allows the
    same when a render resolves an unpinned request.
  • leclap samples list --category effects; the catalog now has 51 samples.
  • leclap render --qc prints output QC findings and exits non-zero on a failing check (--json
    includes the report); --cache <dir> reuses unchanged sections across renders. --output is written
    atomically and refused when it equals the template or a --video input.
  • leclap render --manifest writes <output>.manifest.json. Renders use the deterministic encoder
    profile by default (--no-deterministic turns it off).
  • leclap verify <manifest> [--rerender] checks a video against its render manifest, or re-renders the
    recorded template and compares the template, asset, filtergraph and output digests.
  • leclap validate prints a → hint line under each error that has a known fix, plus advisory motion
    pacing findings that never change the exit code; --json includes hint, suggestion, kind and
    motionWarnings.
  • leclap render --format <name> renders one composition of a template with formats; --formats all|a,b
    renders each to <output>-<format>.mp4.
  • leclap style <image|clip> [--json] [--out style-guide.md] derives a theme and style guide from a reference.
  • leclap beats <audio> [--json] measures tempo, the beat grid, confidence and drop/build/end cues.
  • leclap init --studio <dir> scaffolds a production folder; leclap studio status|pass tracks its gates.
  • leclap diagnose reports which features the FFmpeg build can run, with fixes; --json prints the
    capability report.
  • leclap snapshot, leclap compare and leclap timeline [--json] look at a template's frames, compare
    variants and list where everything sits; all take --format.

Changed

  • Built on ffmpeg-video-composer 3: leclap validate and leclap render reject templates with unknown
    keys (unknown_key) and text a bundled font cannot draw (font_missing_glyphs), which 0.3.0 accepted.

ffmpeg-video-composer 2.4.0

Choose a tag to compare

@heristop heristop released this 28 Sep 23:38
ce74dc2

Added

  • CompileReporter.onError(error): called with the cause when compile() resolves null, so a
    host can report why a render failed instead of a generic "no output": the error that stopped
    the director (a failed section, an FFmpeg command it rejected, a segment it could not probe) or
    the validation error compile() caught itself. A failure the director reported as a bare
    string arrives as that string, not JSON-quoted.
  • compileBrowser takes a loadFFmpegCore option, for a host that serves the ffmpeg.wasm
    core itself rather than having every visitor fetch it from unpkg (the LeClap web app now
    self-hosts it, which also keeps rendering available offline). FFmpegWasmAdapter takes the
    same loader as its second constructor argument, and FFMPEG_CORE_VERSION names the
    @ffmpeg/core version to serve. Without the option nothing changes: that pinned version,
    0.12.10, still comes from unpkg.
  • Any Google Fonts family, by name. A text font now also accepts
    { family, weight?, style? } (e.g. { "family": "Playfair Display", "weight": 700, "style": "italic" }), resolved from Google Fonts at render time. weight is 100..900 in steps
    of 100 (default 400), style is normal or italic. Registry ids and .ttf filenames keep
    working unchanged, and a typo in a registry id is still a validation error rather than a network
    lookup. A family that does not exist fails the render with an error naming it — a missing font is
    never silently swapped for another face.
  • A persistent font cache (Node). Faces resolved by family, and catalog fonts, are copied to
    ~/.cache/leclap/fonts (override with FVC_FONT_CACHE_DIR), so a repeat render of the same font
    needs no network. Entries are written atomically; an unwritable directory only costs the
    re-download.
  • isFontRef, FontRef and FontInput are exported from every entry point, and FontRefSchema /
    FontInputSchema from the Node entry.
  • TemplateValidator.getGeometryWarnings() catches templates that are valid but visually broken,
    before anything renders. It lowers each section through the renderer's own text and box filters
    (captions, title cards, lower thirds and their badges, global text overlays, authored
    drawtext/drawbox) and reports text that runs off the frame or out of the title-safe area,
    collides with other text on screen at the same time, is drawn under a band or panel that paints
    over it, is too small to read on a phone, has too little contrast against what it sits on, or sits
    over footage or an image with no box, outline or shadow. Each finding says what to change.
    Findings are advisory: they never enter errors and never change success. At most 20 are
    returned, worst first; when more exist, the last one says how many were left out.
  • Widths come from the real fonts. The Node entry exports createBundledFontLoader (bundled fonts
    first, then the LeClap asset catalog the renderer uses, 5s timeout per font) and
    nodeGeometryWarnings / geometryApproxNote, plus the GeometryWarning and FontLoader types.
    A finding drawn from an estimate says why: (approx: font unavailable, width estimated),
    (approx: {{ variable }} length unknown until render) or (approx: section duration assumed).
  • renderedGeometryWarnings() (Node entry) measures text contrast from rendered pixels, on request.
    The render-free check never sees a pixel: text over a picture, under a grade or a look is only
    flagged for lacking a box, outline or shadow. This renders the sections that hold text twice — the
    second time with every glyph recoloured, so the glyphs are exactly the pixels that change — reads
    one frame per piece of text where it rests, and scores it against the pixels around it (lower
    quartile, WCAG ratio). It reports text_low_contrast_rendered below 3:1, and over a fixed
    backdrop (colour card, picture) it replaces the render-free contrast and over-footage findings
    for that text. It costs seconds and needs a native FFmpeg with drawtext; without one, or when
    the render fails, it returns the render-free findings and says why in unavailable. The FFmpeg it
    would render with is asked for its filter list first (once per binary), so a build without
    libfreetype is named, with what to install, instead of failing every section mid-render.

Changed

  • AbstractMusic.process returns the track to mix. It resolves { rc, musicPath }: the
    track it was handed when that already covers the video, else a looped copy in the build
    directory. It no longer writes the loop over its input, so a custom AbstractMusic must now
    return musicPath; the bundled Node, browser and device adapters do.
  • The browser/WASM backend cannot request a TrueType face from Google, so it now rejects a template
    that names a font by family up front, before any section is encoded, with an error listing where.
    Ship the fonts you need, or use a bundled font id or a .ttf filename.

Fixed

  • In the browser, a segment FFmpeg fails to write no longer passes for a fresh render. ffmpeg.wasm's
    in-memory filesystem outlives a render, so an output an earlier render left at the same path
    counted as this command's output: a failed segment "succeeded", and a clip FFmpeg couldn't read
    rendered as the previous render's clip. FFmpegWasmAdapter now deletes a command's output path
    before running it, unless the command also reads that path.
  • A section that fails to build now fails the whole compile. The build error (an asset or font
    that can't be resolved, for instance) used to be logged and swallowed: the section rendered
    nothing, the concat skipped it, and compile() resolved output.mp4 without that section.
    compile() now resolves null on the first failing section, the browser and React Native
    entries reject, and the cause is a SectionError whose message names the section —
    Section "outro" failed: <cause>. A segment whose render exits non-zero or leaves no output
    file fails the compile the same way instead of being dropped from the concat.
  • Looping a background track shorter than the video no longer rewrites that track. The loop is
    written to the build directory and mixed from there. Earlier versions replaced the track in
    place — in the CLI's assets/musics, the MCP media library, the package's bundled tracks, or a
    device's staged copy — with a longer MP4/AAC file still named .mp3, so later renders no longer
    looped it. Tracks already rewritten are not repaired.
  • After a failed compile in a long-lived process (e.g. leclap render --watch), a compile with
    music enabled but no track resolved no longer mixes the failed compile's track.
  • A background track shorter than the video now loops whatever its container. The Node loop
    joined copies of the file byte for byte, which only loops a bare MP3 stream: an MP4 container
    named .mp3 (pop.mp3, point-being.mp3 and future-bass-energy.mp3 in the bundled library,
    or a track an earlier version rewrote in place) played once and then fell silent, and an MP3
    with embedded cover art (air-prelude.mp3, anxiety.mp3, arcadia.mp3) failed the render,
    on device too. Both loops now repeat the track's demuxed audio alone.
  • A failed FFmpeg command on Node (system FFmpeg or ffmpeg-static) now reports the lines
    FFmpeg failed on, instead of its whole stderr: ten lines of version banner, then a dump of
    every input and of the stream mapping, ahead of the line naming the problem. Both adapters
    run FFmpeg at -loglevel error and keep the last 20 lines of what it prints.
  • Bundled fonts now resolve from a built dist/ (the CLI, the MCP server, any installed consumer),
    not only when running from source.
  • A bundled asset name carrying a path separator or a leading dot is no longer resolved, so a
    descriptor font such as ../../etc/passwd.ttf cannot read a file outside the library.