Client-side media: convert HEIC/HEIF image sequences (Live Photo / burst) to video

[adamsilverstein]

What

Adds client-side support for HEIC/HEIF image sequences — Apple Live Photos (HEVC sequences) and Android bursts (image/heic-sequence, image/heif-sequence; .heics/.heifs) — by converting them to a web-safe video (MP4/WebM) in the browser before upload.

Resolves part of #79642.

Today these files cannot be handled at all: GD/Imagick decode only a single frame, so WordPress core deliberately collapses sequence uploads to one still frame (see Trac #65297 / wordpress-develop#12352). Client-side conversion is the only path that preserves the motion.

Note

Builds on #78410 (Animated GIF → video via mediabunny), now merged to trunk. The video encoding reuses the @wordpress/video-conversion worker introduced there.

How

The existing still-HEIC pipeline only parses the meta/iloc/pitm structure of a single image. Image sequences instead store their frames as samples of an ISOBMFF movie track (moov → trak → mdia → minf → stbl), so the work splits into three pieces:

1. Demux (packages/upload-media/src/heic-parser.ts): new parseHeicSequence() walks the sample tables (stsd/hvcC, stsz, stsc, stco/co64, stts, stss) and extracts the temporal HEVC access units, their durations, and keyframe flags. Reuses the existing Reader/buildCodecString helpers.
2. Decode + encode (@wordpress/video-conversion): the GIF encode loop is refactored into a shared encodeFramesToVideo helper, and convertHeicSequenceToVideo decodes the frames with the WebCodecs VideoDecoder (platform HEVC codec) and re-encodes them with mediabunny — exactly like the GIF path, but sourced from a VideoDecoder instead of an ImageDecoder. Decoding is pipelined with a bounded look-ahead (8 frames) so a long sequence never materializes all of its full-res frames at once.
3. Route (packages/upload-media/src/store/private-actions.ts): prepareItem detects the sequence MIME types and converts them to video before upload. The video becomes the attachment. When WebCodecs is unavailable it falls through to upload the original file (the server collapses it to a still).

Parsing happens on the main thread (keeping heic-parser in @wordpress/upload-media); the heavy decode + encode runs in the worker, off the main thread.

flowchart LR
    A[".heics / .heifs"] --> B["parseHeicSequence()<br/>(main thread)"]
    B -->|HEVC samples| C["VideoDecoder<br/>(worker)"]
    C -->|VideoFrames| D["mediabunny encode<br/>(worker)"]
    D --> E["MP4 / WebM"]
    E --> F["upload as video attachment"]

Loading

Prior art researched

• swissspidy's media-experiments plugin recognizes the sequence brands (msf1/hevc/hevx) but decodes only the first frame via libheif-js and discards the rest; its only image→video path is GIF→video via ffmpeg.wasm. No sequence→video support to reuse.
• mediabunny is an encoder/muxer; it does not demux HEIF sequences, but its encode side (VideoSampleSource + Output + Mp4OutputFormat/WebMOutputFormat) is exactly what this reuses. The demuxer is the net-new piece.

Testing instructions

• npm run test:unit packages/upload-media packages/video-conversion
• The demuxer is unit-tested against a real 120-frame msf1.heic fixture (256×144, 25fps, 1 keyframe + 119 delta frames), verifying frame count, codec config, sync/delta flags, and per-frame timing.
• Manual: upload an iPhone Live Photo exported as .heics (or an Android burst) into the editor with client-side media enabled; it should upload as a short looping MP4/WebM.

Open questions / follow-ups (from #79642)

• UX: the result is currently a plain video attachment. Whether a Live Photo should become a looping <video> (e.g. the GIF block variation) vs. a still-with-motion is left as a follow-up.
• Original preservation: the original .heics is not yet sideloaded alongside the video (the motion is preserved in the video itself).
• B-frame reordering: the look-ahead window handles typical sequences (the common case, and the test fixture, have no ctts); deeply reordered streams would need composition-time handling.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Size Change: +2.03 kB (+0.03%)

Total Size: 7.68 MB

📦 View Changed

Filename	Size	Change
build/modules/video-conversion/worker.min.js	83.7 kB	+793 B (+0.96%)
build/scripts/upload-media/index.min.js	15.6 kB	+1.23 kB (+8.57%)	🔍

_{compressed-size-action}

[github-actions]

Flaky tests detected in 0b27ad8.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/28458230046
📝 Reported issues:

• [Flaky Test] Should save the changes #78689 in /test/e2e/specs/editor/plugins/wp-editor-meta-box.spec.js