mediaframe

A common media-stream descriptor vocabulary for media processing pipelines — codec, pixel-format, colour, frame, audio, subtitle, container, track-disposition, EXIF capture, and BCP-47 language metadata.

Overview

A common media-stream descriptor vocabulary for media processing pipelines. The codec module covers video + audio + subtitle codec identifiers. Frame-side modules — pixel-format / colour / frame / source — cover the video pipeline. Stream-side modules cover audio (channel layout, sample / container format, bit-rate mode, EBU R128 loudness, fingerprint, tags + cover art), subtitle (format + track origin), top-level multimedia container format, FFmpeg track dispositions, EXIF capture metadata (device + geolocation), and BCP-47 language. Pure data types: no SIMD, no decoder, no codec implementation, no math — just the shared spine that a color- conversion library, a decoder backend, and a frame consumer can all speak to without agreeing on anything heavier.

What it provides

• codec — VideoCodec, AudioCodec, SubtitleCodec stream- descriptor enums covering every codec FFmpeg n8.1 knows under each media type, plus an Other(SmolStr) lossless escape for codecs added upstream before the table is regenerated. Generated by cargo xtask gen-codec from the vendored FFmpeg table — see xtask. Requires alloc (gated behind any(feature = "std", feature = "alloc") for the Other(SmolStr) arm).

• color — ITU-T H.273 colour-metadata enums (color::Matrix, color::Primaries, color::Transfer, color::DynamicRange, color::ChromaLocation) bundled into color::Info, with FFmpeg-exact code points and a lossless Unknown(u32) arm on each. Plus DcpTargetGamut (DCI-XYZ target-gamut selection), Rotation, HDR static side- data (ContentLightLevel, ChromaCoord, MasteringDisplay, HdrStaticMetadata per SMPTE ST 2086 / FFmpeg HDR10), and DolbyVisionConfig (FFmpeg AVDOVIDecoderConfigurationRecord). Colour-enum numbering is CI-checked against the pinned FFmpeg header by cargo xtask check.

• pixel_format — single PixelFormat enum covering every pixel format in FFmpeg n8.1's AVPixelFormat (254 variants excluding GPU-resident HW formats) plus cinema-RAW additions. Coverage is verified by cargo xtask check against vendored pixfmt.h slugs — see xtask.

• frame — structural primitives (Dimensions, Rect, Plane<B>), exact-ratio building blocks (Rational, FrameRate, SampleAspectRatio as a Rational newtype), stream- descriptor metadata (FieldOrder, StereoMode — both with lossless Unknown(u32)), BayerPattern for color-filter-array description, the runtime-tagged VideoFrame<P, B>, and the orthogonal TimestampedFrame<F> wrapper bundling mediatime::Timestamp PTS + duration around any inner frame shape. Plus per-format typed *Frame<'a, BE> zero-copy borrow views + *FrameError validation (feature-gated).

• source — per-format marker ZSTs (Yuv420p, Nv12, Rgb24, …), *Row<'a> borrows, *Sink subtraits, and *_to walker fns that iterate Frame → Row → PixelSink. The walker macro generates the marker / Row / Sink / walker quartet uniformly. Marker construction is Foo::new() (private () field locks shape evolution to additive changes).

• container — top-level multimedia container::Format enum (Mov, Mp4, Mkv, Webm, Avi, MpegTs, …) with an Other(SmolStr) lossless escape. Distinct from audio-only containers, which live on audio::ContainerFormat. Requires the alloc feature (std includes it).

• audio — audio-stream descriptor vocabulary: ChannelLayout (FFmpeg AV_CHANNEL_LAYOUT_* bitmask), ContainerFormat (audio-only containers: mp3 / flac / wav / …), SampleFormat, BitRateMode (CBR / VBR / ABR), Loudness (EBU R128 integrated / true-peak / LRA), Fingerprint, embedded-metadata Tags, and CoverArt payload. Requires the alloc feature.

• subtitle — subtitle::Format (Srt / WebVtt / Ass / image-based HdmvPgs / …) + TrackOrigin (Embedded / Sidecar / External). Pure stream-descriptor vocabulary — no per-cue content. Requires the alloc feature.

• lang — validated BCP-47 Language tag (language subtag + optional script + optional region), wrapping icu_locale_core's TinyAsciiStr-backed subtag types so the value is Copy / heap-free. Validates via LanguageIdentifier::try_from_bytes. Requires the alloc feature.

• disposition — TrackDisposition bitflags mirroring FFmpeg's AV_DISPOSITION_* (DEFAULT, FORCED, HEARING_IMPAIRED, …). Bit values are append-only, never renumbered; unknown bits round-trip losslessly through to_u32 / from_u32 via bitflags::from_bits_retain semantics. No alloc requirement — pure Copy bitflags.

• capture — EXIF / capture-metadata vocabulary: Device (capture device name / model / serial) and GeoLocation (latitude / longitude / altitude with ISO-6709 parse + format). Requires the alloc feature.

• buffa — optional buffa wire serialization (hand-written Message / DefaultInstance impls, no codegen) for the colour / frame / HDR vocabulary so downstream proto schemas can extern-map .mediaframe.v1 → ::mediaframe. Off by default — enable with --features buffa.

• serde — optional serde::{Serialize, Deserialize} for the whole descriptor vocabulary. Wire shape by type:

  • Open codec / format enums (codec::*, container::Format, subtitle::Format, audio::{ChannelLayout, ContainerFormat}) — the as_str() slug, unknown slugs ride Other.
  • FFmpeg-coded enums with an Unknown(u32) arm (colour, pixel-format, frame coded enums, TrackDisposition) — the to_u32() integer; unknown codes round-trip via Unknown (no slug form).
  • Strictly-closed coded enums (subtitle::TrackOrigin, audio::BitRateMode) — the to_u32() integer, but unknown codes are rejected as serde errors rather than collapsing to the default.
  • audio::SampleFormat (both Unknown(u32) and Other(SmolStr)) — bespoke: human-readable formats emit a string for named/Other and a number for Unknown; binary formats use a tagged {Code, Slug} wire.
  • lang::Language — its BCP-47 string. Validated structs (GeoLocation / Fingerprint / CoverArt) deserialize through their checking constructors.

  Orthogonal to the capability tiers (no-alloc Copy types included). Off by default — enable with --features serde.

• arbitrary — optional arbitrary::Arbitrary for the descriptor vocabulary (fuzzing / property tests), hand-written through each type's public constructors so try_new-validated types come out valid and open Other(_) arms are exercised. Generated values are wire-canonical (every value survives a serde round-trip). Std-only — the arbitrary crate itself is std-based — so this serves host-side fuzzing, not an embedded target. Off by default — enable with --features arbitrary.

• quickcheck — optional native quickcheck::Arbitrary for the same surface, via the quickcheck-richderive derive (each type carries #[quickcheck(arbitrary = "…")] pointing at a fn(&mut Gen) -> Self helper). Independent of arbitrary. Std-only. Off by default — enable with --features quickcheck.

• PixelSink + SourceFormat sealed traits re-exported at the crate root.

Installation

[dependencies]
# Lean — codec + color + pixel_format + frame primitives.
# Adds `mediatime` + `derive_more` + `smol_str` (codec `Other` arm).
mediaframe = "0.1"

Opt into typed *Frame<'a> borrow views + the per-format source::* walker quartet per family:

mediaframe = { version = "0.1", features = ["yuv-planar", "rgb"] }

Or take everything via the umbrella:

mediaframe = { version = "0.1", features = ["frame"] }

Per-family feature flags

Enable only the families your pipeline actually consumes — each flag pulls in just the matching *Frame validators, *Row borrow types, marker ZSTs, walker fn, and Sink subtraits. The frame umbrella enables all of them at once.

Feature	Formats
yuv-planar	Yuv420p / 422p / 444p / 440p / 411p / 410p + 9-16 bit
yuv-semi-planar	Nv12 / 16 / 21 / 24 / 42, P010 / 210 / 410 family
yuva	YUVA planar 8-bit + 9-16 bit
yuv-packed	Yuyv422, Uyvy422, Yvyu422, Uyyvyy411
yuv-444-packed	V410, Xv30, Xv36, Ayuv64, Vuya, Vuyx, V30X
y2xx	Y210 / Y212 / Y216
v210	V210
rgb	Rgb24 / Bgr24 / Rgba / Bgra + 10-bit + 16-bit
rgb-float	Rgbf32 / Rgbf16 + Rgbaf16/f32
rgb-legacy	Rgb444 / 555 / 565 + Bgr counterparts
gbr	Gbrp / Gbrap + 9-16 bit + float
gray	Gray8 / 9-16 bit / f32, Ya8 / Ya16
bayer	Bayer 8 / 10 / 12 / 14 / 16-bit × 4 patterns
xyz	Xyz12 (DCI-XYZ)
mono	Monoblack / Monowhite / Pal8
frame	umbrella — enables every sub-feature above

Deps pulled in by family features:

• thiserror — every per-family feature (for *FrameError).
• half — rgb-float, gbr, gray (for half::f16).
• derive_more try_unwrap / unwrap — yuv-444-packed, y2xx.

no_std

# Pure no_std — just enums, marker ZSTs, structural primitives.
mediaframe = { version = "0.1", default-features = false }

# no_std + alloc — adds Vec-using helpers and tests.
mediaframe = { version = "0.1", default-features = false, features = ["alloc"] }

The color, cfa, and pixel_format modules work without alloc (pure enums / Copy types). The codec module is gated on any(feature = "std", feature = "alloc") because of its Other(SmolStr) arm. Per-family frame / source features work under no_std + alloc.

xtask

cargo xtask sync fetches the pinned FFmpeg release tag (currently n8.1) and refreshes the vendored tables under xtask/vendor/:

• ffmpeg-pixfmts.txt — every AV_PIX_FMT_<NAME> slug from libavutil/pixfmt.h.
• ffmpeg-color.txt — every colour-enum code point from libavutil/pixfmt.h (matrix / primaries / transfer / range / chroma location).
• ffmpeg-codecs.txt — every codec identifier under media types video / audio / subtitle from libavcodec/codec_desc.c.

cargo xtask gen-codec regenerates src/codec.rs from ffmpeg-codecs.txt — one VideoCodec / AudioCodec / SubtitleCodec enum variant per FFmpeg codec.

cargo xtask check diffs the vendored tables against the in-tree enums and fails on any missing variant or numbering drift: PixelFormat::as_str() vs ffmpeg-pixfmts.txt, colour-enum code points vs ffmpeg-color.txt, and the codec enum variants vs ffmpeg-codecs.txt. CI runs this so every enum stays exhaustive against the pinned FFmpeg version.

Vendoring only the slug / code-point lists (not the LGPL FFmpeg headers verbatim) sidesteps the license question.

License

mediaframe is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Copyright (c) 2026 FinDIT Studio authors.