Video decode, encode and playback for JVM desktop apps. FFmpeg through
hand-written Java FFM (Panama) bindings: frames out as raw RGBA or Skia
images with a Compose Desktop surface on top, and frames back in to a
muxed file. No JNI wrapper stacks, no embedded player engines. The bundled
FFmpeg is built with --disable-network, so skinema performs no network
I/O of its own -- it works on the file you hand it and nothing more.
(The player, the decoders and the transcoder also take a MediaSource, so
bytes fetched anywhere play directly, a stream that only reads forward
included; see Streams.)
The same API runs on Android, on the platform's own engine: Media3 over the
phone's decoders rather than FFmpeg, so shared code plays, decodes and writes
through one set of calls on both. Media3 brings Guava and a few androidx
libraries with it, and no native code ships for Android. Pass
hardware = HwAccel.AUTO there for the hardware decoders, since the default is
the desktop's software one. What differs, and why, is in
the Android guide.
val player = VideoPlayer(MediaInput.File(Path.of("background.webm")), loop = true)
// Compose Desktop:
VideoSurface(player, Modifier.fillMaxSize(), scale = VideoScale.Cover)
// Or poll frames yourself from any render loop:
player.acquireFrame()?.let { frame -> /* frame.rgba, frame.width, ... */ }positionNanos() and durationNanos carry a timeline (duration is
null for an animated webp until one lap has played, since the format
declares none and the length is only known once it has been read); chapters, tags and
coverArt carry the rest of the container's metadata -- the cover
ships as the stored png/jpeg bytes for your own image stack. A file
whose only video stream is the embedded cover plays frameless, art
exposed, sound intact. Phone footage's orientation surfaces as
rotationDegrees; VideoSurface applies it, a consumer drawing
frames itself must do the same.
Full developer documentation -- a consumer guide and the contributor internals -- lives in docs/.
implementation("dev.hivens:skinema-compose:0.9.0") // brings -core and -skiko
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-linux-x64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-linux-arm64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-linux-musl-x64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-linux-musl-arm64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-windows-x64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-windows-arm64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-macos-arm64")
runtimeOnly("dev.hivens:skinema-natives:9.0.1-1:decode-macos-x64")On Android the same coordinate resolves to the Android build, and there are no natives to add:
implementation("dev.hivens:skinema-compose:0.9.0") // brings -core; Media3 underneathThat one is Gradle reading the module metadata. A Maven, sbt or Bazel build
reads the root POM instead, which for a multiplatform module carries no
classes, so from 0.9.0 such a build names the desktop artifacts directly:
skinema-core-jvm and skinema-compose-jvm.
0.9.0 compiles against Skiko 0.150.2, which is what Compose Multiplatform
1.12.2 ships. Skiko is compileOnly here, so the copy that runs is the one
your Compose brings: that makes Compose 1.12 a floor rather than a
preference. On an older Compose, take 0.7.0.
The two versions are independent, and that is deliberate. skinema-natives
is versioned as the FFmpeg build it carries plus a repack revision
(<ffmpeg>-<revision>), so it moves only when the bundles themselves change
-- most library releases leave it untouched. Take the natives version the
release notes name and pair it with any library version that names it.
Independent, but not unrelated: the bindings are valid for one set of FFmpeg
soname majors, so a library release that moves the pin needs the natives that
carry it. Gradle consumers do not have to track that by hand -- skinema-core
publishes a constraint naming the natives version it was built against, so a
stale pin is raised to it rather than failing at runtime. Override with
version { strictly("...") } to hold a specific bundle (a local build, a
repack under test); the runtime refuses a mismatched major either way.
The natives classifier is <tier>-<platform>: pick one tier for the
platforms you target. The tier sets what the bundle carries -- and its
license:
| Tier | Carries | License |
|---|---|---|
core |
the modern essentials (H.264/HEVC/VP8/VP9/AV1, mainstream audio, images), no subtitles, no GPU decode | LGPL |
decode |
core + GPU decode + subtitles + every format FFmpeg decodes natively; and writing -- AV1 video, Opus/AAC/FLAC audio, the mp4/mkv/webm containers and the audio-only .opus/.flac/.wav ones, plus GPU H.264/HEVC encode on Linux | LGPL |
full |
decode + H.264 and HEVC software encode (x264/x265) | GPL |
core is the lean tier -- the modern codecs only, no subtitles, no GPU
decode -- for apps that just play current video. It is also the portable one:
it needs nothing from the host but the C library, so it loads in a bare
container and on a distribution that keeps no libraries at the usual paths.
The desktop tiers carry GPU decode, which means they need libva alongside
fontconfig. decode (used above) is the complete LGPL
player: it adds the libass subtitle stack and the broad formats set (the
legacy and broadcast containers and codecs in "What it plays" below). It is
also the complete LGPL WRITER -- SVT-AV1 and libopus are BSD, so AV1 video
and Opus audio come out of a bundle carrying no GPL at all. What full adds
is software H.264 and HEVC specifically, and that is what makes it GPL,
since x264 and x265 are; HEVC encode ships everywhere since the x265 source
patch (issue #22). On first use the jars unpack to a per-user cache. Without a
natives jar, skinema looks for matching system libraries -- fine for
development, not what you ship.
On JDK 24+ launch with --enable-native-access=ALL-UNNAMED (or grant
your named module): skinema's FFM calls -- and Skiko's, if you use
Compose -- are restricted methods that otherwise warn on every run.
Every tier ships for eight platforms -- x64 and arm64 on all three desktop operating systems, and both C libraries on Linux:
| Platform | Bundle built | Acceptance suite |
|---|---|---|
linux-x64 |
on metal | on metal |
linux-arm64 |
on metal | on metal |
linux-musl-x64 |
in Alpine, on metal | on metal, musl JVM |
linux-musl-arm64 |
in Alpine, on metal | on metal, musl JVM |
windows-x64 |
on metal (MSYS2 MINGW64) | on metal |
windows-arm64 |
on metal (MSYS2 CLANGARM64) | on metal |
macos-arm64 |
on metal | on metal |
macos-x64 |
cross-compiled on an arm mac | none |
Linux ships twice because a glibc shared object cannot load into a musl
process at all -- Alpine and Void-musl take the linux-musl-* bundles, and
nativesPlatform() picks between them by reading which C library the running
JVM is itself linked against, not by looking at what the system has installed.
arm64 is first-class here, not an afterthought: linux-arm64 and
windows-arm64 are built on real arm machines and must pass the same
acceptance suite as their x64 siblings before a bundle uploads. macos-x64
is the one exception, and it runs the other way -- Intel mac runners are
scarce, so it cross-compiles on an arm mac and ships without an on-metal
run, because an arm JVM cannot load x86_64 dylibs. That is what
community-tier support means here.
How old a system the bundles still load on. glibc 2.39 for linux-arm64
full, 2.38 for the other glibc bundles, 2.35 for core-linux-x64 -- so
Ubuntu 24.04, Debian 13 and Fedora 39 upward for the desktop tiers, and not
RHEL 9. core-linux-x64 alone still loads on Ubuntu 22.04.
macOS bundles are built for 14.0, so Sonoma upward. The linux-musl-*
bundles have no such floor: musl does not version its symbols. The decode
and full Linux bundles want libfontconfig from the host -- libass's font
provider links it, and without it none of their libraries load, which is
worth knowing before putting them in a headless container. core needs only
libc and libm and has no such requirement. VAAPI used to be a second such
requirement and is not any more: those bundles carry libva, libva-drm
and libdrm themselves, so a machine with no libva installed still loads
them. The host's own copy is still preferred where there is one, because the
driver on the machine is versioned against it rather than against ours.
| Containers | mp4/mov/m4a, webm/mkv, avi, MPEG-PS/TS, flv, asf/wmv, dv, RealMedia, ogg, mp3, flac, wav, gif, apng, webp, raw ac3/eac3 |
| Video | H.264, HEVC, H.266/VVC, VP8, VP9 (incl. webm alpha), AV1; MPEG-1/2, MPEG-4 Part 2, VC-1, WMV 7-9, H.263, Theora, ProRes, DNxHD, FFV1, RealVideo, Cinepak, Indeo, VP6; MJPEG |
| Animated images | GIF, APNG, animated WebP |
| Audio | AAC, AC-3/E-AC-3, DTS, TrueHD, ALAC, Opus, Vorbis, MP1/MP2/MP3, FLAC, WMA (v1/v2/Pro), AMR, WavPack, Monkey's Audio, TTA, ADPCM, G.72x, RealAudio, ATRAC, GSM, WAV PCM -- the device clock masters A/V sync |
| Subtitles | ASS/SSA, SRT, mov_text, WebVTT (libass-rendered); PGS, VobSub, DVB (bitmap); CEA-608/708 closed captions, read out of the video bitstream; external .srt/.ass |
| Pixels out | RGBA8888, straight alpha, exact-pts pacing, BT.601/709/2020 matrix and range honored, PQ/HLG tone-mapped to SDR |
The table names what is worth naming, not everything that is there. The decode and full tiers carry every decoder, demuxer and parser FFmpeg builds without an external library -- some five hundred of them -- so the legacy, broadcast and archival formats are in whether or not a row above mentions them. The lean core tier is the one with a list: the modern essentials (H.264/HEVC/VP8/VP9/AV1 and the mainstream audio), and nothing else. H.266/VVC decodes through FFmpeg's native decoder (CPU-only, no GPU path yet).
What is not there is short and stays short: anything needing a library the bundle does not carry -- JPEG XL, SVG, tracker music, DVD and Blu-ray navigation, teletext -- and TTML, which FFmpeg has no decoder for at all.
Audio: pass audio = true to VideoPlayer -- aac, ac3/eac3, alac,
opus, vorbis, mp3, flac and WAV pcm (16/24/32-bit and float) decode
through the same bindings, and the audio device becomes the player's
clock (video follows sound, never the reverse). The sound reaches the
device in the shape the file has it: six channels stay six, twenty-four
bits stay twenty-four, and the rate is never converted. A device that
cannot take that shape refuses, the player walks down to one it can, and
activeAudioFormat reports what was actually taken rather than what was
asked for. audioChannels = ChannelPreference.Stereo folds first for a
consumer that knows the fold is wanted, since a device accepting six
channels is not evidence that six speakers exist. Audio-only files play frameless. Files with several audio
tracks expose them (audioTracks, language and title included) and
switch in place (selectAudioTrack) -- the picture keeps playing and
the sound re-anchors at the playhead. setRate plays at 0.5x-4x with
the pitch preserved (FFmpeg's atempo). volume opens at a gain rather
than at full, and is re-applied to every line the player opens -- a
track switch, a device coming back after an outage -- so a quiet player
stays quiet; setVolume changes it live. Opening at one is not the same
as setting it straight after: the sink opens and takes its first chunk
on the audio thread's own schedule, so a call racing that has already
lost. HDR content (PQ/HLG over BT.2020) is tone-mapped to SDR on the
decode path -- the transfer is inverted, highlights roll off against
BT.2408 diffuse white, and the gamut maps to BT.709 -- so it no longer
plays washed out. This is a
fixed reasonable-look mapping, not a metadata-accurate one: the stream's
HDR metadata (MaxCLL, mastering-display luminance, HDR10+, Dolby Vision)
is not read, so the roll-off uses an assumed peak rather than the
content's. Native-HDR passthrough (driving an HDR display) is out of scope.
Subtitles: subtitleTracks enumerates what the container carries
(language, title, default/forced), and grows a CEA-608/708 track the moment
one is seen -- closed captions are not a stream but ATSC A53 payload inside
the H.264/HEVC bitstream, so no container can be asked whether a file has
them and nothing is advertised until a frame answers; selectSubtitleTrack
turns one on
-- off by default, nothing subtitle-related runs until then. Text
tracks (ASS/SSA, SRT, mov_text, WebVTT) render through libass with the
full typesetting, mkv-embedded fonts included; bitmap tracks (PGS,
VobSub, DVB) decode to pixels and need no libass at all. External .srt and
.ass files join via addExternalSubtitles on the same timeline.
VideoSurface composites the overlay and keeps glyphs crisp at window
size; a consumer drawing frames itself polls acquireSubtitles -- the
acquireFrame contract for text. libass is an optional capability:
without it text tracks refuse selection and everything else plays on.
Text rendering uses the system's fonts (fontconfig, DirectWrite,
CoreText); a fontless headless box renders blank overlays.
MediaWriter is the player inverted -- RGBA8888 frames and S16LE stereo PCM
in, a muxed file out -- and Transcoder joins the two halves to convert a
file. What a bundle can write is set by its tier, and there the split is about
licence rather than size: everything below except the software H.264/HEVC pair
is LGPL or BSD and rides decode.
| Video | AV1 (SVT-AV1), and on Linux H.264/HEVC on the GPU through VAAPI -- both in decode; H.264 and HEVC (x264/x265) in full, which is what makes that tier GPL |
| Audio | AAC, Opus, FLAC, WAV PCM |
| Containers | mp4/mov, mkv and webm; .opus, .flac and .wav for a file with sound and no picture |
| Pixels in | RGBA8888, reverse-scaled into whatever the encoder accepts; the matrix the conversion used is tagged on the file, so a player reading the tag and one guessing from the geometry agree |
Nothing is assumed about an encoder: the pixel format, sample format, channel layout and sample rate are all negotiated against what it advertises, and one it cannot take is refused by name rather than as a bare errno. Hardware encode is fail-closed -- naming a GPU encoder that cannot open throws instead of quietly falling back to software. See the encoding guide.
- Fail closed. A file the pipeline cannot handle surfaces as
VideoPlayer.State.Failed-- show your fallback. No partial recovery, no garbage frames, no hangs. - Drop late. A slow consumer skips frames; the clock never lags.
- It stops for a consumer that stops looking. A window nobody can
see is noticed, and the player stops decoding and converting for it.
VideoSurfacetells by how long its window takes to answer for a frame and reports it throughreportVisible. Measured under XWayland on Hyprland, that catches a window left on a workspace that is not on screen. Behind a fullscreen window it answers the same way, and a covered macOS window is expected to, going by Skiko's source. Minimising is taken from the lifecycle everywhere. A covered window on Windows most likely keeps drawing at full speed and is not seen. Outside Compose, a mailbox that was being read and then is not is noticed on its own. What stops is the policy's to say, and the default (WhenUnwatched.FollowSound, which replacedFreezeas the default) does what a browser does with a tab put behind another: a player you can hear plays on without its picture, which catches up with the sound when it is wanted again, and a silent one or one at volume zero pauses.FreezeandKeepTimefix one answer for every case, andsetPresentingmarks the moment exactly. - Seeks answer immediately. An exact seek previews its keyframe
while the frame-precise landing decodes behind it;
exact = falselands on the keyframe outright -- picture and sound at once, position as coarse as the file's keyframe spacing. Skip buttons want inexact; timeline scrubbing wants exact.stepForward/stepBackwardmove a single frame and leave the player paused on it. - Two threads per player, four with sound. Decode fills a small frame queue and a pacer presents from it; sound adds an audio thread and its watchdog, and a selected subtitle track a fifth. Players are independent and self-synced; play as many as your CPU affords (a desktop comfortably runs dozens of 1080p30 streams).
- A player can open paused.
startPaused = truesettlesState.Pausedinstead ofPlayingand stays on the first frame -- which is on screen, because a poster frame with no picture is a black rectangle. With sound, nothing reaches the line at all: the audio side is told at construction rather than paused by a command, which would arrive after the device had already started.resumeis what starts the file, and this is the one pauseWhenUnwatchednever lifts. - One player, many files.
setSourceputs the next file on the player you already have -- the same threads, the same audio line, the same volume, rate, looping and subtitle canvas -- so a playlist is a list you own plus one call, and the queue, the order and what "next" means stay yours. A file that will not open is refused (sourceFailure) and the one playing carries on, which is what a queue needs from one broken item.loopis live in the same way: turning repeat on no longer costs the position by rebuilding the player. - Read-ahead is opt-in.
readAheadFrames(default 1) holds that many decoded frames of inventory, so a decode stall does not stall the screen while inventory lasts. Each step of depth costs one full RGBA frame of memory -- 8.3 MB at 1080p. Backgrounds stay at 1; a player UI wants 3-5.
| Module | Contents | Floor |
|---|---|---|
skinema-core |
FFM bindings, demux/decode, pacing, VideoPlayer; on Android the Media3 engine |
JDK 22; Android 10 (API 29) |
skinema-skiko |
VideoFrameImage/SubtitleOverlayImage: pixels as org.jetbrains.skia.Image |
Skiko (provided by your Compose) |
skinema-compose |
VideoSurface, rememberPlayerState, VideoScale, VideoFrameHolder |
Compose 1.12, on the desktop and Android |
skinema-natives |
trimmed FFmpeg in tiers, classifier jar per tier+platform | -- |
ROADMAP.md is the project's working memory: every architectural decision, with its reasoning, lives there.
Decode correctness for well-formed files in the formats above is a bug when broken. Exotic, damaged or adversarial files get the fail-closed treatment by design; compatibility issues beyond that are triaged at the maintainer's discretion.
skinema itself (core, skiko, compose) is Apache-2.0. The natives bundle's
license is set by its tier (see Dependencies): core and decode are LGPL
-- decode does carry encoders, but only ones that add no GPL surface (the
BSD SVT-AV1 and libopus, FFmpeg's own AAC and FLAC, and the VAAPI encoders
whose codec lives in the GPU driver); the full tier adds the x264/x265
software encoders, which are GPL, so its FFmpeg build is configured
--enable-gpl and its libraries are GPL. Either way the natives ride in separate per-tier,
per-platform classifier jars, dynamically linked, never statically embedded
into the Apache code, and every bundle ships its license texts (the GPL text
on full, LGPL on the rest). An application that distributes the full
natives takes on FFmpeg's GPL obligations, as anyone distributing a GPL
FFmpeg build does; a consumer that needs to stay LGPL takes core or
decode. skinema's own Apache code is unaffected either way.
libvpx, dav1d, SVT-AV1 and libopus are BSD-family -- SVT-AV1 additionally
carries the Alliance for Open Media patent grant, whose text ships beside its
license; x264 and x265 are GPL (the reason the full build is
--enable-gpl). libass (ISC) ships with FreeType and
HarfBuzz folded in
-- portions of the bundled software are copyright The FreeType Project
(freetype.org), licensed under the FreeType License -- while FriBidi stays a
separate shared library precisely because it is LGPL. On Windows, where
libtool cannot fold a static archive into a DLL, FreeType and HarfBuzz ship
as their own DLLs instead, and the bundle also carries the MinGW runtime
they link -- zlib, bzip2, and lzma (permissive), winpthread, and libiconv
(LGPL, dynamically linked) -- each with its license text.