Skip to content

Releases: MediaCacheDev/alpha-video

alpha-video 1.0.1

Choose a tag to compare

@danielkpl2 danielkpl2 released this 26 Oct 10:11

For every package but the alpha-video pod, which stays at 1.0.0: its subspecs already included range-source. dev.mediacache:alpha-video-vpx and alpha-video-av1 are unchanged, at 1.0.1 to keep one version.

  • On react-native-web, source.dataSource names a data source registered with range-source-web's RangeDataSources, as it names a native one on Android and iOS, so the same source={{ uri, dataSource }} plays on all three; MediaCache registers as 'mediacache' with cache.registerRangeDataSource(). An unregistered name reports onError. Its bytes are stored only with storage="disk" or "memory", as on Android and iOS.
  • Web: WebM files that need the WebCodecs engine play in apps bundled by Metro (Expo and react-native-web), and the CommonJS build loads the demuxer. Its WebAssembly glue used import.meta.url, a syntax error outside an ES module, so the demuxer failed to load with "Requiring unknown module" or "Cannot use 'import.meta' outside a module".
  • React Native, Android: the new architecture builds with React Native 0.76 and earlier, whose codegen task has no jsInputFiles property to set, so the build failed.
  • iOS, Swift Package Manager: range-source comes with a new product, AlphaVideoSource, which apps add beside the codec products, and leave out to go without range-source. It replaces the package trait Source, which Xcode before 26.4 doesn't apply, default traits included: 1.0.0's notes said Xcode apps always had range-source, but they never got it. Apps without AlphaVideoSource (or range-source's own RangeSource product) download URLs whole, and their data sources report NO_SOURCE_MODULE. A package that depended on the player with traits: [] drops that and doesn't add AlphaVideoSource.
  • Android: taking an AlphaVideoView off the screen after release() no longer crashes the app with a RejectedExecutionException. release() now frees the view's OpenGL surface and context itself, which nothing did once its render thread had stopped.

Where

  • Swift Package Manager: this tag. Add AlphaVideoSource beside AlphaVideoVPX / AlphaVideoAV1 for range-source. The codec xcframeworks are still the v1.0.0 release's.
  • Maven Central: dev.mediacache:alpha-video, alpha-video-vpx, alpha-video-av1 1.0.1.
  • npm: alpha-video-web@1.0.1, react-native-alpha-video@1.0.1.
  • CocoaPods: alpha-video stays at 1.0.0.

alpha-video 1.0.0

Choose a tag to compare

@danielkpl2 danielkpl2 released this 23 Oct 14:57

Transparent video for Android, iOS, React Native and every modern browser: VP8 or VP9 WebM with alpha, and dual-stream VP8, VP9 or AV1.

Install

Platform Package
React Native npm install react-native-alpha-video react-native-range-source
Android dev.mediacache:alpha-video:1.0.0, with alpha-video-vpx (VP8, VP9) and alpha-video-av1 (AV1)
iOS, Swift Package Manager https://github.com/MediaCacheDev/alpha-video, products AlphaVideoVPX and AlphaVideoAV1
iOS, CocoaPods pod 'alpha-video', '1.0.0'
Web npm install alpha-video-web

See the README for usage, and the demo app to compare it with other transparent formats.

Changes

First public release.

  • Plays single-stream VP9 WebM with alpha (YUVA) on Android, iOS and the web. On Android and iOS a C++ core demuxes with libwebm's parser and decodes with libvpx 1.14.0 (VP9, with row multithreading) and dav1d 1.5.1 (AV1), without FFmpeg; the web decodes in WebAssembly with FFmpeg 6.1's VP9 decoder and dav1d. iOS renders with Metal, Android with OpenGL ES on a TextureView.
  • On Android, two MediaCodec decoders decode colour and alpha in hardware, told the clip's colour range and matrix by the same rules as the C++ core. They take dual-stream WebM (colour and alpha in separate VP9 or AV1 tracks), and standard alpha WebM too: the demuxer hands each block's BlockAdditional, a VP9 frame of the alpha, to the second decoder. The view picks the path from the file, and sends it to MediaCodec's software decoders once the hardware decoder hasn't 2 instances free, then to the C++ core. Where the GPU has GL_EXT_YUV_target, the shader reads the decoders' raw YUV and converts it with the clip's matrix itself, since drivers can ignore the colour tags of a software decoder's frames (BT.709 AV1 drew as BT.601 on a Snapdragon 778G with c2.android.av1-dav1d). Without the extension the driver converts. A VP9 track in a profile no MediaCodec decoder lists (profile 1, which a gbrp alpha track encodes to) sends the file to the C++ core instead. Elsewhere the C++ core decodes dual-stream VP9, and dual-stream AV1 on dav1d (8- to 12-bit, drawn at 8 bits), in every playback mode, and expands a limited-range alpha track to the full 0-255. iOS plays dual-stream AV1 this way too. 10-bit VP9 reports onError on the C++ core. MediaCodec's software decoders are asked for 8-bit output, since their 10-bit P010 frames drew blank on Adreno.
  • VP8 everywhere VP9 plays, with its alpha in BlockAdditional (the YUVA WebM FFmpeg and libvpx write) or in a second VP8 track. The core reads each keyframe's colour space and clamping bits as FFmpeg's VP8 decoder does: BT.601, limited range unless the clamping bit says full. libvpx decodes it on Android and iOS (the vpx module, whose libraries already held the VP8 decoder), MediaCodec (video/x-vnd.on2.vp8) on Android, and on the web <video> in Chromium and Firefox, WebCodecs (vp8) and FFmpeg's VP8 decoder.
  • The Android decoder prop (setDecoder on AlphaVideoView) forces the engine: mediacodec (hardware per track where an instance is free, else software), mediacodec-hw, mediacodec-sw or software (the C++ core). A forced engine reports onError rather than fall back. The default, auto, takes MediaCodec's hardware decoders, else its software ones, else the C++ core. onLoad reports the engine and decoders used, e.g. mediacodec-hw (c2.qti.vp9.decoder) or software (dav1d). iOS and the web accept the prop.
  • Colour is converted with the clip's range and matrix (limited or full; BT.601, BT.709 or BT.2020), matching FFmpeg's own conversion. An untagged clip is taken as limited range, BT.601 up to 720x576 and BT.709 above. Odd frame sizes keep their last chroma column and row.
  • Playback modes for the software path: stream (the default: keeps streamPrefetchAhead frames decoded after the shown one, 1 or more, and drops each frame once played; with verbose logging on, each pass logs the fewest frames it had ahead and how often it ran out), preload (decode every frame once) and streamAndPreload. A global decode queue (useDecodeQueue) keeps many players from saturating the CPU.
  • Sources: https://, file://, absolute paths, asset:// (Android assets, the iOS bundle, the web root), res/raw names and content:// on Android. On Android and iOS, http(s) files are read by range requests as they play, through the range-source add-on, with source.headers sent with every request: the stream modes start with the first frames, a seek fetches the Cues and the cluster it lands in, and a server without range support works too. Between plays the bytes are kept on disk in the app's cache folder (storage: 'auto', 'disk', 'memory' or 'none'; maxStorageBytes, default 512 MB), partially, so a clip played through plays again offline; cacheKey names the stored copy.
  • Playback control: play, pause, stop, seekToFrame and seekToTime, plus paused, autoPlay, loop and playbackSpeed. With autoPlay off the first frame shows, in every mode. Events: onLoad (frames, size and decoder), onError and onEnd.
  • The software decoders are codec modules, so an app ships only those it uses: vpx (libvpx, VP8 and VP9) and av1 (dav1d), each behind a versioned C plugin interface (cpp/AlphaVideoDecoderPlugin.h) and registering itself. Android: dev.mediacache:alpha-video plays through MediaCodec alone, and alpha-video-vpx and alpha-video-av1 add the software decoders, each its own AAR and native library. iOS: Swift Package Manager products AlphaVideoVPX and AlphaVideoAV1 (no core-only product, as iOS has no platform decoder for these files), pod subspecs alpha-video/VPX and alpha-video/AV1 (pod 'alpha-video' has both). React Native: one package, both codecs unless the app's package.json lists them ("react-native-alpha-video": { "codecs": ["vpx"] }), read by Gradle and pod install. A file whose codec has no module reports onError with the code NO_DECODER and a message naming the module; AlphaVideoCodecs.softwareDecoders lists what's registered. The core parses AV1 sequence headers itself, so the web demuxer has no dav1d either. On the web each codec module is its own lazily loaded module: FFmpeg's VP8 and VP9 decoders (1.2 MB) or dav1d (480 KB), loaded only for streams they decode.
  • Android decoder status: getDecoderStatus (with softwareDecoders), getDecoderInfo, getMaxDecoderInstances and getActiveDecoderCount.
  • React Native: AlphaVideoView on both the new architecture (a Fabric view, and a TurboModule for the decoder status) and the old one. On react-native-web it plays through alpha-video-web, a dependency: <video> where the browser keeps the alpha, otherwise WebCodecs or WebAssembly decoding drawn in WebGL2, so the alpha shows in Safari and for dual-stream files too. The webDecoder prop forces an engine, and web errors carry a code. source takes headers, and on the web a Blob or byte source too.
  • alpha-video-web for plain JS/TS and React, from web/. Its canvas engines run the C++ core's demuxer (libwebm's parser) in WebAssembly, reading the file by HTTP range requests as it plays through range-source-web (an npm dependency of alpha-video-web): playback starts with the first frames, a seek fetches the Cues and the cluster it lands in, and a server without range support works too. Between plays a URL's bytes are kept in OPFS, else IndexedDB (storage, default 'auto'; maxStorageBytes, default 512 MB), so a clip played through plays again with no requests. <video> does the same through range-source's Service Worker, which an app registers before rendering (registerRangeSourceWorker, re-exported): it reads by range from the same store, sends headers, and plays Blobs, bytes and byte sources. Without the worker, <video> takes a URL without headers, or a Blob as a blob: URL of itself; other sources play on a canvas, and decoder: 'browser' with one of them reports NEEDS_SERVICE_WORKER. src takes a URL (with headers), a Blob, bytes or a byte source of the app's own; cacheKey names the stored copy. The canvas engines share one WebGL2 context, so a page can show more of them than the browser's limit of about 16 contexts. The demuxer module is 183 KB (63 KB gzipped).
  • range-source is an add-on the player finds at runtime, and includes by default: a dependency of dev.mediacache:alpha-video (dev.mediacache:range-source), the pod's Source subspec, and the Swift package's default trait Source. React Native apps install react-native-range-source beside the player. An app can leave it out (Gradle exclude, naming the pod's subspecs, traits: [] from a package, or not installing it); the player then downloads a URL whole into a temporary file for each play, and logs once that range-source isn't installed. An Xcode app project can't turn a package's traits off, so SwiftPM apps added in Xcode always have it.
  • Data sources, for apps that fetch, cache or decrypt the bytes themselves: range-source's RangeDataSource on Android and iOS, registered by name in RangeDataSources for source.dataSource, one for this player and alpha-video-gpu. dev.mediacache:range-source-media3 adapts a Media3 DataSource.Factory (OkHttp, Cronet, CacheDataSource). Without range-source, a data source reports onError with the code NO_SOURCE_MODULE.
  • Standalone Android and iOS libraries.
  • MIT. The native core bundles libwebm's parser (BSD-3-Clause), the vpx module libvpx (BSD-3-Clause) and the av1 module dav1d (BSD-2-Clause); alpha-video-web bundles libwebm's parser, FFmpeg's VP8 and VP9 decoders (LGPL-2.1-or-later) in its vpx Workers only, and dav1d in its AV1 Wor...
Read more