feat: render HLS/HEIC inline + compress uploads + transcode GIFs#465
Merged
feat: render HLS/HEIC inline + compress uploads + transcode GIFs#465
Conversation
Extends the content parser's extension and MIME sets so HLS streams and Apple HEIF images route to the existing video/image renderers instead of falling through to plain links. HLS plays via the media3-exoplayer-hls module that's already on the classpath; HEIC decodes via Coil3 -> ImageDecoder on API 28+.
Every upload path (profile picture, compose attachment, DM attachment) now runs through a shared preprocessing pipeline: - image/gif -> transcoded to H.264 MP4 via MediaCodec + MediaMuxer through an EGL input surface (typical 5-20x size reduction) - other image/* -> decoded with ImageDecoder (handles HEIC on API 28+), EXIF-rotated when needed, proportionally scaled, JPEG re-encoded - videos and other types -> passed through unchanged Profile pictures cap at 400px wide / JPEG quality 75. Compose and DM uploads cap at 1920px / quality 82. imeta dim and Kind 15 size/dim tags are recomputed from post-pipeline bytes so clients see accurate metadata. Gallery-mode video/image mixing check treats GIFs as video since they become MP4s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.m3u8(HLS) and.heic/.heifURLs as inline video/image inside notes — the HLS module and Coil3ImageDecodersupport were already on the classpath; only the extension/MIME sets in the content parser needed updating.ImageDecoder(handles HEIC natively on API 28+), EXIF-rotated when needed, proportionally scaled, and JPEG re-encoded. Profile pictures cap at 400px / quality 75; compose and DM content cap at 1920px / quality 82.imetadimand Kind-15size/dimtags are recomputed from post-pipeline bytes so clients see metadata that matches what was actually uploaded. Gallery-mode mixing check now treats GIFs as video since they become MP4s.Test plan
https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8in the feed — renders as an inline video player and plays..heicURL — renders as an inline image on API 28+.image/jpeg.imetadimreflects compressed dimensions..heicin compose — uploads successfully as.jpg, renders in Damus/Amethyst.Kind 15dim/sizereflect post-compression values..mp4, dramatically smaller than the GIF, renders as an inline video that auto-plays muted../gradlew assembleDebugpasses.