feat(browser): FFmpeg video API, subtitles, HLS variants, and PiP

[niklabh]

Implement Phase 1 video from the roadmap using ffmpeg-next for demux/decode and libswscale to RGBA for the canvas.

Host (oxide-browser):

• Add VideoPlayer with incremental decode, seek, single send_eof at true EOF, post-packet drain, and decoder_eof_sent to avoid repeated EOF errors and choppy playback.
• Register imports: video_load/detect_format/load_url, playback, seek, position, duration, video_render, volume/loop, PiP, subtitle_load_srt/vtt/clear, HLS variant count/url/open_variant, last URL content type.
• Extend HostState with video, video_pip_frame, video_pip_serial; wire video_render_at for canvas + optional caption text + PiP texture refresh.
• Add video_format (sniff + Accept), subtitle (SRT/WebVTT), and egui floating PiP window.
• Dependencies: ffmpeg-next, tempfile, url.

SDK:

• Expose VideoFormat and full video/subtitle wrappers for guests.

Example:

• examples/video-player: sample Big Buck Bunny MP4, loop, PiP, pause/resume.

Tooling:

• CI and release workflows: pkg-config and libav dev packages on Ubuntu.
• CONTRIBUTING: FFmpeg library prerequisites.
• ROADMAP: mark video subsection complete (embedded audio mixing noted as follow-up).

Made-with: Cursor

Summary by CodeRabbit

• New Features

  • Video playback: format detection, load from bytes/URL, play/pause/stop/seek, position/duration queries
  • HLS support with variant introspection/selection
  • Picture-in-picture preview, volume and loop controls
  • Subtitle support for SRT and VTT, rendered overlays
  • Example video-player demo included

• Documentation

  • Updated contributing guide with FFmpeg/pkg-config setup
  • Roadmap updated to mark video features complete

[coderabbitai]

Warning

Rate limit exceeded

@niklabh has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 13 minutes and 35 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 13 minutes and 35 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 42935717-649d-47f4-87c3-05764b45c0e4

📥 Commits

Reviewing files that changed from the base of the PR and between fc5dbc1 and cbaa2f9.

📒 Files selected for processing (5)

• .github/workflows/ci.yml
• .github/workflows/docs.yml
• .github/workflows/release.yml
• CONTRIBUTING.md
• oxide-browser/src/video_format.rs

📝 Walkthrough

Walkthrough

Added FFmpeg-backed video playback, HLS variant support, subtitle (SRT/VTT) parsing, picture-in-picture rendering, example video-player crate, workspace/dependency updates, SDK host APIs for video control/rendering, and CI/docs prerequisite additions for FFmpeg tooling.

Changes

Cohort / File(s)	Summary
CI/CD Workflows .github/workflows/ci.yml, .github/workflows/release.yml, .github/workflows/docs.yml	Install list extended to include pkg-config and FFmpeg dev packages (libavcodec-dev, libavformat-dev, libavutil-dev, libavfilter-dev, libavdevice-dev, libswscale-dev, libswresample-dev).
Documentation & Roadmap CONTRIBUTING.md, ROADMAP.md	Added FFmpeg install instructions and marked video-related roadmap items (load, playback, seek, render, volume, adaptive streaming, subtitles, PiP) as completed/updated.
Workspace & Example Cargo.toml, examples/video-player/Cargo.toml, examples/video-player/src/lib.rs	Added examples/video-player workspace member and example crate; example exposes start_app/on_frame, loads URL, controls playback, renders frames, overlays position/duration, and provides Pause/Resume UI.
Browser Crate Exports oxide-browser/src/lib.rs	Exported new modules: subtitle, video, and video_format.
Video Backend & Playback State oxide-browser/src/video.rs, oxide-browser/src/video_format.rs	New FFmpeg-backed VideoPlayer with decode-at-timestamp, PTS-to-ms handling, HLS master parsing, format sniffing, MIME mapping, and helpers for temp-file suffixes. Added VideoPlaybackState with playback clock, looping, volume, PiP, subtitles, HLS metadata, and open-bytes support.
Subtitle Parsing oxide-browser/src/subtitle.rs	Added SubtitleCue and parsers for SRT and WebVTT plus cue_text_at lookup.
Host Capabilities & Rendering oxide-browser/src/capabilities.rs	Extended HostState with video, video_pip_frame, video_pip_serial; added video_render_at helper and registered host imports for video, HLS introspection, rendering, volume/loop/PiP, and subtitle load/clear.
UI PiP Integration oxide-browser/src/ui.rs	Per-tab PiP texture caching and render_video_pip that loads updated frames by serial and displays a floating egui window.
SDK Public API oxide-sdk/src/lib.rs	Added VideoFormat enum and safe wrappers for video operations: format detect/load (bytes/URL), HLS variant queries/open, play/pause/stop/seek, position/duration, render, volume/loop/PiP, subtitle load/clear.

Sequence Diagram(s)

sequenceDiagram
    participant Guest as Guest App (video-player)
    participant SDK as oxide-sdk
    participant Host as oxide-browser (capabilities)
    participant State as VideoPlaybackState
    participant FFmpeg as FFmpeg Backend (video.rs)
    participant Canvas as Canvas State

    Guest->>SDK: video_load_url(url)
    SDK->>Host: api_video_load_url(url)
    Host->>State: open_url(url)
    State->>FFmpeg: VideoPlayer::open_url(url)
    FFmpeg-->>State: VideoPlayer (duration, streams)

    Guest->>SDK: video_play()
    SDK->>Host: api_video_play()
    Host->>State: play()

    Guest->>SDK: video_render(x,y,w,h)
    SDK->>Host: api_video_render(x,y,w,h)
    Host->>State: current_position_ms()
    Host->>FFmpeg: decode_frame_at(position_ms)
    FFmpeg-->>Host: RGBA frame
    Host->>Canvas: append DrawCommand::Image(frame)
    note over Canvas: Frame available for egui draw

    alt PiP enabled
        Host->>Host: update video_pip_frame, increment video_pip_serial
        Host->>UI: render_video_pip detects serial change
        UI->>UI: convert RGBA -> egui ColorImage
        UI->>UI: display floating window with texture
    end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 A hop, a frame, a tiny cheer! 🎥

FFmpeg fluffs each pixel bright,
Subtitles whisper through the night.
PiP peeks in a cozy square,
Buttons pause and resume with care.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 58.82% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly and concisely summarizes the main change: introducing FFmpeg-backed video playback with subtitles, HLS variant support, and picture-in-picture functionality.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/video-playback

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}