subtitles: FFmpeg-based Subtitle Authoring and Rendering in R

subtitles provides tools to generate standard subtitle files (SRT, ASS) from structured transcription outputs and to render those subtitles into videos using the FFmpeg command-line interface.

This package does not perform speech-to-text itself. Instead, it works with structured transcription results — for example, those produced by audio.whisper — to produce subtitle artifacts and to orchestrate FFmpeg for subtitle rendering.

The core design principles are:

• CLI-first: FFmpeg is the authoritative engine for subtitle rendering.
• Transparent: The package surfaces process invocation and errors directly.
• Artifact-focused: Subtitle files (.srt, .ass) are first-class citizens.

---

Quick Start

Prerequisites

You must have FFmpeg installed and available on your system PATH. A standard FFmpeg build with subtitle filter support (libass) is required.

Verify installation:

ffmpeg -version

---

Installation

This package is available on GitHub:

# install remotes if you don’t have it
install.packages("remotes")

# install subtitles from GitHub
remotes::install_github("cornball-ai/subtitles")

---

Basic Workflow

1. Generate structured transcription output

Use your preferred speech-to-text tool. For example, with audio.whisper:

srt_text <- predict(
  model,
  newdata = "audio.wav",
  language = "en",
  token_timestamps = TRUE
)

This returns a structured transcription object with segments and token timing.

---

2. Write an SRT subtitle file

whisper_to_srt(srt_text, file = "captions.srt")

This produces a valid SubRip (.srt) file representing the timed segments.

---

3. Optionally write an ASS subtitle file

ASS format supports richer styling and optional karaoke-style word highlighting:

whisper_to_ass(srt_text, file = "captions.ass", karaoke = TRUE)

---

4. Burn subtitles into a video

burn_subtitles(
  video     = "input.mp4",
  subtitles = "captions.ass",
  output    = "output_with_subtitles.mp4"
)

You can also pass style overrides:

burn_subtitles(
  "input.mp4",
  "captions.srt",
  "output.mp4",
  style = "FontSize=36,Outline=2,Shadow=1"
)

---

Function Reference

whisper_to_srt(x, file)

Converts a structured transcription object into a SubRip .srt file.

• x: Object returned from a transcription tool with segments.
• file: Path where the SRT file will be saved.

---

whisper_to_ass(x, file, karaoke = TRUE)

Converts a structured transcription object into an Advanced SubStation Alpha (.ass) file.

• karaoke: If TRUE, include token-level timing for word highlighting.

---

burn_subtitles(video, subtitles, output, style = NULL)

Invokes the FFmpeg CLI to render subtitles into a video.

• video: Path to the source video.
• subtitles: Path to .srt or .ass subtitle file.
• output: Path where the subtitled video will be saved.
• style: Optional ASS style overrides (passed to FFmpeg’s filter).

---

Call-record sidecars

Every file these functions write gets a JSON record at <output>.json: the resolved call (defaults filled in) plus a media block describing the delivered file (event count and time span for subtitles; fps, dimensions, and duration for video). Downstream tooling can lift these records into edit metadata. Disable with options(subtitles.sidecar = FALSE).

---

Why This Package Exists

Existing R packages for subtitle manipulation either focus on parsing (SRTtools) or linguistics. There is no dedicated, FFmpeg-first R tool for generating subtitle artifacts from structured transcriptions and invoking FFmpeg to render them.

subtitles fills that gap by providing:

• SRT / ASS subtitle generation
• Optional karaoke timing
• Transparent, reproducible video rendering

---

Philosophy

This package embraces the fact that the FFmpeg command line is the authoritative interface for video processing. Rather than hiding or abstracting it, subtitles builds on it:

• Arguments are surfaced directly
• Errors from FFmpeg are passed through
• Users can drop down to raw CLI if needed

This aligns with how video professionals work and ensures your R code stays in sync with FFmpeg documentation and community tooling.

---

System Requirements

• FFmpeg with subtitle filter support (libass)
• processx (for process execution)
• Compatible with Windows, macOS, Linux

---

Contributing

Contributions are welcome. Suggested areas:

• Additional styling presets
• Automatic line breaking
• Support for other transcription outputs (non-Whisper)
• CRAN packaging and documentation improvements

---

License

MIT

---

Acknowledgments

This package is not a speech-to-text provider. It is designed to integrate with tools such as:

• audio.whisper
• Other R wrappers around ASR systems