add subtitle burn-in support by KyleTryon · Pull Request #36 · TechSquidTV/Cliparr

KyleTryon · 2026-04-21T03:16:09Z

Summary

add subtitle track discovery for Plex and Jellyfin playback sessions
add editor controls and preview rendering for styled subtitle burn-in
burn supported text subtitles into exports and surface unsupported subtitle states more clearly

Notes

Plex embedded/internal subtitle tracks are still detected but not supported for styled burn-in yet
HLS subtitle support is deferred until upstream MediaBunny support lands

Validation

pnpm --filter @cliparr/frontend lint
pnpm --filter @cliparr/server lint:types

Blocked By

add HLS editing support for Plex and Jellyfin #35

…rtDialog

…preview component

… fonts in the subtitle editor

…ule usage

Copilot

Pull request overview

Adds end-to-end subtitle burn-in support by exposing provider subtitle tracks (Plex/Jellyfin), adding subtitle selection + styling controls in the editor (including local font discovery), rendering styled subtitles in the live preview, and burning them into exported clips.

Changes:

Extend provider playback payloads with subtitle track discovery/selection metadata (Plex + Jellyfin) and shared subtitle normalization helpers.
Add a subtitle panel to the editor for track selection, styled preview, and local font options; download/parse subtitle text into cues for preview/export validation.
Implement subtitle rendering overlays for both editor preview frames and export transcoding (burn-in processor), and surface subtitle export status/validation in the export dialog.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
README.md	Updates feature list to mention subtitle overlay support.
apps/server/src/providers/types.ts	Adds subtitle track/selection types and new preview fields to the playback item contract.
apps/server/src/providers/shared/subtitles.ts	Introduces shared helpers for subtitle codec/format/flags normalization.
apps/server/src/providers/plex/playback.ts	Discovers Plex subtitle streams, exposes selectable tracks, and emits subtitle metadata in playback items.
apps/server/src/providers/jellyfin/playback.ts	Discovers Jellyfin subtitle streams, exposes selectable tracks, and emits subtitle metadata in playback items.
apps/frontend/src/providers/types.ts	Mirrors server playback contract changes for subtitles + preview fields.
apps/frontend/src/lib/subtitles/types.ts	Defines subtitle cue + style setting types used by preview and export.
apps/frontend/src/lib/subtitles/trimSubtitleCues.ts	Trims/subsets cues to the clip range for export burn-in.
apps/frontend/src/lib/subtitles/settings.ts	Persists subtitle style settings and loads local font options (where supported).
apps/frontend/src/lib/subtitles/renderSubtitleCue.ts	Renders a styled subtitle cue to a canvas context.
apps/frontend/src/lib/subtitles/parseSubtitleText.ts	Parses downloaded VTT/SRT-ish subtitle text into cues.
apps/frontend/src/lib/subtitles/getActiveSubtitleCue.ts	Finds the active cue at a timestamp (binary search).
apps/frontend/src/lib/selectPreferredSubtitleTrack.ts	Chooses the best subtitle track and surfaces unsupported-state messaging.
apps/frontend/src/lib/exportClip.ts	Adds a burn-in video processor to render subtitles into exported frames.
apps/frontend/src/components/EditorScreen.tsx	Wires subtitle track selection, cue loading, preview enablement, and export validation into the editor screen.
apps/frontend/src/components/editor/useSubtitleFontOptions.ts	Hook to load/merge bundled + local + saved font options.
apps/frontend/src/components/editor/useEditorPlayback.ts	Renders subtitles into the live preview canvas and keeps the frame updated on style/cue changes.
apps/frontend/src/components/editor/EditorSubtitleStylePreview.tsx	Adds an in-panel subtitle style preview component.
apps/frontend/src/components/editor/EditorSubtitlePanel.tsx	Adds the main subtitle burn-in control panel UI.
apps/frontend/src/components/editor/EditorExportDialog.tsx	Adds subtitle export summary + disables export when subtitle state blocks burn-in.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+export function deriveSubtitleTracks(
+  session: ProviderSessionRecord,
+  context: JellyfinSourceContext,
+  item: any,
+  mediaSourceId: string | undefined
+) {
+  const mediaSource = currentMediaSource(undefined, item, mediaSourceId);
+  if (!mediaSource) {
+    return [];
+  }
+
+  const itemId = stringValue(item?.Id);
+
+  return asArray(mediaSource?.MediaStreams)
+    .filter((stream) => isSubtitleMediaStream(stream))
+    .map((stream) => jellyfinSubtitleTrack(session, context, itemId, mediaSourceId, stream));
+}


+                {subtitleTracks.map((track) => (
+                  <SelectItem
+                    key={`${track.streamId ?? "stream"}:${track.index ?? "index"}`}
+                    value={track.streamId ? `stream:${track.streamId}` : `index:${track.index ?? "unknown"}`}
+                  >
+                    {subtitleTrackLabel(track)}
+                  </SelectItem>
+                ))}
+              </SelectGroup>
+            </SelectContent>
+          </Select>


 interface UseEditorPlaybackProps {
  hlsUrl?: string;
-  mediaUrl?: string;
+  mediaUrl: string;
  initialDuration: number;
  startTime: number;
  endTime: number;
  sessionId: string;
  selectedAudioTrack?: PlaybackAudioSelection;
+  subtitleCues?: readonly SubtitleCue[];
+  subtitlesEnabled?: boolean;
+  subtitleStyleSettings?: SubtitleStyleSettings;


+    path,
+    transcodeSessionId: playbackSessionId,
+    mediaIndex: String(resolvedSelection?.mediaIndex ?? 0),
+    partIndex: String(resolvedSelection?.partIndex ?? 0),
+    subtitles: "sidecar",
+    advancedSubtitles: "text",
+    autoAdjustSubtitle: "0",
+  });
+
+  return `/video/:/transcode/universal/subtitles?${params.toString()}`;
+}
+
+function plexSubtitleTrack(
+  session: ProviderSessionRecord,
+  context: PlexSourceContext,
+  item: any,
+  playbackSessionId: string,
+  selection: PlexMediaSelection | undefined,
+  stream: any
+): PlaybackSubtitleTrack {
+  const codec = normalizeSubtitleCodec(stream?.codec);
+  const directSubtitlePath = buildPlexSubtitlePath(stream);
+  const isText = isTextSubtitleCodec(codec);
+  const transcodeSubtitlePath = directSubtitlePath
+    ? undefined
+    : buildSelectedPlexSubtitleTranscodePath(item, playbackSessionId, selection, stream);
+  const contentFormat = directSubtitlePath
+    ? plexDirectSubtitleContentFormat(codec)
+    : transcodeSubtitlePath
+      ? "srt"
+      : subtitleContentFormat(codec);
+  const contentPath = directSubtitlePath ?? transcodeSubtitlePath;
+
+  return {
+    streamId: idValue(stream?.id),
+    index: numberValue(stream?.index) ?? numberValue(stream?.streamIdentifier),
+    languageCode: stringValue(stream?.languageCode) ?? stringValue(stream?.languageTag),
+    title: selectedSubtitleTrackTitle(stream),
+    codec,
+    contentFormat,
+    isText,
+    isDefault: booleanFlag(stream?.default),
+    isForced: booleanFlag(stream?.forced),
+    isHearingImpaired: booleanFlag(stream?.hearingImpaired),
+    isExternal: Boolean(stringValue(stream?.key)),


Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

+
+  return asArray(mediaSource?.MediaStreams)
+    .filter((stream) => isSubtitleMediaStream(stream))
+    .map((stream) => jellyfinSubtitleTrack(session, context, itemId, mediaSourceId, stream));


+  const codec = normalizeSubtitleCodec(selectedSubtitleStream?.codec);
+  return {
+    streamId: idValue(selectedSubtitleStream?.id),
+    index: numberValue(selectedSubtitleStream?.index) ?? numberValue(selectedSubtitleStream?.streamIdentifier),
+    languageCode: stringValue(selectedSubtitleStream?.languageCode)
+      ?? stringValue(selectedSubtitleStream?.languageTag),
+    title: selectedSubtitleTrackTitle(selectedSubtitleStream),
+    codec,
+    contentFormat: subtitleContentFormat(codec),
+    isText: isTextSubtitleCodec(codec),
+  };


KyleTryon changed the title ~~[codex] add subtitle burn-in support~~ add subtitle burn-in support Apr 21, 2026

KyleTryon added 7 commits May 23, 2026 02:59

feat(subtitles): expose provider subtitle tracks

48142ec

feat(subtitles): add burn-in editor controls

821ec50

feat: add subtitle export status summary and validation to EditorExpo…

5369f55

…rtDialog

feat: implement collapsible sidebar for subtitle panel and add style …

de6ec36

…preview component

feat: add support for querying and selecting locally installed system…

cbd9c59

… fonts in the subtitle editor

refactor: restrict scope of subtitle helper functions to internal mod…

7fce355

…ule usage

fix(subtitles): stabilize provider subtitle support

c43b5af

KyleTryon force-pushed the codex/subtitle-burn-in branch from c36e92b to 23431e4 Compare May 23, 2026 07:08

fix: resolve subtitle branch lint warnings

8150d0e

KyleTryon force-pushed the codex/subtitle-burn-in branch from 23431e4 to 8150d0e Compare May 23, 2026 07:09

KyleTryon added 8 commits May 23, 2026 23:39

Add Plex subtitle content support

1dc70ae

Use Sintel for dev media seeding

a48cc87

Merge remote-tracking branch 'origin/main' into codex/subtitle-burn-in

3a200f6

Update workspace dependencies

1c290ac

Disable Plex subtitles in HLS previews

b20bac3

Add Jellyfin subtitle parity coverage

99d81b6

Simplify subtitle settings panel

f9e7f05

Hide unsupported subtitle tracks

6b4e137

KyleTryon marked this pull request as ready for review May 24, 2026 05:15

KyleTryon requested a review from Copilot May 24, 2026 05:15

Copilot started reviewing on behalf of KyleTryon May 24, 2026 05:15 View session

Fix Docker dev Cliparr startup

ec89494

Copilot AI reviewed May 24, 2026

View reviewed changes

Address subtitle review feedback

c1f3aa7

KyleTryon requested a review from Copilot May 24, 2026 06:16

Copilot started reviewing on behalf of KyleTryon May 24, 2026 06:16 View session

Copilot AI reviewed May 24, 2026

View reviewed changes

KyleTryon added 2 commits May 24, 2026 02:24

Align subtitle preview timestamps

fd208a1

Skip subtitle transcode without overlapping cues

96e7edf

KyleTryon added 9 commits May 24, 2026 02:25

Move Docker dev tooling into image

23a1239

Address provider subtitle review feedback

2179ba7

Cache subtitle cue layout

43bfe28

Make subtitle export summary clip-aware

493db29

Expose only burn-in subtitle tracks

0c9c676

Share provider playback types

4102556

Add Docker dev stack commands

fec670a

Remove unused provider type export

08e27b4

Include shared package in Docker build

92a7a23

KyleTryon merged commit a23b914 into main May 24, 2026
2 checks passed

KyleTryon deleted the codex/subtitle-burn-in branch May 31, 2026 07:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add subtitle burn-in support#36

add subtitle burn-in support#36
KyleTryon merged 29 commits into
mainfrom
codex/subtitle-burn-in

KyleTryon commented Apr 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

KyleTryon commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Notes

Validation

Blocked By

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

KyleTryon commented Apr 21, 2026 •

edited

Loading