Skip to content

feat: rework audio and subtitle track selection - #202

Merged
MakD merged 10 commits into
masterfrom
sub-fix
Aug 4, 2026
Merged

feat: rework audio and subtitle track selection#202
MakD merged 10 commits into
masterfrom
sub-fix

Conversation

@MakD

@MakD MakD commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Problem

Jellyfin numbers every stream in a media source. The player only numbers the tracks it can actually see. We were treating those as the same thing, which falls apart the moment an item has external streams, because then the two lists aren't the same length.

That's where "No track selected", the wrong audio track and the wrong subtitle were all coming from. Progress reports went through the same mapping, so we were also writing nonsense back to the server and clobbering its remembered audio/subtitle indices with values that never matched anything that actually played.

Changes

TrackSelection is a new pure resolver, shared by ExoPlayer and MPV. Handles all five Jellyfin subtitle modes, normalises ISO 639-2 B/T codes (so ger, deu and de all land on the same track), and knows the difference between forced, full and SDH subtitles.

TrackMapping deals with the index side. Embedded ordinals now skip external streams, audio groups get sorted by format id instead of trusting whatever order the player hands back, and side-loaded subtitles are matched by identity rather than position (Format.id on ExoPlayer,
external-filename on MPV). uiState carries Jellyfin stream indices everywhere now, so progress reports write the right thing.

Settings: subtitle mode picker, "Follow server setting" for the preferred languages (and that's the new default), a prefer-SDH toggle, and the server's PlayDefaultAudioTrack is actually honoured now.

There's some unrelated work on this branch as well: Live TV codec negotiation via a device profile plus seek fixes, parallel person detail loading, and a series loading optimisation.

Behaviour

External (sidecar) audio isn't selectable any more. Jellyfin can't deliver it without a transcode, so it's excluded rather than sitting in the picker doing nothing when you tap it.

Resuming restores whatever tracks you were using. Starting something fresh applies your preferences instead.

MakD added 10 commits August 3, 2026 14:23
…rides

This commit introduces a robust track selection system that supports complex subtitle playback modes and improved language matching. It allows users to override default server behavior with specific modes like "Smart" (foreign audio only) or "Always Play," and refactors the internal track selection logic to use standardized language normalization and aliases.

### Key Changes:

*   **Advanced Track Selection**:
    *   Implemented a new `TrackSelection` utility to handle language normalization (ISO-639 bibliographic vs. terminologic codes) and alias matching.
    *   Created a comprehensive selection engine supporting multiple `SubtitlePlaybackMode` behaviors: `DEFAULT`, `ALWAYS`, `SMART`, `ONLY_FORCED`, and `NONE`.
    *   Updated `PlayerViewModel` to use this centralized logic for both initial track selection and automatic subtitle updates when audio tracks change.

*   **Subtitle Mode Configuration**:
    *   Added a "Subtitle Mode" setting in Player Options, allowing users to choose their preferred playback behavior or follow the server-defined configuration.
    *   Implemented `SubtitleModeSelectorItem` in the UI using a `DropdownMenu` with detailed descriptive hints for each playback mode.
    *   Updated `SessionManager` to fetch and store the server's `UserConfiguration` to support the "Follow server" setting.

*   **Playback Engine Improvements**:
    *   Refactored `PlayerViewModel` to use `trackSelectionParameters` and `TrackSelectionOverride` for more reliable track switching in Media3.
    *   Enhanced MPV player initialization by passing comprehensive language alias lists for `alang` and `slang` parameters.
    *   Improved the handling of "pending" track selections to ensure they are applied correctly once track groups are fully loaded.

*   **Data & Persistence**:
    *   Added `SUBTITLE_MODE_OVERRIDE` to `PreferencesRepository` with associated Flow and suspend functions for state management.
    *   Updated `SettingsViewModel` and `PlayerUiState` to reactively handle subtitle mode changes.
    *   Introduced localized strings and tooltips for all new subtitle playback preferences.
…ngs UI

This commit introduces a robust fallback mechanism for live TV playback, ensuring that if a direct stream fails, the app automatically attempts to retry using transcoding. It also refactors the live stream initialization logic to better align with server capabilities and replaces the subtitle mode selection UI with a more accessible dialog-based picker.

### Key Changes:

*   **Live TV Playback Robustness**:
    *   Implemented a retry mechanism in `PlayerWrapperViewModel` that catches live stream failures and attempts to reload the channel without forcing "Direct Play."
    *   Added `liveStreamFailedEvent` to `PlayerViewModel` to signal when a `DIRECT_PLAY` attempt results in a playback error.
    *   Updated `PlayerScreenWrapper` to reactively trigger a stream retry when a live playback failure is detected.

*   **Repository & Stream Handling**:
    *   **Codec Detection**: Enhanced `LiveTvRepository` to dynamically build device profiles for MPEG-TS streams by filtering native video and audio codecs (H.264, HEVC, AAC, AC3, etc.).
    *   **API Refactoring**: Updated `getChannelPlaybackInfo` to use the structured `OpenLiveStreamDto`, enabling better support for device profiles and protocol negotiation during live stream initialization.
    *   **Streaming Methods**: Explicitly enabled transcoding support for live TV and added handling for the `TRANSCODE` play method, including proper URL resolution via `ApiClient`.
    *   Removed legacy private host detection and manual absolute URL conversion in favor of standardized SDK URL creation.

*   **UI Enhancements**:
    *   **Subtitle Mode Picker**: Replaced the constrained `DropdownMenu` in `PlayerOptionsScreen` with a comprehensive `SubtitleModePickerDialog`.
    *   Improved the settings UI by providing a scrollable dialog with detailed descriptions and checkmark indicators for subtitle playback behaviors (e.g., Smart, Always Play, Only Forced).

*   **Error Handling & Logging**:
    *   Integrated `onPlayerError` in `PlayerViewModel` to specifically monitor and log failures during live playback transitions.
    *   Added detailed Timber logging across the playback wrapper to track stream selection methods and retry attempts.
This commit improves the robustness of the player's seeking logic, specifically handling edge cases for live content and media with undefined durations. It prevents invalid seek operations and disables certain playback gestures when they are not applicable to the current stream type.

### Key Changes:

*   **Seeking Safety & Validation**:
    *   **Relative Seeks**: Updated `SeekRelative` logic to handle cases where the player duration is unknown (e.g., live streams). The target position is now safely coerced to a minimum of 0 even when `duration` is unavailable.
    *   **Seek Bar Interactions**: Modified `OnSeekBarDragFinished` to prevent calling `player.seekTo` if the content is not seekable (duration ≤ 0).
    *   **State Updates**: Ensured the `currentPosition` in the UI state is never set below 0 during seek operations.

*   **Gesture Management**:
    *   Disabled the **Double-Tap to Seek** gesture on live channels to prevent inconsistent playback states in real-time streams.

*   **Error Prevention**:
    *   Added checks for `player.duration` before applying `coerceIn` constraints to avoid logic errors when the media hasn't fully loaded or is a continuous broadcast.
… improve sideloaded subtitle handling

This commit transitions the player's track selection logic from using list-based positional indices to server-provided stream indices. This change ensures more reliable track synchronization between the UI and the playback engine, particularly when handling mixed internal and external (sideloaded) media streams. It also introduces a robust retry mechanism for track resolution during media transitions.

### Key Changes:

*   **Stream-Based Track Management**:
    *   Replaced `pendingAudioTrackPosition` and `pendingSubtitleTrackPosition` with `pendingAudioStreamIndex` and `pendingSubtitleStreamIndex` in `PlayerViewModel`.
    *   Implemented mapping utilities (`audioGroupForStreamIndex`, `streamIndexForAudioGroup`, etc.) to translate between Media3 `Tracks.Group` objects and server-defined stream indices.
    *   Introduced `MAX_PENDING_TRACK_ATTEMPTS` to gracefully give up on track resolution if the requested streams do not become available after a set number of track-change events.

*   **Sideloaded Subtitle Enhancements**:
    *   Standardized the identification of external subtitles using `EXTERNAL_SUBTITLE_ID_BASE` and `customData` URI matching.
    *   Updated `PlayerControls` to correctly identify and label external tracks by resolving them against `sideLoadedSubtitleUris`.
    *   Improved subtitle URI construction to support negotiated delivery URLs from the server when available.

*   **Track Selection Logic**:
    *   **Audio Selection**: Added `audioRank` to prioritize internal default tracks over external ones unless specifically requested.
    *   **Subtitle Selection**: Refined fallback logic for `ALWAYS` and `ONLY_FORCED` modes to better handle cases where a preferred language is not available or when a forced track is marked as default.
    *   **External Audio**: Added support for an `allowExternalAudio` flag in the selection engine.

*   **UI & Controls**:
    *   Refactored `PlayerControls` to use the new stream-index mapping, ensuring the checkmarks and labels in the track selection panels accurately reflect the active media streams.
    *   Improved audio track sorting in the UI to maintain a consistent order based on their server-defined identifiers.

*   **Playback Engine (MPV)**:
    *   Updated the MPV track metadata parser to populate Media3 `Format.customData` with the external filename, enabling better tracking of sideloaded subtitle files.
…loading

This commit introduces support for synchronizing audio and subtitle language preferences with the server-defined user configuration. It also optimizes playlist initialization for TV series by implementing bulk episode fetching and refactors track mapping logic to use centralized utilities.

### Key Changes:

*   **Server-Synchronized Preferences**:
    *   Added a "Follow server" option to audio and subtitle language settings.
    *   Implemented `resolvePreferredAudioLanguage` and `resolvePreferredSubtitleLanguage` in `PlayerViewModel` to dynamically fetch preferences from the server's `UserConfiguration` when the "server" setting is selected.
    *   Updated both ExoPlayer and MPV initialization flows to respect these resolved preferences.

*   **Series Playback Optimizations**:
    *   **Repository**: Introduced `getSeriesEpisodes` in `MediaRepository` and `JellyfinMediaRepository` to fetch all episodes for a series in a single request.
    *   **`PlaylistManager`**: Refactored series queue loading to use the new bulk fetching method, eliminating the need to iterate and perform separate API calls for every season.
    *   **Filtering**: Enhanced playlist logic to automatically filter out "missing" episodes (virtual entries for episodes not present on the server).

*   **Track Mapping & Management**:
    *   Refactored track selection logic to use a new `TrackMapping` utility for handling side-loaded subtitle IDs and generating audio track sort keys.
    *   Replaced hardcoded `EXTERNAL_SUBTITLE_ID_BASE` offsets with standardized methods in `TrackMapping` across the player and UI components.
    *   Improved the reliability of mapping between Media3 `Tracks.Group` objects and internal `AfinityMediaStream` indices.

*   **UI & Data Refinements**:
    *   **Settings**: Updated `arrays.xml` to include "Follow server" in the language selection lists.
    *   **Offline Playback**: Updated `ItemDetailViewModel` to ignore Season 0 (Specials) when calculating the next episode to play in offline mode.
    *   **Stream Resolution**: Simplified the logic for determining the first unplayed episode in a season within the repository.
This commit updates the default behavior for preferred audio and subtitle languages in the application's settings. Instead of defaulting to an empty string, the repository now uses a standardized constant to indicate that the player should follow the server-defined language configuration.

### Key Changes:

*   **Preferences & Defaults**:
    *   Updated `PreferencesRepositoryImpl` to use `TrackSelection.FOLLOW_SERVER_LANGUAGE` as the fallback value for both audio and subtitle language preferences.
    *   Refactored `getPreferredAudioLanguage` and `getPreferredSubtitleLanguage`, along with their respective `Flow` variants, to ensure consistent default state management when no user preference is explicitly set.
This commit introduces a new setting that allows users to prefer SDH (Subtitles for the Deaf and Hard-of-Hearing) during automatic track selection. The track selection engine has been updated to prioritize hearing-impaired streams when the preference is enabled, and the setting is integrated into the player options UI and data persistence layer.

### Key Changes:

*   **Track Selection Logic**:
    *   Updated `TrackSelection.kt` to incorporate the `preferHearingImpaired` flag into the subtitle selection algorithm.
    *   Refactored `firstFull` to sort subtitle candidates, prioritizing those that match the user's hearing-impaired preference and fallback to default tracks.
    *   Ensured `isHearingImpaired` metadata is correctly propagated through the `AfinityMediaStream` model.

*   **Settings & UI**:
    *   **Player Options**: Added a `SettingsSwitchItem` to the `PlayerOptionsScreen` allowing users to toggle the SDH preference.
    *   **SettingsViewModel**: Integrated the new preference into the UI state and added `togglePreferSdhSubtitles` to handle user interactions.
    *   **Localization**: Added `pref_prefer_sdh_title` and `pref_prefer_sdh_summary` strings.

*   **Data & Persistence**:
    *   Added `PREFER_SDH_SUBTITLES` to `PreferencesRepository` using DataStore for persistence.
    *   Implemented Flow-based observation and suspend functions for retrieving and updating the SDH preference.

*   **Playback Integration**:
    *   Updated `PlayerViewModel` to fetch the SDH preference from the repository and pass it to the track selection utility during playback initialization and track updates.
This commit fixes the player's track selection logic to better handle resumed playback sessions. It ensures that when a user continues a video, the player automatically falls back to the server-saved audio and subtitle indices if no explicit user selection is provided, maintaining continuity across different devices.

### Key Changes:

*   **Player State Handling**:
    *   **`PlayerViewModel.kt`**: Introduced an `isResuming` check based on the initial playback position.
    *   Updated the `TrackSelection.select` call to prioritize `serverSavedAudioIndex` and `serverSavedSubtitleIndex` specifically when `isResuming` is true and no manual stream overrides are active.

*   **Track Selection Logic**:
    *   **`TrackSelection.kt`**: Refactored `resolveSubtitleIndex` to immediately return the mapped internal index when a valid `requestedStreamIndex` is matched against available subtitle streams.
    *   Improved the readability of the stream index resolution flow to ensure requested indices are handled before falling through to language-based matching logic.
This commit updates the hint text for various subtitle playback modes in the player settings to be more descriptive and user-friendly. The revised strings provide clearer explanations for "Always," "Smart," and "Only Forced" behaviors, helping users better understand how language preferences and forced flags affect automatic subtitle selection.

### Key Changes:

*   **Subtitle Mode Hints**:
    *   Updated `subtitle_mode_always_hint` to emphasize that subtitles will be loaded for every item regardless of audio.
    *   Refined `subtitle_mode_smart_hint` to clarify the distinction between full subtitles for foreign audio and forced subtitles for preferred languages.
    *   Expanded `subtitle_mode_only_forced_hint` to provide context on what "forced" typically entails, such as on-screen text and foreign-language dialogue.
…ities

This commit introduces a progressive loading strategy for the person detail screen to improve perceived performance and responsiveness. It also adds a centralized track mapping utility to handle the conversion between internal stream indices and external side-loaded subtitle IDs.

### Key Changes:

*   **Person Detail Optimization**:
    *   **Progressive Loading**: Refactored `PersonViewModel` to fetch data concurrently using `async`. The UI now updates immediately with stored metadata while a full refresh and item fetch (movies/shows) continue in the background.
    *   **Repository Enhancements**: Added `getPersonWithoutRefresh` to `MediaRepository` and `JellyfinMediaRepository` to allow retrieving cached person data without triggering heavy network synchronization.
    *   **Metadata Validation**: Implemented `hasIncompleteMetadata` and `isRicherThan` checks to intelligently update the UI only when more comprehensive metadata (e.g., longer biographies or higher-quality images) is retrieved.
    *   **UI Resilience**: Updated `PersonScreen` to prioritize displaying existing content over full-screen loading states and ensured that network errors no longer hide already-loaded data.

*   **Player Track Management**:
    *   **Track Mapping Utility**: Introduced `TrackMapping.kt` to standardize the handling of embedded vs. external media streams.
    *   **Side-loaded Subtitles**: Implemented logic to map external subtitles using a consistent ID base (`128`), facilitating better tracking of side-loaded SRT/VTT files.
    *   **Sorting & Selection**: Added utilities for audio track sort keys and URI-based stream indexing to improve track selection reliability.

*   **Resources**:
    *   Added `ic_subtitles_settings.xml` vector drawable for use in player configuration menus.
@MakD
MakD merged commit 126b75a into master Aug 4, 2026
@MakD
MakD deleted the sub-fix branch August 4, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant