The Playback Rewrite

[nielsvanvelzen]

🚀 What is The Playback Rewrite?

The goal of the rewrite is to create new playback code from scratch for video playback, audio playback and live tv. The new code should be easier to maintain and be prepared for a migration to Jellyfin 11. New features like SyncPlay can be added without much trouble. All code is Kotlin-first and uses the new SDK. It should be used by all our official Android apps for consistent behavior between (Android-) devices when playing files.

The rewrite is a very long project because I'm just one person volunteering to work on Jellyfin in my spare time. There is no hard deadline or specific version goals. This issue contains a task list of the bigger features/changes.

Status updates will be posted occasionally in the comments. Please read the bottom paragraph of this issue before commenting yourself.

🏆 Tasks

This list will change over time, some features might not be available on the initial release of the new playback code. When possible issues/pull requests will be linked for tracking purposes. There is no particular order for this list, new features might be added before restoring older features.

The amount of checked boxes does not indicate progress or completeness of the rewrite. Some changes required to make this happen have been running in production since version 0.15.0.

🏗️ Preparations

Various tasks that need to be completed to work on the rewrite.

• Create new Kotlin-first SDK
• Add new SDK
• Support multiple Gradle modules when building - Add (empty) playback module and CODEOWNERS #1072
• Add a way to enable the new playback module - Move useExternalPlayer and getPlaybackActivityClass functions to new PlaybackLauncher interface #1064

🎆 Proof of Concept

• Design architecture - Add playback rewrite backend and plugin systems #2401
• Add playlist support - everything is a playlist, even single item playback - Add playback rewrite backend and plugin systems #2401
• Add bare-bones player UI
• Add ExoPlayer backend - Add ExoPlayer backend to playback rewrite #2427
• Load video from server
• Report playback progress to server - Add PlaySessionService to playback rewrite #2428

🔄 Restoring features

• Support transcoding/remuxing
• Add remote control API - Add PlaySessionService to playback rewrite #2428
  • Support multiple items in PlayMessage POST /Sessions/{sessionId}/Playing only starts playing the first itemId parameter provided #2394
• Add intros
• Add chapter support
• Move "next up" screen to new playback code

🛠️ Integration (ATV)

• Use new playback module for audio (music, podcasts, audiobooks etc.) - Split MediaManager into interface and implementation #2453, Add option to enable playback rewrite for audio #2455
• Use new playback module for video (movies, series etc.)
• Use new playback module for Live TV
• Delete old playback code

🛠️ Integration (mobile)

• Make player UI touch compatible
• Release playback module to Maven Central (same versioning as the app)
• Replace native player with playback module
• Restore missing features

💥 Adding new features

• Add smart codec detection for directplay, remux and transcoding
• Add system preferences to overwrite codec support manually
• Add playback speed controls
• Add Media Session integration - Add MediaSessionPlugin to playback rewrite #2457
  • Use ImageProvider to proxy images
• Add Picture in Picture support
• Add gappless playback - Add ExoPlayer backend to playback rewrite #2427
• Add SyncPlay
• Add player stats (similar to YouTube's "stats for nerds") for debugging

Server blockers

Some features cannot be made or won't work great without changes to the server. This is a list of them.

• Add "speed" property to playback state API's jellyfin#8971
  Required to properly implement speed controls, it would also need changes to the dashboard of the web client
• Syncplay GroupUpdate class not properly added in OpenAPI spec jellyfin#6052
  Required to implement SyncPlay

❗ This issue

Please do not reply to this issue for any questions about the playback rewrite. The comment sections is meant for our contributors to give updates or discuss the implementation. For anything else, please create a new issue or discussion. We may delete your comment if we consider it off topic.