New Year, New Video Player + Other Goodies

[LePips]

With the new year, this means another one of my huge PRs encompassing months worth of work and not adhering to my own git guidelines.

iOS/iPadOS - New Video Player

The video player has been entirely re-designed with new visuals and customization options. Now mostly in SwiftUI, future features can be more easily implemented and managed which is great for future contributions.

The new video player is influenced by iOS 16's new native video player:

iOS - Movie Default

iOS - TV Show Default

iPadOS - Movie Default

iPadOS - TV Show Default

More customization is provided to enhance your playback experience.

Video Player Settings

Below are highlights for some of the new features.

Gestures

Actions performed during a gesture are now fully customizable and more can be easily added in the future. During most actions a toast will appear to provide information about the results of the gesture.

New gestures include:

• long press
• multi-tap in same area
• double touch
• horizontal pan
• vertical pan with different actions on left/right side
• pinch
• swipe

Note: horizontal scrub and horizontal swipe cannot be used together due to gesture detection conflicts

Buttons

There are two places where buttons can appear:

• the bar, which are buttons always shown on the overlay
• the menu, which can be accessed by tapping the menu icon.

Now, you are freely able to choose which buttons appear in each area and their ordering.

Slider Color

The slider color can be customized using the new default iOS color picker.

iOS - Slider Color Picker

Slider Type

With the introduction of a new slider type, Capsule, the more familiar Thumb type is also available. These two types contain different behaviors, where Capsule allows the whole slider to be dragged where Thumb only receives input on the thumb.

iOS - Slider Type

Subtitles

• Font: Subtitle font now only uses the font family instead of the font family + style. This was done for simplification.
• Size: Subtitle size is now a value in the range of 8-24 instead of a list of static values.
• Color: Subtitle colors have been implemented.

An additional note has been added to Settings to clarify that these settings may not take effect on all subtitle types.

• Closes colors of the subtitles  #646

Timestamp

The timestamp can even be configured:

• Scrub Current Time: Show the current playback time during scrubbing
• Timestamp Type: Split the time values across the bar or have them compact
• Trailing Value: Show either the time left or the total duration

Native Player

The native player was previously known as the "experimental player" and was, frankly, only implemented to appease a few people who requested it. Now it is being provided as an explicit option for clarity. It may be removed when the MPV rendering layer has been implemented.

Closed Issues

• Closes Too many drop downs #641, due to new button customizations
• Closes The subtitle selection box will automatically jump back after flipping down. #624, due to new menus
• Closes Increase icon touch areas #599, due to explicitly adding better touch areas for buttons
• Closes Subtitle delay #591, due to subtitle delay being a new option
• Closes Player is off center on iOS 16 #590, due to new overlay
• Closes iOS 16 - Persistent Home Indicator #582, due to new APIs
• Closes seach to play and time display problems #580, due to new overlay
• Closes iOS/iPadOS - Flashing Player Menu #559, due to new overlay
• Closes Lock iOS to only Portrait #528, due to refactor
• Closes ipadOS - Tapping to skip opens the overlay #455, due to new gestures
• Closes Next/Previous controls and seek slider broken #414, due to new overlay
• Closes tvOS - video sessions not reported as stopped #402, due to refactor

JellyfinAPI

With the new async/await generated JellyfinAPI, the entire networking stack has been rewritten and is now up to date with new features in Swift. This update fixes a few outstanding issues, like crashes due to force unwrapped optionals, and allows a few other architectural changes to take place, like session dependency injection.

• Closes Crash on tvos when accessing a show #636
• Closes tvOS - Movies library empty - but not on iphone #621

Logs

The entire logging layer has been replaced by Pulse, an amazing framework for network and general logging. Logs can now be viewed inside the app (iOS, iPadOS, and tvOS) for troubleshooting and exporting. All of the UI for logging is provided by PulseUI and fits comfortably in the design of Swiftfin. Lastly, sensitive information like IPs, access tokens, and passwords used in network calls can be redacted so that logs can be safely shared when reporting an Issue.

The issue template will be updated to provide instructions exporting and sharing logs.

iOS Logs

tvOS Logs

Other UX Improvements

Many UX improvements have also been incorporated. I would work on these to take my mind of off the refactor as they give me little wins to keep up motivation.

Below are some highlights.

App Accent Color

The accent color used throughout the app can be changed from the default purple used by Swiftfin. Some views that contain text use this accent color and will dynamically update the text color for the best contrast.

Note: some views may require an app restart to update as they are redrawn and receive the new color.

iOS - Color Picker

iOS - Blue Play Button

App Icon

The app icon is now customizable in a few different colors and styles. More app icons can be added in the future by the community and guidelines for what designs will be accepted will be made in the future.

iOS - App Icon Selection

iOS - Home Screen

• Closes Icon color change #596

iOS - Cinematic Item Image

When using the Cinematic item view type, currently it will use the backdrop along with the logo. The backdrop is a landscape image being used in a portrait context so most posters won't be framed nicely. So, an option was added when using the Cinematic item view type to instead use the portrait photo. This does not bring any tangible benefits, but is an option nonetheless. What should ultimately happen is the server allowing another portrait image with no language tag which would work in portrait design contexts.

Extras

Basic functionality for extras have been added. While extras technically have progress user data associated with them, I have deliberately left out indicators/played state handling as I don't see much utility in them. However, this can be added in the future if truly desired.

• Closes Extras/Special Features #453
• Closes Trailers  #350

Haptics

Basic haptics were added for basic feedback such as favoriting, setting played status, and scrubbing over chapters on the slider. Haptics can be disabled in Settings.

Indicators

Indicators on posters were added to indicate whether an item has been favorited, played, unplayed, or is in progress. The presence of these individual indicators is customizable in Settings.

• Closes Watched Overlay Missing #616
• Closes Show Played Media as Played #685

Item Pages

Items contain a lot of fields and the app retrieves a lot of items. Network calls for items will now only retrieve a minimum set of fields to increase network and memory performance. Only when an item is selected will the full information for the item be retrieved.

• Closes Seasons do not appear when first looking at a show #611, due to refactors
• Closes Dynamic Item Views #560, since the item page now just essentially requires an item object with the id field. However, there may be unforeseen work in the future.

Item Video Player Routing

Previously, item pages retrieved playback information in the background, which allowed a quick launch of video playback, however caused more issues as this was a bad UX decision. Now, when launching the video player it will retrieve playback information synchronously and present a loading view.

• Closes tvos - Start playback on item doesn't respond #622

Library Image

In a previous release, I had made it so that libraries would retrieve a random item to use as a background image. However, I had neglected the fact that library images can be set by the server. You may now decide between the random and server-set option.

• Closes Fix Library Random Images #546, due to refactor

Settings

All settings are now in new menus with a consideration for descriptions, organization, and ordering.

Other

• Closes Constrain height of BackgroundParallaxHeaderModifier #558, due to refactor
• Closes Books are NOT supported, but still shows up on home screen. #656, due to filtering

tvOS

tvOS didn't receive as much attention as iOS/iPadOS, but some features are highlighted below.

Horizontal Scrolling Lag

The latest tvOS releases have introduced significant lag when scrolling horizontally through items. I tracked this down to the .focused(...) modifier that was used to indicate that an item was, well, focused. While performance is fine among just a few items (like user buttons for signing in), lag compounds when used among many items. This issue has been tracked to SwiftUI itself, among many other performance issues.

However, knowing what item is focused is essential to views like the cinematic view on the home screen. To mostly fix this the modifier is only set if the focus state is desired, improving performance for all horizontal scrolling views except for the cinematic view. I find this acceptable since users tend to slow down on this view and not rapidly scroll.

Overall I find this saddening, as it would have been beneficial to know the focus state to set the title color between white/grey, better mirroring the UIKit (but unusable) views.

• Closes tvOS Horizontal scrolling judder #674
• Closes TVOS UI Lag and Refresh #619

Cinematic Library Background

Libraries now have a background that will change to the background of the selected item. Currently, the tab bar is still shown as a limitation of SwiftUI, but I have some ideas to work around it.

tvOS Cinematic LIbrary

Other

Just to list a few:

• video player refactor

• poster indicators

• settings refactor

• Closes tvOS - Subtitle Font #584

Future Work

Currently, this PR does break some stuff and will live in TestFlight while fixes and localizations are implemented.

My Thoughts

I worked way too long on this and wanted this done forever ago. However, life happens and gets in the way sometimes. Nothing we can do but keep moving forward.

[PangMo5]

This is a large PR. I think it would be a good idea to merge this PR first, and then post a separate PR to fix any issues or comments you have with the code.
The changes are very cool.
Thanks for the great work!

[PangMo5]

CI failed, do we need a fix?

[LePips]

Yes, but I have messed up Homebrew on my machine so it may take a while.

Regarding "reviewing this PR", that will just take place as active development instead of a direct response to this PR.

[LePips]

The build errors come from my VLCUI package which is new as of this PR. The package itself does not contain the Carthage dependencies and instead links to them through the master project. This same error does come up upon first building the project from a package cache reset during normal development but is fixed by just rebuilding a second time. So, it looks like either the CI system will need some work or I will need to find a different way to link/incorporate VLCUI.

@PangMo5 If you're able to check my branch out and build both projects on the latest Xcode, we can consider that good in the meantime.

[PangMo5]

@PangMo5 If you're able to check my branch out and build both projects on the latest Xcode, we can consider that good in the meantime.

I'm using Xcode 14.3 and there seem to be two errors.

The first is that the R.swift (named R.swift, but the internal command is swiftgen) Build Phase doesn't seem to be working correctly.

This could be an issue with my settings.

The second is https://github.com/gunterhager/UDPBroadcastConnection

carthage update --use-xcframeworks fails to build the framework with the error below.

Build Failed
	Task failed with exit code 70:
	/usr/bin/xcrun xcodebuild -project /Users/pangmo5/gitRepo/Swiftfin/Carthage/Checkouts/UDPBroadcastConnection/UDPBroadcast.xcodeproj -scheme UDPBroadcast\ tvOS -configuration Release -derivedDataPath /Users/pangmo5/Library/Caches/org.carthage.CarthageKit/DerivedData/14.3_14E222b/UDPBroadcastConnection/v5.0.3 -sdk appletvos ONLY_ACTIVE_ARCH=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= CARTHAGE=YES archive VALIDATE_WORKSPACE=NO -archivePath /var/folders/0p/wl3js02j5dncgjlf_nqd368m0000gn/T/UDPBroadcastConnection SKIP_INSTALL=YES GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=NO CLANG_ENABLE_CODE_COVERAGE=NO STRIP_INSTALLED_PRODUCT=NO (launched in /Users/pangmo5/gitRepo/Swiftfin/Carthage/Checkouts/UDPBroadcastConnection)

After that, when I build the project, I get the following error due to that library.

Module compiled with Swift 5.6.1 cannot be imported by the Swift 5.8 compiler: /Users/pangmo5/Library/Developer/Xcode/DerivedData/Swiftfin-elzjbtxrpsakmmflotepvnfjletj/Index.noindex/Build/Products/Debug-iphoneos/UDPBroadcast.framework/Modules/UDPBroadcast.swiftmodule/arm64-apple-ios.swiftmodule

If you pull your project into a new folder and build it, does it still build?

[LePips]

swiftgen has always complained since it is run on every build but that is just a warning.

The Carthage dependencies are complaining because they were built with Swift 5.6.1 and not 5.8. You will need to delete the contents of the Carthage folder and rebuild.

[PangMo5]

swiftgen has always complained since it is run on every build but that is just a warning.

The Carthage dependencies are complaining because they were built with Swift 5.6.3 and not 5.8. You will need to delete the contents of the Carthage folder and rebuild.

Ah, right, I cleared the Carthage folder and did carthage update --use-xcframeworks again and it works fine.

There's a swift 5.8 syntax issue, so I get an error, but after fixing that, it works.

I checked that both Swiftfin and Swiftfin tvOS build fine in Xcode 14.3.