[video_player] seamless looping on iOS

[nt4f04uNd]

Flutter video_player plugin doesn't use AVPlayerLooper, for setLooping which causes a small hiccup in playback

https://github.com/flutter/plugins/blob/622ba57c3960e511bdd4adabe800aef844e263fc/packages/video_player/video_player/ios/Classes/FLTVideoPlayerPlugin.m#L97-L106

Minimal reproducible:
https://github.com/nt4f04unds-archive/issues/tree/flutter_72878

[✓] Flutter (Channel master, 1.24.0-8.0.pre.374, on Microsoft Windows [Version 10.0.19041.685], locale ru-RU)
    • Flutter version 1.24.0-8.0.pre.374 at c:\dev\src\flutter
    • Framework revision 183f0e797a (4 weeks ago), 2020-11-26 19:12:28 +0100
    • Engine revision 20caf54969
    • Dart version 2.12.0 (build 2.12.0-76.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at C:\Users\danya\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    • All Android licenses accepted.

[✓] Android Studio (version 4.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 50.0.1
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[✓] VS Code (version 1.52.1)
    • VS Code at C:\Users\danya\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.17.0

[✓] Connected device (1 available)
    • Redmi Note 5 (mobile) • 40c9f14 • android-arm64 • Android 9 (API 28)

• No issues found!

[nt4f04uNd]

I tried to use AVPlayerLooper together with changing AVPlayer to AVQueuePlayer like this

_player = [[AVQueuePlayer alloc] initWithItems:@[item]];
_player.actionAtItemEnd = AVPlayerActionAtItemEndNone;
_looper = [AVPlayerLooper playerLooperWithPlayer:_player templateItem:item];

But for some reason, adding the looper makes player never load and never register the observers, so app also crashes on player disposal (though, if I comment the looper instantiation line, all works fine)
Help wanted asap

[nt4f04uNd]

could someone please take a look at this by any chance?
maybe @cyanglaz, i know you work on plugins
this is very severe for my application

[darshankawar]

@nt4f04uNd
Can you provide a complete minimal reproducible code sample ?

[nt4f04uNd]

@darshankawar added

[darshankawar]

Issue replicable on latest master on iOS.

72878_android.mov

72878.mov

flutter doctor -v

[✓] Flutter (Channel master, 1.26.0-2.0.pre.137, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-IN)
    • Flutter version 1.26.0-2.0.pre.137 at
      /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 8f5d0371af (3 days ago), 2020-12-26 11:07:36 +0330
    • Engine revision f5364860ab
    • Dart version 2.12.0 (build 2.12.0-179.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin
        code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see
      https://guides.cocoapods.org/using/getting-started.html#installation for
      instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.51.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[✓] Connected device (4 available)
    • sdk gphone x86 arm (mobile)         • emulator-5554
      • android-x86    • Android 11 (API 30) (emulator)
    • iPhone SE (2nd generation) (mobile) • 6C85835D-FBFD-4AB3-8DE8-B4FAD35E5367
      • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-0 (simulator)
    • macOS (desktop)                     • macos
      • darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)                        • chrome
      • web-javascript • Google Chrome 87.0.4280.88

! Doctor found issues in 1 category.

[nt4f04uNd]

i tried to implement this with two ways:

1. with AVPlayerLooper
2. with AVMutableComposition

AVPlayerLooper

i had no luck in achieving seamless playback with with class. i wired it up to create multiple AVPlayerItemVideoOutputs (this SO question helped me with this https://stackoverflow.com/questions/65802373/how-to-use-avplayerlooper-with-avplayeritemvideooutput), and it is rendering fine, but for some reason, there's still a hiccup in looping

i'm not sure why this happens. was testing this on my iPhone 7, so this might either be sort of Apples bug, or there's still some expensive operation i missed that causes that

implementation is here https://github.com/nt4f04uNd/plugins/tree/looper

memory usage

AVMutableComposition

here we just compose our asset to be played a lot of times (for example 500) and it seems to be looping seamlessly. however, this disables an optinos of easy turning on and off looping, because each time we change it, we have to recreate our asset

implementation is here https://github.com/nt4f04uNd/plugins/tree/composition

memory usage

as we can see, there seems to be no difference in memory usage

[rgillera]

any updates on this? thanks

[TobiasDuelli]

Are there any updates on this?

[zhouyuanbo]

Are there any updates on this?

[LeoSandbox]

Hi all - Is there any update on this issue? That would be greatly appreciated given that background loop videos are very common and the issue has been opened 3 years ago.

Here's my workaround

I am making an indie game RPG with Flutter and using background video loop for multiple screens.
The (inelegant but working) workaround I found is to add the following code that identifies if the video filename you are playing includes the string "loop" and then use AVMutableComposition like @nt4f04uNd suggested. This way I can use both looped and regular videos while using the AVMutableComposition workaround.

In my Flutter code I also set the video on loop with the same criteria so the hiccup issue is now only occurring on looped videos at the 500th iteration which the player will very likely never noticed, like so:
await _controller.setLooping(widget.path.toLowerCase().contains('loop'));

Add the following code in: Users/[YourUserName]/.pub-cache/hosted/pub.dev/video_player_avfoundation-[VersionNumber]/ios/Classes/FLTVideoPlayerPlugin.m
Inside the function: (instancetype)initWithPlayerItem, after this line: AVAsset *asset = [item asset];

  NSString *urlString = nil;
  if ([asset isKindOfClass:[AVURLAsset class]]) {
    AVURLAsset *urlAsset = (AVURLAsset *)asset;
    urlString = urlAsset.URL.lastPathComponent;
  }
  BOOL isLoopURL = (urlString && [urlString rangeOfString:@"loop" options:NSCaseInsensitiveSearch].location != NSNotFound);

  if (isLoopURL) {
    CMTimeRange timeRange = CMTimeRangeMake(CMTimeMake(0, 1), CMTimeMake(asset.duration.value, asset.duration.timescale));
    AVMutableComposition *mutableComposition = [AVMutableComposition composition];
    for (int i = 0; i < 500; i++) {
        [mutableComposition insertTimeRange:timeRange ofAsset:asset atTime:mutableComposition.duration error:nil];
    }
    item = [[AVPlayerItem alloc] initWithAsset:mutableComposition];
    asset = [item asset];
  }

[LeoSandbox]

@darshankawar @Hixie @cyanglaz @luanpotter @spydon
Taking this opportunity that some of you may be at FlutterCon, would you help put some resources to solve this long standing issue? 🙏🥺 Thank you for your time on this.

[spydon]

@darshankawar @Hixie @cyanglaz @luanpotter @spydon
Taking this opportunity that some of you may be at FlutterCon, would you help put some resources to solve this long standing issue? 🙏🥺 Thank you for your time on this.

I am not connected to any video stuff in Flutter at all, I've honestly never even used video in Flutter. 😅

[LeoSandbox]

@Hixie Can you please change the label to "found in release: 3.10.5" currently this case is tagged to "found in release: 1.26", maybe this is why it wasn't looked at for years? Thanks.

[Hixie]

Happy to add that but we don't use those labels for prioritising. We use thumbs-up emoji on the first comment, and general judgement regarding the importance of an issue. See https://github.com/flutter/flutter/wiki/Issue-hygiene#when-will-my-bug-be-fixed

[LeoSandbox]

Thanks @Hixie for your reply. I didn't know about the thumbs up ranking.

Hey @spydon @zhouyuanbo @TobiasDuelli @rgillera @nt4f04uNd - can you put a thumbs up on @nt4f04uNd top comment dated Dec 2020 ? That would help bump the issue.

[desmeit]

I have this issue as well.
Especially if the clip is very short, it jerks at the end of the video just before the loop.
This is not nice, because it is not possible to create a seamless loop.
Is there a fork @LeoSandbox?
Is it intended by the flutter team to improve this in the video player package?

I think loops should made with AVPlayerLooper.
#41156 (comment)
https://developer.apple.com/documentation/avfoundation/avplayerlooper

[nt4f04uNd]

Recently I tried debugging my looper implementation, but couldn't find the reason the hiccup still occurs.

The branch with the implementation
https://github.com/nt4f04uNd/packages/tree/video_player/looper

Although, I recorded a trace in XCode and it is evident there is a CPU workload spike in places where video goes into new loop. Perhaps this spike is the reason of the hiccup

video_player_looper.trace.zip

[LeoSandbox]

I have this issue as well. Especially if the clip is very short, it jerks at the end of the video just before the loop. This is not nice, because it is not possible to create a seamless loop. Is there a fork @LeoSandbox? Is it intended by the flutter team to improve this in the video player package?

I think loops should made with AVPlayerLooper. #41156 (comment) https://developer.apple.com/documentation/avfoundation/avplayerlooper

There is no fork that I know of. The only chance it'll get resolved is if the count of 👍 on OP post get this issue noticed, or if the flutter team dev or another contributor pick up the issue.
I found a workaround (scroll up the thread) but it makes iPhones heat up if you stay on the page with the video loop on background for a few minutes for instance. This looks due to the iPhone buffering the next (same) video over and over...