Video Player HDR Problem

[ibrahimdevs]

Hi,

Video player plugin is not supporting HDR videos. When you try to play HDR video, it plays like SDR and different color.
You can try it video_player plugin's example, simply try to play any HDR video, you will notice different color video.

Video Player is top of AVPlayer and AVPlayer should play HDR videos automatically, but it didn't. I tried everything to find a workaround but no luck. Maybe https://developer.apple.com/news/?id=rwbholxw this helps.

Thanks,

[ibrahimdevs]

Hey @jmagman , I suspect about "kCVPixelFormatType_32BGRA" in FLTVideoPlayerPlugin.m > createVideoOutputAndDisplayLink() function. But I'm not sure which value should be there.

[danagbemava-nc]

Hi @ibrahimdevs,

Can you please provide a link to any HDR videos so that I may confirm this behaviour?

Thank you

[ibrahimdevs]

Hi @danagbemava-nc ,

Thanks for your interest. I'm really stuck in this issue and very surprise that nobody even mentioned it until now.
Example HDR video link: https://drive.google.com/file/d/12plWrjPfp1Hh1G7LHHTBXaOAi8ki7m5d/view?usp=sharing

I'm getting lots of complaints about this issue from my app. If you provide any workaround, it will be great.
I moved video_player package files to locale, so I can try your workarounds.

Screenshots:

Thank you so much again,

[danagbemava-nc]

Hi @ibrahimdevs, thanks for the sample. I can reproduce this with the video_player: ^2.2.5.
I haven't looked dived deep into the source code of the plugin so I can't suggest a work around at the moment, I will however label it so that someone with more experience can offer some more insight.

screenshots

AVPlayer iOS	video_player ios

flutter doctor -v

[✓] Flutter (Channel stable, 2.5.2, on macOS 11.5.1 20G80 darwin-arm, locale en-GB)
    • Flutter version 2.5.2 at /Users/nexus/dev/sdks/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 3595343e20 (5 days ago), 2021-09-30 12:58:18 -0700
    • Engine revision 6ac856380f
    • Dart version 2.14.3

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.60.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.27.0

[✓] Connected device (3 available)
    • M2007J20CG (mobile) • 5dd3be00 • android-arm64  • Android 11 (API 30)
    • macOS (desktop)     • macos    • darwin-arm64   • macOS 11.5.1 20G80 darwin-arm
    • Chrome (web)        • chrome   • web-javascript • Google Chrome 94.0.4606.61

• No issues found!

code sample

import 'package:video_player/video_player.dart';
import 'package:flutter/material.dart';

void main() => runApp(const VideoApp());

class VideoApp extends StatefulWidget {
  const VideoApp({Key? key}) : super(key: key);

  @override
  _VideoAppState createState() => _VideoAppState();
}

class _VideoAppState extends State<VideoApp> {
  late VideoPlayerController _controller;

  @override
  void initState() {
    super.initState();
    _controller = VideoPlayerController.asset('assets/hdr_video.MOV')
      ..initialize().then((_) {
        // Ensure the first frame is shown after the video is initialized, even before the play button has been pressed.
        setState(() {});
      });
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Video Demo',
      home: Scaffold(
        body: Center(
          child: _controller.value.isInitialized
              ? AspectRatio(
                  aspectRatio: _controller.value.aspectRatio,
                  child: VideoPlayer(_controller),
                )
              : Container(),
        ),
        floatingActionButton: FloatingActionButton(
          onPressed: () {
            setState(() {
              _controller.value.isPlaying
                  ? _controller.pause()
                  : _controller.play();
            });
          },
          child: Icon(
            _controller.value.isPlaying ? Icons.pause : Icons.play_arrow,
          ),
        ),
      ),
    );
  }

  @override
  void dispose() {
    super.dispose();
    _controller.dispose();
  }
}

video asset

hdr_video.MOV.zip

[timcreatedit]

better_player displays HDR correctly for some reason, I'd be very happy if this plugin could do it too! I'll try to see if I can find something.

[timcreatedit]

I have noticed that I have way too little experience in native iOS development to understand anything 😅 So any help would be greatly appreciated! I'm happy to provide assistance wherever I can help.

The main thing that I found out is that better_player uses a UIKitView on iOS for the video player. This explains why it's behaving a little differently from this packages VideoPlayer (e.g. BackdropFilter isn't applied to better_player but is for video_player)

[ibrahimdevs]

Hi @danagbemava-nc and @stuartmorgan ,
It's been a long time, what is the latest status of this issue?

[blackox626]

I have the same problem
but I only fix the color by tonemap,so the video color look correct
I have no idea how to fix the brightness

this is my solution ，I hope it helps

- (CVPixelBufferRef)pixelBufferFormCIImage:(CIImage *)image {
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                                    @{}, kCVPixelBufferIOSurfacePropertiesKey,
                                    @YES, kCVPixelBufferCGImageCompatibilityKey,
                                    @YES, kCVPixelBufferCGBitmapContextCompatibilityKey, nil];
    
    CVPixelBufferRef pixelBufferCopy = NULL;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
                                          image.extent.size.width,
                                          image.extent.size.height,
                                          kCVPixelFormatType_32BGRA,
                                          (__bridge CFDictionaryRef) options,
                                          &pixelBufferCopy);
    
    if (status == kCVReturnSuccess) {
        CIRenderDestination *destination = [[CIRenderDestination alloc] initWithPixelBuffer:pixelBufferCopy];
        [self.mContext startTaskToRender:image toDestination:destination error:nil];
    }
    return pixelBufferCopy;
}

- (CVPixelBufferRef)copyPixelBuffer {
    CMTime outputItemTime = [_videoOutput itemTimeForHostTime:CACurrentMediaTime()];
    if ([_videoOutput hasNewPixelBufferForItemTime:outputItemTime]) {
        
        CVPixelBufferRef p = [_videoOutput copyPixelBufferForItemTime:outputItemTime itemTimeForDisplay:NULL];
        CFTypeRef colorPrimaries = CVBufferGetAttachment(p, kCVImageBufferTransferFunctionKey, NULL);
        if (colorPrimaries && CFEqual(colorPrimaries, kCVImageBufferTransferFunction_ITU_R_2100_HLG)) {
            if (@available(iOS 14.1, *)) {
                CIImage *image = [CIImage imageWithCVPixelBuffer:p options:@{kCIImageToneMapHDRtoSDR : @(YES)}];
                CVPixelBufferRef newP = [self pixelBufferFormCIImage:image];
                CVPixelBufferRelease(p);
                return newP;
            }
        }
        return p;
    } else {
        return NULL;
    }
}

[jmagman]

@blackox626 we take pull requests 🙂

[timcreatedit]

@blackox626 Yeah tonemapping would at least help this package deal with HDR video in a decent way. As far as I understand we could only achieve HDR brightness by using an embedded native view, so I think this is the way to go for now?

[blackox626]

you mean platform view？

[BurykinNikolay]

@blackox626
"
if (status == kCVReturnSuccess) {
CIRenderDestination *destination = [[CIRenderDestination alloc] initWithPixelBuffer:pixelBufferCopy];
[self.mContext startTaskToRender:image toDestination:destination error:nil];
}
"

Where did you get in self mContext?

[TMcClain5]

@blackox626 " if (status == kCVReturnSuccess) { CIRenderDestination *destination = [[CIRenderDestination alloc] initWithPixelBuffer:pixelBufferCopy]; [self.mContext startTaskToRender:image toDestination:destination error:nil]; } "

Where did you get in self mContext?

@BurykinNikolay Did you ever find out what mContext was?

[bac-huuk]

anything new?

[blackox626]

My approach is to make modifications by duplicating the plugin code and defining a new variable called mcontext to handle the pixelBuffer.

[arakcheev]

Hello everyone,

I am seeking recommendations on playing HDR videos in Flutter. Currently, I see two potential solutions:

1. Transcoding the video with tonemap and pixel format adjustments. However, this approach results in color loss and still doesn't leverage the HDR playback capabilities of modern smartphones.
2. Creating a custom UiKitView for video rendering, although this method presents other challenges as well.

I find it hard to believe that there aren't any Flutter video applications addressing this issue. So, I am curious to know how others have tackled the problem of HDR playback in Flutter. Could someone help with ideas, plz)

Thank you!

[TMcClain5]

I just found this package recently, I don't know if it addresses the issue with HDR, I have not been able to test it.
https://pub.dev/packages/media_kit

[hellohejinyu]

I just found this package recently, I don't know if it addresses the issue with HDR, I have not been able to test it. https://pub.dev/packages/media_kit

It works! #129238 (comment)

[qpalzm963]

I just found this package recently, I don't know if it addresses the issue with HDR, I have not been able to test it. https://pub.dev/packages/media_kit

It works! #129238 (comment)

What should I do if I encounter the same problem?

[hellohejinyu]

@qpalzm963 You can refer to the documentation of media_kit and use it to play videos. But you should test it more to avoid other problems. I simply tested it on an HDR video.

[qpalzm963]

@qpalzm963 You can refer to the documentation of media_kit and use it to play videos. But you should test it more to avoid other problems. I simply tested it on an HDR video.

i try video_player_media_kit example but same question myvideo is .mov

[indacurdev]

I am trying to install media_kit but I get the following error:

• Where:
  Build file 'C:\Users\pc\AppData\Local\Pub\Cache\hosted\pub.dev\media_kit_libs_android_video-1.3.6\android\build.gradle' line: 102

• What went wrong:
  A problem occurred evaluating project ':media_kit_libs_android_video'.

media_kit: arm64-v8a JAR verification failed.

[vanvixi]

@hellohejinyu
This problem also appears on media_kit (media-kit/media-kit#615)

[mertalev]

This is a big issue with the explosion of HDR content in recent years, especially when it's increasingly common for phones to shoot HDR videos.

I see that the re-triage tag was recently removed from this issue. Are there any plans underway to bring HDR to Flutter? This is the single feature I'm most looking forward to.

[benmccann]

Perhaps this would be fixed by switching to PlatformView: #86613