Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flutter Camera plugin captures video rotated 90 degrees when in Landscape on iPhone #29951

Closed
Dwarfie opened this issue Mar 26, 2019 · 25 comments
Closed

Comments

@Dwarfie
Copy link

@Dwarfie Dwarfie commented Mar 26, 2019

Using the detailed example when I capture video in Landscape on an iPhone the preview is in Portrait aspect ratio with video rotated 90 degrees.
I am uploading the video to Firebase to confirm that video is rotated 90 degrees.

I don't have a physical Android device to check if it is the same.

pubspec.yaml

version: 1.0.0+1

environment: sdk: ">=2.0.0-dev.68.0 <3.0.0"

dependencies: flutter: sdk: flutter

cupertino_icons: ^0.1.2

camera: ^0.4.2

path_provider: ^0.5.0

video_player: ^0.10.0

firebase_core: ^0.2.5

flutter doctor

[✓] Flutter (Channel unknown, v1.1.0, on Mac OS X 10.14.3 18D109, locale en-AU)

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)

[✓] Android Studio (version 3.2)

[✓] Connected device (1 available)

IMG_2941

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented Apr 3, 2019

Still no answer on this.
How do I contact the authors of the plugin ?

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented Apr 5, 2019

Hi @Dwarfie. I've been working on a fix for the rotation on iOS for the last 2 weeks. It's a complicated piece as metadata are handle differently on different platform and different video players. It looks like I found a way to fix it though and I'm creating a few PR. I'll keep you updated.

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented Apr 5, 2019

I added a fix for video orientation metadata when recording a video with the camera plugin. The PR is this one: flutter/plugins#1452
I also added a fix to the video_player. The PR is here: flutter/plugins#1451
Please don't hesitate to have a look at the PRs and leave comments or suggestions to push the review process.

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 3, 2019

Hi @quentinleguennec , thanks for the response and the fix. I'm just not sure how I test it.
Is it similar to the way I have referenced map_view in the pubspec.yaml ?

map_view:
git:
url: git://github.com/Eimji/flutter_google_map_view.git

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 3, 2019

I think I have figured that out, I copied your updated codebase to my project level and added the path: in pubspec.yaml

camera:
path: ./plugins-tengio_camera_added_rotation_metadata_ios/packages/camera/
path_provider: ^0.5.0
video_player:
path: ./plugins-tengio_improved_video_player/packages/video_player/
firebase_storage: ^2.1.0+1

I can report that I can record video in Landscape and have checked it by saving to Firebase Storage

However the view finder on the sample app is messed up for both Portrait and Landscape.

IMG_2969
IMG_2970

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 3, 2019

Hi @Dwarfie
The cleanest way to test and use this modified version of the plugin would be to reference the Github commit in your pubspec.yaml like this:

 video_player:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 9d00507431d3d11e9d7b5f3db91bf4c86614c99b
      path: packages/video_player
  camera:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 0793000a6b4e9f141dd2a18e9336a39cf98b4a07
      path: packages/camera
@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 3, 2019

Now, when I worked on solving those issue I did 4 fixes in the camera plugin and video_player plugin. And it is quite possible it will only work well only with the 4 fixes together. Here is the list of PR (only one of them has been merged so far and I have yet to get any answer from reviewers on the others):

I created a branch with those 4 fixes merged together, I suggest you try this branch first and see if it works on your side:
https://github.com/Tengio/plugins/tree/tested4you_video_player_camera_merge

You can use it by adding this in your pubspec.yaml:

 video_player:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 310a8f06c4981121a5553b0f6f3a8cdf475b8af1
      path: packages/video_player
  camera:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 310a8f06c4981121a5553b0f6f3a8cdf475b8af1
      path: packages/camera
@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 6, 2019

@quentinleguennec thanks for the response, I have changed the pubspec.yaml to reference the packages from the latest PR.

This fixes the orientation of the _thumbnailWidget that displays after you have captured video.

Unfortunately the Portrait and Landscape _cameraPreviewWidgets still display the same way as the images attached to my previous post.
This is using the example code from here: https://pub.dev/packages/camera#-example-tab-

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 7, 2019

@Dwarfie That's because the example doesn't work out of portrait. The reason behind this is that when you rotate the phone flutter rotates everything (and you can't choose what is rotated), and the preview widget ends up rotated. But since only the preview was rotated and not what you are actually recording (hopefully the actual room doesn't rotate when you rotate your phone ^^) the preview is not showing what you would expect.

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 7, 2019

One solution is to lock the app on portrait when you are on the recording page, and if you want to update the UI (button and other) listen to changes in device orientation. For this you need to use a 3rd party plugin, flutter doesn't expose enough of the accelerometer and gyroscope to do this without some native code. I did something that looks good using this plugin:
native_device_orientation
PS: if using this plugin you will want to set useSensor to true.

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 7, 2019

My solution was to have the preview in full screen in the background (not rotating) and listen to change in orientation (with native_device_orientation) to rotate the buttons individually when the users rotate the device. This is also what the default Android and iOS camera apps do (that's where I draw inspiration).

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 8, 2019

Using the current version of the camera plugin the Camera Preview fills the whole screen when the phone is in Portrait, and gets rotated when the phone is rotated.

The PR request version has a squashed down preview in Portrait and rotated 90 degrees when phone is in Landscape, but this does actually capture the video in the right orientation.

These shots are form within my app using the current released plugin, and would do the job if the video acted in the same way as your PR version

IMG_3015
IMG_3014

I may still try to save the video from the image_picker plugin which use the native camera/video controls ( well on iOS I don't have a physical android device yet )

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 9, 2019

@Dwarfie Are you using the code from packages/camera/example to do the preview? Because it doesn't display as expected when I build the example on my device and rotate it. (both on Android and iOS).

IMAGE 2019-05-09 18:16:34

IMAGE 2019-05-09 18:16:16

Could you share the code of the widget where you have the CameraPreview?

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 9, 2019

And here is the code we use (with the version of the plugin I modified and the native_device_orientation plugin) to record videos (in a way similar to the Android camera app):

import 'dart:io';

import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:native_device_orientation/native_device_orientation.dart';
import 'package:video_player/video_player.dart';

class VideoRecordingPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp]);
    return Container(
      color: Colors.black,
      child: VideoRecording(),
    );
  }
}

class VideoRecording extends StatefulWidget {
  @override
  _VideoRecordingState createState() => _VideoRecordingState();
}

class _VideoRecordingState extends BasePresenter<VideoRecording>
    with TickerProviderStateMixin<VideoRecording>, VideoControllerMixin
    implements VideoRecordingPresenterView {
  static const int startValue = 30;

  Stream<NativeDeviceOrientation> orientationChangeListener;
  int rotationQuarterTurns = 0;
  RecordingMode mode;
  String videoFilePath;
  VideoRecordingPresenter _presenter;
  List<CameraDescription> _cameras;
  CameraController _controller;
  int _selectedCamera;
  AnimationController _animationController;
  String _filePath;
  VideoPlayerController _videoController;
  VoidCallback _videoListener;

  _VideoRecordingState() {
    this._presenter = VideoRecordingPresenter(this);
  }

  @override
  void initState() {
    NativeDeviceOrientationCommunicator().orientation().then(
        (orientation) => setMountedState(() => rotationQuarterTurns = convertOrientationToQuarterTurns(orientation)));
    orientationChangeListener = NativeDeviceOrientationCommunicator().onOrientationChanged(useSensor: true)
      ..listen((NativeDeviceOrientation orientation) =>
          setMountedState(() => rotationQuarterTurns = convertOrientationToQuarterTurns(orientation)));

    mode = RecordingMode.ready;
    _selectedCamera = 0;
    _cameras = [];
    _animationController = AnimationController(vsync: this, duration: Duration(seconds: startValue));
    _animationController.addStatusListener((status) {
      if (status == AnimationStatus.completed) _presenter.onRecordingTap(true);
    });
    super.initState();
    _presenter.loadCameras();
  }

  @override
  void dispose() {
    SystemChrome.setPreferredOrientations([
      DeviceOrientation.portraitUp,
      DeviceOrientation.portraitDown,
    ]);
    _animationController?.dispose();
    _presenter?.dispose();
    _videoController?.dispose();
    _controller?.dispose();
    super.dispose();
  }

  int convertOrientationToQuarterTurns(NativeDeviceOrientation orientation) {
    int quarterTurns;
    switch (orientation) {
      case NativeDeviceOrientation.portraitUp:
      case NativeDeviceOrientation.unknown:
        quarterTurns = 0;
        break;
      case NativeDeviceOrientation.landscapeLeft:
        quarterTurns = 1;
        break;
      case NativeDeviceOrientation.portraitDown:
        quarterTurns = 2;
        break;
      case NativeDeviceOrientation.landscapeRight:
        quarterTurns = 3;
        break;
    }
    return quarterTurns;
  }

  @override
  Widget build(BuildContext context) => WillPopScope(
        onWillPop: () async {
          _presenter.onCancelRecording();
          return false;
        },
        child: Stack(children: <Widget>[
          buildMainContent(),
          buildVideoRecordingController(),
        ]),
      );

  Container buildMainContent() =>
      Container(child: _controller != null ? buildCameraOrVideoPreview() : ProgressLoader());

  VideoRecordingController buildVideoRecordingController() => VideoRecordingController(
        mode,
        _cameras,
        StepTween(begin: startValue, end: 0).animate(_animationController),
        Tween<double>(begin: 0.0, end: 1.0).animate(_animationController),
        () => _presenter.onCancelRecording(),
        () => _presenter.onSelectNextCamera(_cameras, _selectedCamera),
        () => _presenter.onRecordingTap(mode == RecordingMode.recording),
        () => _presenter.onContinue(_filePath),
        () => _presenter.onRetry(_filePath),
        _presenter.onPlay,
        rotationQuarterTurns,
      );

  Widget buildCameraOrVideoPreview() => mode == RecordingMode.preview ? buildVideoPreview() : buildCameraPreview();

  @override
  void showCameras(CameraController controller, int selectedIndex, List<CameraDescription> cameras) =>
      setMountedState(() {
        _selectedCamera = selectedIndex;
        _cameras = cameras;
        _controller = controller;
      });

  @override
  void startRecording() => setMountedState(() {
        _animationController.forward();
        mode = RecordingMode.recording;
      });

  @override
  void stopRecording() => setMountedState(() {
        _animationController
          ..stop()
          ..reset();
        mode = RecordingMode.end;
      });

  @override
  void updateFilePath(filePath) => setMountedState(() => _filePath = filePath);

  @override
  void retryRecording() => setMountedState(() => mode = RecordingMode.ready);

  @override
  void startPreview() {
    if (mode != RecordingMode.preview) {
      setMountedState(() => mode = RecordingMode.preview);
    }
    _videoListener = () {
      if (mode == RecordingMode.preview && !_videoController.value.isPlaying) {
        _videoController.removeListener(_videoListener);
        setMountedState(() => mode = RecordingMode.end);
        return;
      }
    };
    _videoController = VideoPlayerController.file(File(_filePath))
      ..setLooping(false)
      ..initialize().then((_) => refreshState()) // refresh the state here to get the correct aspect ratio of the video.
      ..addListener(_videoListener)
      ..play();
  }

  Widget buildCameraPreview() => Center(
        child: CameraPreview(_controller),
      );

  double get videoPreviewAspectRatio {
    if (_videoController.value.size == null) {
      return 1.0;
    }

    double aspectRatio = _videoController.value.aspectRatio;
    bool isDeviceInLandscape = rotationQuarterTurns == 1 || rotationQuarterTurns == 3;

    return isDeviceInLandscape ? 1.0 / aspectRatio : aspectRatio;
  }

  Widget buildVideoPreview() => Center(
        child: Container(
          child: AspectRatio(
            aspectRatio: videoPreviewAspectRatio,
            child: RotatedBox(quarterTurns: rotationQuarterTurns, child: VideoPlayer(_videoController)),
          ),
        ),
      );

  @override
  void showCancelConfirmation() =>
      showGenericAlert("upload_alert_delete_post_title", "upload_alert_delete_post_message", [
        "upload_progress_page_alert_action_cancel",
        "upload_progress_page_alert_action_delete_recording",
      ]).then((value) => _presenter.onCancelAnswered("upload_progress_page_alert_action_cancel" == value));
}
@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 9, 2019

You won't be able to copy-paste it and run it as is, but you should have all you need to adapt it for your code :)

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 9, 2019

Thanks again for the suggestions, I will try to integrate them into my code.

Nice Party Duck by the way :)

I remember now that I changed the parts of the camera example that I have used on a suggestion from Kenneth Li, and I am using a RoatedBox for the Camera Preview.

/// Display the preview from the camera (or a message if the preview is not available).
  Widget _cameraPreviewWidget() {
    if (controller == null || !controller.value.isInitialized) {
      return const Text(
        'Tap a camera',
        style: TextStyle(
          color: Colors.white,
          fontSize: 24.0,
          fontWeight: FontWeight.w900,
        ),
      );
    } else {
      return RotatedBox(
        quarterTurns: MediaQuery.of(context).orientation == Orientation.landscape ? 3 : 0,
        child: AspectRatio(
          aspectRatio: controller.value.aspectRatio,
          child: CameraPreview(controller),
        ),
      );
    }
  }
@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 10, 2019

One important change I made to the camera preview in those PR was to change the aspect ratio. Before the aspect ratio on the camera preview was returningheight/width which is the opposite of what it should be. With my changes it becomes width/height as one would expect.

Having the wrong aspect ratio will lead to a weirdly distorted image (as if it was squashed or stretched). The only way I found to know what aspect ratio to use was by checking what the orientation of the device was and invert the aspect ratio if the device is in landscape.

 double get videoPreviewAspectRatio {
    if (_videoController.value.size == null) {
      return 1.0;
    }

    double aspectRatio = _videoController.value.aspectRatio;
    bool isDeviceInLandscape = rotationQuarterTurns == 1 || rotationQuarterTurns == 3;

    return isDeviceInLandscape ? 1.0 / aspectRatio : aspectRatio;
  }

And Mr Ducky says hi ^^

EDIT: This videoPreviewAspectRatio is when displaying the video after recording, not when preview the camera feed. I suggest trying to invert the aspectRatio: controller.value.aspectRatio, in your code into aspectRatio: 1 / controller.value.aspectRatio, to see if it helps.

@nilobarp

This comment has been minimized.

Copy link

@nilobarp nilobarp commented May 13, 2019

I used ffmpeg to set proper metadata. It is an extra step and requires me to bundle ffmpeg with the app but for time being this a feasible workaround. See https://stackoverflow.com/a/56105289/1790045 for code sample.

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 14, 2019

@quentinleguennec aspectRatio: 1 / controller.value.aspectRatio, fixed the camera preview for Portrait, and I can capture and save Landscape video referencing your PR. Any idea when the PR will get released ?

@quentinleguennec

This comment has been minimized.

Copy link

@quentinleguennec quentinleguennec commented May 14, 2019

@Dwarfie Good to know it works properly :)
Do you have any other issue with the PR? (To know if I need to fix anything)
I don't know when the PR will be reviewed. Hopefully now that Google IO is over they will have more time. You can leave a comment on the PRs to tell you'd like it reviewed and merged ASAP to encourage them to look at it.

@Dwarfie

This comment has been minimized.

Copy link
Author

@Dwarfie Dwarfie commented May 15, 2019

I give Mr Ducky some credit for the fix

@aleksvujic

This comment has been minimized.

Copy link

@aleksvujic aleksvujic commented Jul 1, 2019

You said that your fix works for videos. Does it work for photos as well?

@ShubhraDeshpande

This comment has been minimized.

Copy link

@ShubhraDeshpande ShubhraDeshpande commented Jul 8, 2019

@aleksvujic It works, using RotatedBox works even for photos.

@escamoteur

This comment has been minimized.

Copy link
Contributor

@escamoteur escamoteur commented Sep 24, 2019

should be solved by flutter/plugins#1952

@escamoteur escamoteur closed this Sep 24, 2019
@tonypottera24

This comment has been minimized.

Copy link

@tonypottera24 tonypottera24 commented Jan 30, 2020

I think it is not solved, still needs to rotate manually.
camera: ^0.5.7+3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.