Future of the Camera Plugin (Refactor/Rework)

[bparrishMines]

As our team decides on the direction of the camera plugin, I wanted to create this issue to keep the community updated on our decisions and to catalog the highest requested features and issues with the current implementation.

Below are what I found are a combination of the main requested features and issues. Please feel free to let us know if there is anything else we should also consider.

We’ll update this issue when we have come up with a design.

Support Android Camera 16+:

flutter/plugins#1482
#19083

In order to support 16+, we will have to use the Camera API. (As opposed to Camera2). This comes at the expense of features only available on Camera2. The alternative would be to maintain both an api for 16-20 and 21+.

Also note that CameraX will be discussed at I/O and could be used in the Android implementation.

Support iOS Camera 8.0+

flutter/plugins#801
#20708

We currently use AVCapturePhotoOutput to take photos. For 8.0-10.0, we would need to use AVCaptureStillImageOutput.
We also use AVCaptureDeviceDiscoverySession to find available devices. We could use AVCaptureDevice.devicesWithMediaType: for 8.0-10.0.

Camera Orientation Control

flutter/plugins#1452
#27201
#25232

On iOS, this feature is controlled by AVCaptureVideoOrientation. On Android, the Camera API has setDisplayOrientation and there is no equivalent feature on the Camera2 API.

Video/Preview/Picture output sizes

flutter/plugins#1403
flutter/plugins#1186
flutter/plugins#1107
#15953
#20994

Currently we use preview presets. But, it would probably be best to expose all available formats.

Android Pause and Resume video recording (MediaRecorder)

flutter/plugins#1370

Zoom Control

flutter/plugins#1304

We would expose the zoom feature on both platforms. Similar to the above PR.

Flash

flutter/plugins#1047
#19845

We would expose the flash feature on both platforms. Similar to the above PR.

Automatic Exposure

flutter/plugins#1035

We would expose the AE feature on both platforms. Similar to the above PR.

Permission Handling (Camera/Audio)

flutter/plugins#904
flutter/plugins#837
flutter/plugins#823
#19670

The solution to this one is a little bit more complicated. Below are proposed solutions:

1. Remove permissions from the plugin entirely. This would require users to use a separate plugin to handle permissions. Currently, there isn’t any first party plugin that supports this, but there are a few popular external ones. (e.g. permission_handler, simple_permissions)

2. Don’t automatically handle permissions when accessing the camera, but expose the api and allow the user to ask for permissions when they want.

Auto Focus

flutter/plugins#709

We would expose the AF feature on both platforms. Similar to the above PR.

Memory Leak

#29586

Currently no solution for this, but could likely be related to not deallocating the FlutterTexture.

Provide Complex Camera Example

#19515
#18989

Expand Controls over video and image formats

flutter/plugins#1525

Direct access to image buffer when taking a photo

[jbts6]

How about support a USB Camera?

[josh-burton]

Might be worth mentioning that there is a CameraX library for Android expected to be announced at Google IO. This might simplify implementing the Android side of things quite a bit.

[bparrishMines]

@fh I believe iOS and Camera2 for Android both support external cameras automatically by referencing their ids. However, I haven't tried using one yet.

https://source.android.com/devices/camera/external-usb-cameras
https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture?language=objc

[ghost]

CameraX was announced to be min api 21 , so we still need camera instead of camera2 to support 16 -to 21

[Goolpe]

@bparrishMines there is a problem on some devices(Xiaomi, Samsung Tab...) with stopVideoRecording (camera: 0.5.2+1)
#31414
#18138
#23039
googlearchive/android-Camera2Video#46

Solution for me was

try {
  Camera.this.cameraCaptureSession.stopRepeating();
  Camera.this.cameraCaptureSession.abortCaptures();
} catch (CameraAccessException e) {
  result.error("cameraException", e.getMessage(), null);
} 

add to

private void stopVideoRecording(@NonNull final Result result) {
  if (!recordingVideo) {
    result.success(null);
    return;
  }

  try {
    recordingVideo = false;

    try {
      Camera.this.cameraCaptureSession.stopRepeating();
      Camera.this.cameraCaptureSession.abortCaptures();
    } catch (CameraAccessException e) {
      result.error("cameraException", e.getMessage(), null);
    } 
    
    mediaRecorder.stop();
    mediaRecorder.reset();
    startPreview();
    result.success(null);
  } catch (CameraAccessException | IllegalStateException e) {
    result.error("videoRecordingFailed", e.getMessage(), null);
  }
}

[PerLycke]

camerax surely sounds promising. The amount of testing needed on different devices using the current implementation with camera2 is very resource demanding. I guess we need to figure out how to bind the preview in camerax to Flutter's texture

[chunhunghan]

I have a request.
Can we consider to support new function "takePictureNoSave" return Image directly without save to disk, since CameraX has support this function to captures a new still image for in memory access.

public void takePicture (ImageCapture.OnImageCapturedListener listener)

Besides, use startImageStream need to convert YUV420
and Image_Picker also consider a public API change to provide selection, you can reference this thread
#31775 (comment)

I previously use Xamarin Forms and they will provide this after CameraX ready.
Thanks.

[bparrishMines]

I have a request.
Can we consider to support new function "takePictureNoSave" return Image directly without save to disk, since CameraX has support this function to captures a new still image for in memory access.

public void takePicture (ImageCapture.OnImageCapturedListener listener)

Besides, use startImageStream need to convert YUV420
and Image_Picker also consider a public API change to provide selection, you can reference this thread
#31775 (comment)

I previously use Xamarin Forms and they will provide this after CameraX ready.
Thanks.

@chunhunghan I added this to the issue description: Direct access to image buffer when taking a photo

[stevepsharpe]

Would it also be possible to consider adding a duration limit on video capture?

[acherkashyn]

@bparrishMines @Goolpe My team at Ebay needs ability to pause/resume video recording, and an option to increase video bitrate/quality. Setting duration limit on video recording would also be great to have, though that's not as important.
Is there any timeline you see for this camera plugin rework? I see that there's PRs open for both pause/resume, and bitrate, perhaps you could merge those and make a version of camera plugin with those changes, while this refactor/redesign is in progress?

[g-apparence]

As the flutter team is working on a complete rework of this plugin, we worked on camerawesome.
The difference with the official plugin :

• we bounded start and stop of the camera with the widget state
• we tried to abstract more things (size, scale)
• flashmodes, autofocus, switch sensor use a valuenotifier to switch automatically
• implemented e2e test with android and start them on firebase test lab on all available devices (as this cost money this is manual, would be great to find a better solution and make it work on iOS too)

As I see many people here trying to suggest pull requests here. We could join our efforts on the camerawesome repository until the camera plugin reworks release.

camera awesome repo url

[khalid-alsaleh-dev]

What about external USB cameras ?

[aytunch]

@g-apparence Camera_Awesome is awesome. I am personally waiting for Video support to be ready for both mobile platforms. How are things going for that feature?

[g-apparence]

@g-apparence Camera_Awesome is awesome. I am personally waiting for Video support to be ready for both mobile platforms. How are things going for that feature?

This is planed for our next free time between sprints/projects we allocate to this. But Yes, that's planed 👍

[wottpal]

Will the syntax of the plugin rework more or less the same compared to camerawesome, or will I have to touch all that code again and make major changes when I'm going all in with camerawesome now and want to adapt the new official plugin later?

(Sorry I don't read the whole thread and hope I'm not getting anything completely wrong)

[g-apparence]

@wottpal Cannot say as the flutter team didn't communicate on this.
Camerawesome is not official, we worked on it freely to help the community with it.

[chuongdevJC]

I got crash when run camera on ios 14. error: "-[NSNull unsignedIntegerValue]: unrecognized selector sent to instance 0x1e678dee0". Did anyone face that?

[saurabhchalke]

I wanted to know if there are any benchmark/rough ballpark stats for performance seen with the imageStream here vs Native Android?
Is there expected to be any difference in performance there?

[jdeltoft]

I'm not sure where to best ask this, but this stack overflow seems to have little traction so far. https://stackoverflow.com/questions/65901443/flutter-camera-plugin-get-path-before-recoding-stopped
I have that same question. How to get the captured video location/path. Thanks!

[ditman]

cc @mvanbeusekom see this message for an use case of why users may want to pass the path to start recording video, instead of awaiting for the recording to end.

Maybe this is something that can be added in a backwards-compatible way by adding a named optional parameter to recordVideo (?).

@jdeltoft can you please create a new issue in flutter/flutter to track this? Sounds like a legit regression. When we migrated the plugin to the federated architecture a few months back, we couldn't think of any use-case where the video path would be set beforehand.

[mvanbeusekom]

@ditman thanks for pointing this out. I think adding an optional named parameter could indeed be a nice solution. I will look into it!

[jdeltoft]

@ditman @mvanbeusekom do you still want a new flutter issue? I can create then now. Curious, though. For cases where this parameter isn't supplied, how do I know where the video was saved?

[mvanbeusekom]

@jdeltoft yes please it would be easier to track if we have a separate issue.

At the moment a temp file is generated when you call startVideoRecording and it’s returned as an XFile when calling the stopVideoRecording method.

[jdeltoft]

Thanks @mvanbeusekom , i was able to get the XFile from stopVideoRecording and use that with video player to preview it. It worked suprisingly well out of the gate, but I had to be careful with my pubspec as I was including vimeo player which has older video player dependencies still. Anyway, for my case I don't really care where this file is temporarily stored, as I just want to email out a clip if the user chooses. Thanks!

[amirjalali1]

Is there any way to pass the video path to FFmpeg as input for the RTMP stream while recording? Is there any way to stream camera-input without recording?

[wenxiangjiang]

Do you have any updates about usb camera?

[jordanhart]

Are there any thoughts or timelines for Web and Desktop support?

[bparrishMines]

Since most of the issues and PRs have been closed and I want to discourage adding additional work for this issue, I'm going to go ahead and close it. For additional feature requests or bugs, please file new issues or upvote existing ones. The only issues/PRs included in this issue that are not closed or merged are:

#27201
#25232
#15953
#19515

[ccadieux]

@bparrishMines The Readme.md links to this issue.

[bparrishMines]

Thanks, @ccadieux. I created #83722

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.